00010000:00010000:0.0:1787825948.298581:0:154178:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6dc94 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154178 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.298591:0:154178:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6dc94 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154178 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.298600:0:154178:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6dc94 lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 154178 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.298609:0:154178:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787825948.298618:0:154178:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6dc94 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154178 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.298625:0:154178:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6dc94 lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154178 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825948.298646:0:154178:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181d1b30380 x1874663042756096/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787825964 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787825948.298662:0:154178:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787825948.298697:0:154178:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787825948.298844:0:154178:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825948.298863:0:154178:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825948.298873:0:154178:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825948.298886:0:154178:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825948.298893:0:154178:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825948.298898:0:154178:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825948.298902:0:154178:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825948.298909:0:154178:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb9750493bbe8 00010000:00010000:0.0:1787825948.298913:0:154178:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825948.298923:0:154178:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825948.298940:0:154178:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825948.298949:0:154178:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=16408 00000080:00200000:0.0:1787825948.298955:0:154178:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825948.298971:0:154178:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=16392 00000080:00200000:0.0:1787825948.298976:0:154178:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787825948.298989:0:154178:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787825948.299050:0:154178:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6dc9b lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154178 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.299058:0:154178:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6dc9b lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154178 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.299067:0:154178:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6dc9b lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154178 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825948.299091:0:154178:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d1b30380 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154178:1874663042756224:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787825948.299168:0:154178:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c86c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787825948.304184:0:154178:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d1b30380 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154178:1874663042756224:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787825948.304215:0:154178:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6dc9b lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154178 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.304234:0:154178:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6dc9b lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154178 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.304245:0:154178:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6dc9b lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154178 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.304253:0:154178:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6dc9b lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154178 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.304263:0:154178:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6dc9b lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154178 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.304270:0:154178:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0512c00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787825948.304275:0:154178:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6dc9b lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154178 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.304283:0:154178:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6dc9b lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154178 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.304291:0:154178:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6dc9b lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 154178 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.304299:0:154178:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787825948.304308:0:154178:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6dc9b lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154178 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.304315:0:154178:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6dc9b lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154178 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825948.304336:0:154178:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181d1b30380 x1874663042756224/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787825964 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787825948.304353:0:154178:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787825948.304391:0:154178:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787825948.304489:0:154178:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825948.304509:0:154178:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825948.304520:0:154178:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825948.304532:0:154178:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825948.304550:0:154178:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825948.304555:0:154178:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825948.304559:0:154178:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825948.304565:0:154178:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb9750493bb58 00010000:00010000:0.0:1787825948.304569:0:154178:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825948.304577:0:154178:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825948.304594:0:154178:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825948.304603:0:154178:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=13 00000080:00200008:0.0:1787825948.304614:0:154178:0:(file.c:6296:ll_getattr_dentry()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1787825948.304620:0:154178:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08),name=d71a.replay-single 00000002:00010000:0.0:1787825948.304635:0:154178:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x20001a9e3:0x17e0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825948.304640:0:154178:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00010000:00010000:0.0:1787825948.304686:0:154178:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6dca2 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 154178 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.304698:0:154178:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6dca2 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 154178 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.304708:0:154178:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6dca2 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 154178 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825948.304733:0:154178:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d1b30380 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154178:1874663042756352:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787825948.304882:0:154178:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c86c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787825948.312000:0:154178:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d1b30380 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154178:1874663042756352:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787825948.312049:0:154178:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6dca2 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4616 expref: -99 pid: 154178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.312072:0:154178:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6dca2 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4616 expref: -99 pid: 154178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.312081:0:154178:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6dca2 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4616 expref: -99 pid: 154178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.312097:0:154178:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6dca2 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4616 expref: -99 pid: 154178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.312110:0:154178:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6dca2 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4616 expref: -99 pid: 154178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.312127:0:154178:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6dca2 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4616 expref: -99 pid: 154178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.312141:0:154178:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6dca2 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4616 expref: -99 pid: 154178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.312150:0:154178:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6dca2 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4616 expref: -99 pid: 154178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825948.312164:0:154178:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181d1b30380 x1874663042756352/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787825964 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787825948.312182:0:154178:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6dca2 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4616 expref: -99 pid: 154178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.312195:0:154178:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787825948.312203:0:154178:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1787825948.312242:0:154178:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x17e0:0x0], updating i_size 4096 00000080:00010000:0.0:1787825948.312252:0:154178:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) for lock 0xbef42c0221d6dca2 00000080:00010000:0.0:1787825948.312261:0:154178:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00000080:00010000:0.0:1787825948.312265:0:154178:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6dca2 from it ffffb9750493bcb8 00010000:00010000:0.0:1787825948.312269:0:154178:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6dca2 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4616 expref: -99 pid: 154178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.312279:0:154178:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6dca2 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4616 expref: -99 pid: 154178 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:0.0:1787825948.313341:0:154178:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1787825948.313351:0:154178:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825948.313377:0:154178:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825948.313522:0:154178:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000100:00100000:0.0:1787825948.339256:0:154179:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c7105500 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154179:1874663042756736:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:0.0:1787825948.339284:0:154179:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6dca9 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce465c expref: -99 pid: 154179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.339305:0:154179:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6dca9 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1b57:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce465c expref: -99 pid: 154179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.339313:0:154179:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6dca9 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1b57:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce465c expref: -99 pid: 154179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.339323:0:154179:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6dca9 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1b57:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce465c expref: -99 pid: 154179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.339333:0:154179:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6dca9 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1b57:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce465c expref: -99 pid: 154179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.339346:0:154179:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6dca9 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1b57:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce465c expref: -99 pid: 154179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.339353:0:154179:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6dca9 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1b57:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce465c expref: -99 pid: 154179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.339360:0:154179:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6dca9 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1b57:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce465c expref: -99 pid: 154179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825948.339378:0:154179:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c7105500 x1874663042756736/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/672 e 0 to 0 dl 1787825964 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787825948.339393:0:154179:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6dca9 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1b57:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce465c expref: -99 pid: 154179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.339404:0:154179:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/6999 (0 0) 00000080:00200000:0.0:1787825948.339410:0:154179:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00000002:00010000:0.0:1787825948.339475:0:154179:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200025d90:0x378:0x0]) in obj [0x200025d90:0x378:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825948.339521:0:154179:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6dcb0 lrc: 3/1,0 mode: --/CR res: [0x200025d90:0x378:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 154179 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.339529:0:154179:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6dcb0 lrc: 3/1,0 mode: --/CR res: [0x200025d90:0x378:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154179 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.339548:0:154179:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6dcb0 lrc: 3/1,0 mode: --/CR res: [0x200025d90:0x378:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154179 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825948.339568:0:154179:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d1b30380 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154179:1874663042756864:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787825948.339649:0:154179:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c86c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787825948.344515:0:154179:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d1b30380 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154179:1874663042756864:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:0.0:1787825948.344551:0:154179:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6dcb0 lrc: 4/1,0 mode: --/CR res: [0x200025d90:0x378:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce466a expref: -99 pid: 154179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.344562:0:154179:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6dcb0 lrc: 4/1,0 mode: --/PR res: [0x200025d90:0x378:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce466a expref: -99 pid: 154179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.344577:0:154179:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6dcb0 lrc: 4/1,0 mode: PR/PR res: [0x200025d90:0x378:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce466a expref: -99 pid: 154179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.344596:0:154179:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6dcb0 lrc: 4/1,0 mode: PR/PR res: [0x200025d90:0x378:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce466a expref: -99 pid: 154179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.344608:0:154179:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6dcb0 lrc: 5/2,0 mode: PR/PR res: [0x200025d90:0x378:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce466a expref: -99 pid: 154179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.344615:0:154179:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6dcb0 lrc: 5/2,0 mode: PR/PR res: [0x200025d90:0x378:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce466a expref: -99 pid: 154179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.344621:0:154179:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6dcb0 lrc: 4/1,0 mode: PR/PR res: [0x200025d90:0x378:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce466a expref: -99 pid: 154179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825948.344631:0:154179:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181d1b30380 x1874663042756864/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787825964 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787825948.344644:0:154179:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6dcb0 lrc: 3/1,0 mode: PR/PR res: [0x200025d90:0x378:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce466a expref: -99 pid: 154179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.344656:0:154179:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590089616/888 (0 0) 00010000:00010000:0.0:1787825948.344663:0:154179:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6dcb0 lrc: 3/1,0 mode: PR/PR res: [0x200025d90:0x378:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce466a expref: -99 pid: 154179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.344672:0:154179:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6dcb0 lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x378:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce466a expref: -99 pid: 154179 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000002:00010000:0.0:1787825948.344717:0:154179:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x24000178a:0x378:0x0]) in obj [0x24000178a:0x378:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825948.344808:0:154179:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6dcb7 lrc: 3/1,0 mode: --/CR res: [0x24000178a:0x378:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 154179 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.344816:0:154179:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6dcb7 lrc: 3/1,0 mode: --/CR res: [0x24000178a:0x378:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154179 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.344824:0:154179:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6dcb7 lrc: 3/1,0 mode: --/CR res: [0x24000178a:0x378:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154179 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825948.344842:0:154179:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d1b30380 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154179:1874663042756992:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787825948.344902:0:154179:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c86c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787825948.358074:0:154179:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d1b30380 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154179:1874663042756992:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:0.0:1787825948.358102:0:154179:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6dcb7 lrc: 4/1,0 mode: --/CR res: [0x24000178a:0x378:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4678 expref: -99 pid: 154179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.358114:0:154179:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6dcb7 lrc: 4/1,0 mode: --/PR res: [0x24000178a:0x378:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4678 expref: -99 pid: 154179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.358125:0:154179:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6dcb7 lrc: 4/1,0 mode: PR/PR res: [0x24000178a:0x378:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4678 expref: -99 pid: 154179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.358135:0:154179:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6dcb7 lrc: 4/1,0 mode: PR/PR res: [0x24000178a:0x378:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4678 expref: -99 pid: 154179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.358148:0:154179:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6dcb7 lrc: 5/2,0 mode: PR/PR res: [0x24000178a:0x378:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4678 expref: -99 pid: 154179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.358173:0:154179:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6dcb7 lrc: 5/2,0 mode: PR/PR res: [0x24000178a:0x378:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4678 expref: -99 pid: 154179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.358181:0:154179:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6dcb7 lrc: 4/1,0 mode: PR/PR res: [0x24000178a:0x378:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4678 expref: -99 pid: 154179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825948.358192:0:154179:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181d1b30380 x1874663042756992/t0(0) o101->lustre-MDT0001-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787825964 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787825948.358206:0:154179:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6dcb7 lrc: 3/1,0 mode: PR/PR res: [0x24000178a:0x378:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4678 expref: -99 pid: 154179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.358218:0:154179:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d0f54800 type 13 mode 4 res 9663682442/888 (0 0) 00010000:00010000:0.0:1787825948.358226:0:154179:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6dcb7 lrc: 3/1,0 mode: PR/PR res: [0x24000178a:0x378:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4678 expref: -99 pid: 154179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.358235:0:154179:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6dcb7 lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x378:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4678 expref: -99 pid: 154179 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787825948.358274:0:154179:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1b57:0x0], updating i_size 8192 00000080:00200000:0.0:1787825948.358279:0:154179:0:(namei.c:147:ll_iget()) got inode: [0x20001a9e3:0x1b57:0x0](ffff9181c8373a88): rc = 0 00000080:00010000:0.0:1787825948.358288:0:154179:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x1b57:0x0](ffff9181c8373a88) for lock 0xbef42c0221d6dca9 00000080:00010000:0.0:1787825948.358306:0:154179:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x1b57:0x0](ffff9181c8373a88) 00000080:00010000:0.0:1787825948.358310:0:154179:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6dca9 from it ffffb975048e3e08 00010000:00010000:0.0:1787825948.358314:0:154179:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6dca9 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1b57:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce465c expref: -99 pid: 154179 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.358328:0:154179:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6dca9 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1b57:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce465c expref: -99 pid: 154179 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787825948.358349:0:154179:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 3 00000080:00200000:0.0:1787825948.358357:0:154179:0:(namei.c:2328:ll_rmdir()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00010000:00010000:0.0:1787825948.358387:0:154179:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6dca9 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1b57:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ce465c expref: -99 pid: 154179 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825948.358431:0:154179:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6dca9 lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x1b57:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce465c expref: -99 pid: 154179 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825948.358437:0:154179:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787825948.358440:0:154179:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6dca9 lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x1b57:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce465c expref: -99 pid: 154179 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:0.0:1787825948.358465:0:154179:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787825948.358472:0:154179:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181d1b30380 x1874663042757248/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787825948.358491:0:154179:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d1b30380 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154179:1874663042757248:192.168.203.147@tcp:36:rmdir.0 00000100:00100000:0.0:1787825948.358618:0:154179:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c86c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787825948.363181:0:15150:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:0.0:1787825948.363201:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048632704 00000100:00100000:0.0:1787825948.363209:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048632704 00000100:00100000:0.0:1787825948.363264:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9027 00000100:00100000:0.0:1787825948.363275:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c34fb450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:14309:x1874663048632704:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825948.363293:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6dcb0 lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x378:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce466a expref: -99 pid: 154179 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825948.363366:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825948.363372:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c34fb450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:14309:x1874663048632704:12345-192.168.203.147@tcp:104: Request processed in 100us (294us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825948.363380:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9027 00000100:00100000:0.0:1787825948.369693:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048632960 00000100:00100000:0.0:1787825948.369703:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048632960 00000100:00100000:0.0:1787825948.369789:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9028 00000100:00100000:0.0:1787825948.369799:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c7107100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8414:x1874663048632960:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825948.369816:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6dcb7 lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x378:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce4678 expref: -99 pid: 154179 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825948.369884:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825948.369890:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c7107100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8414:x1874663048632960:12345-192.168.203.147@tcp:104: Request processed in 94us (306us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825948.369897:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9028 00000100:00100000:0.0:1787825948.388309:0:154179:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d1b30380 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154179:1874663042757248:192.168.203.147@tcp:36:rmdir.0 00000080:00200000:0.0:1787825948.388355:0:154179:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1b57:0x0](ffff9181c8373a88) 00000100:00100000:0.0:1787825948.521311:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048634368 00000100:00100000:0.0:1787825948.521321:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048634368 00000100:00100000:0.0:1787825948.521377:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9029 00000100:00100000:0.0:1787825948.521396:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c7105c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:41299:x1874663048634368:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825948.521413:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481000/0xbef42c0221d6dcda lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x379:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce472e expref: -99 pid: 154181 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825948.521494:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825948.521501:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c7105c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:41299:x1874663048634368:12345-192.168.203.147@tcp:104: Request processed in 116us (346us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825948.521509:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9029 00000100:00100000:0.0:1787825948.526683:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00010000:00010000:0.0:1787825948.614959:0:154183:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6dd04 lrc: 3/1,0 mode: --/CR res: [0x200025d90:0x37a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 154183 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.614967:0:154183:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6dd04 lrc: 3/1,0 mode: --/CR res: [0x200025d90:0x37a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154183 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.614975:0:154183:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6dd04 lrc: 3/1,0 mode: --/CR res: [0x200025d90:0x37a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154183 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825948.614995:0:154183:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc3100 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154183:1874663042759936:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787825948.615103:0:154183:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8780 going to sleep for 11 seconds 00000100:00100000:0.0:1787825948.618246:0:154183:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc3100 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154183:1874663042759936:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:0.0:1787825948.618282:0:154183:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6dd04 lrc: 4/1,0 mode: --/CR res: [0x200025d90:0x37a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce47f2 expref: -99 pid: 154183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.618294:0:154183:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6dd04 lrc: 4/1,0 mode: --/PR res: [0x200025d90:0x37a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce47f2 expref: -99 pid: 154183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.618307:0:154183:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6dd04 lrc: 4/1,0 mode: PR/PR res: [0x200025d90:0x37a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce47f2 expref: -99 pid: 154183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.618319:0:154183:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6dd04 lrc: 4/1,0 mode: PR/PR res: [0x200025d90:0x37a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce47f2 expref: -99 pid: 154183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.618332:0:154183:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6dd04 lrc: 5/2,0 mode: PR/PR res: [0x200025d90:0x37a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce47f2 expref: -99 pid: 154183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.618341:0:154183:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6dd04 lrc: 5/2,0 mode: PR/PR res: [0x200025d90:0x37a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce47f2 expref: -99 pid: 154183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.618348:0:154183:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6dd04 lrc: 4/1,0 mode: PR/PR res: [0x200025d90:0x37a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce47f2 expref: -99 pid: 154183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825948.618360:0:154183:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181d0fc3100 x1874663042759936/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787825964 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787825948.618379:0:154183:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6dd04 lrc: 3/1,0 mode: PR/PR res: [0x200025d90:0x37a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce47f2 expref: -99 pid: 154183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.618393:0:154183:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590089616/890 (0 0) 00010000:00010000:0.0:1787825948.618400:0:154183:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6dd04 lrc: 3/1,0 mode: PR/PR res: [0x200025d90:0x37a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce47f2 expref: -99 pid: 154183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.618410:0:154183:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6dd04 lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x37a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce47f2 expref: -99 pid: 154183 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000002:00010000:0.0:1787825948.618439:0:154183:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x24000178a:0x37a:0x0]) in obj [0x24000178a:0x37a:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825948.618528:0:154183:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6dd0b lrc: 3/1,0 mode: --/CR res: [0x24000178a:0x37a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 154183 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.618547:0:154183:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6dd0b lrc: 3/1,0 mode: --/CR res: [0x24000178a:0x37a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154183 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.618556:0:154183:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6dd0b lrc: 3/1,0 mode: --/CR res: [0x24000178a:0x37a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154183 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825948.618579:0:154183:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d1b31500 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154183:1874663042760064:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787825948.618654:0:154183:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8780 going to sleep for 11 seconds 00000100:00100000:0.0:1787825948.628401:0:154183:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d1b31500 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154183:1874663042760064:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:0.0:1787825948.628431:0:154183:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6dd0b lrc: 4/1,0 mode: --/CR res: [0x24000178a:0x37a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4800 expref: -99 pid: 154183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.628444:0:154183:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6dd0b lrc: 4/1,0 mode: --/PR res: [0x24000178a:0x37a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4800 expref: -99 pid: 154183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.628545:0:154183:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6dd0b lrc: 4/1,0 mode: PR/PR res: [0x24000178a:0x37a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4800 expref: -99 pid: 154183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.628558:0:154183:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6dd0b lrc: 4/1,0 mode: PR/PR res: [0x24000178a:0x37a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4800 expref: -99 pid: 154183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.628571:0:154183:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6dd0b lrc: 5/2,0 mode: PR/PR res: [0x24000178a:0x37a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4800 expref: -99 pid: 154183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.628580:0:154183:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6dd0b lrc: 5/2,0 mode: PR/PR res: [0x24000178a:0x37a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4800 expref: -99 pid: 154183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.628596:0:154183:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6dd0b lrc: 4/1,0 mode: PR/PR res: [0x24000178a:0x37a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4800 expref: -99 pid: 154183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825948.628607:0:154183:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181d1b31500 x1874663042760064/t0(0) o101->lustre-MDT0001-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787825964 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787825948.628625:0:154183:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6dd0b lrc: 3/1,0 mode: PR/PR res: [0x24000178a:0x37a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4800 expref: -99 pid: 154183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.628637:0:154183:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d0f54800 type 13 mode 4 res 9663682442/890 (0 0) 00010000:00010000:0.0:1787825948.628645:0:154183:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6dd0b lrc: 3/1,0 mode: PR/PR res: [0x24000178a:0x37a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4800 expref: -99 pid: 154183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.628654:0:154183:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6dd0b lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x37a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4800 expref: -99 pid: 154183 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787825948.628689:0:154183:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1b59:0x0], updating i_size 8192 00000080:00200000:0.0:1787825948.628693:0:154183:0:(namei.c:147:ll_iget()) got inode: [0x20001a9e3:0x1b59:0x0](ffff9181c837aa08): rc = 0 00000080:00010000:0.0:1787825948.628951:0:154183:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x1b59:0x0](ffff9181c837aa08) for lock 0xbef42c0221d6dcfd 00000080:00010000:0.0:1787825948.628973:0:154183:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x1b59:0x0](ffff9181c837aa08) 00000080:00010000:0.0:1787825948.628978:0:154183:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6dcfd from it ffffb9750494be08 00010000:00010000:0.0:1787825948.628982:0:154183:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7800/0xbef42c0221d6dcfd lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1b59:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce47e4 expref: -99 pid: 154183 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.628991:0:154183:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7800/0xbef42c0221d6dcfd lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1b59:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce47e4 expref: -99 pid: 154183 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787825948.630436:0:154183:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 3 00000080:00200000:0.0:1787825948.630939:0:154183:0:(namei.c:2328:ll_rmdir()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00010000:00010000:0.0:1787825948.630980:0:154183:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7800/0xbef42c0221d6dcfd lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1b59:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ce47e4 expref: -99 pid: 154183 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825948.631041:0:154183:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7800/0xbef42c0221d6dcfd lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x1b59:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce47e4 expref: -99 pid: 154183 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825948.631061:0:154183:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787825948.631065:0:154183:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7800/0xbef42c0221d6dcfd lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x1b59:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce47e4 expref: -99 pid: 154183 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:0.0:1787825948.631098:0:154183:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787825948.631111:0:154183:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181d1b31500 x1874663042760320/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787825948.631136:0:154183:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d1b31500 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154183:1874663042760320:192.168.203.147@tcp:36:rmdir.0 00000100:00100000:0.0:1787825948.631239:0:154183:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8780 going to sleep for 11 seconds 00000100:00100000:0.0:1787825948.636733:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048636032 00000100:00100000:0.0:1787825948.636768:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048636032 00000100:00100000:0.0:1787825948.636826:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9031 00000100:00100000:0.0:1787825948.636843:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c7105500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6491:x1874663048636032:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825948.636861:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6dd04 lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x37a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce47f2 expref: -99 pid: 154183 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825948.636958:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825948.636965:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c7105500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6491:x1874663048636032:12345-192.168.203.147@tcp:104: Request processed in 132us (339us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825948.636973:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9031 00000100:00100000:0.0:1787825948.646719:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048636288 00000100:00100000:0.0:1787825948.646727:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048636288 00000100:00100000:0.0:1787825948.646801:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9032 00000100:00100000:0.0:1787825948.646811:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c7106d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663048636288:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825948.646827:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6dd0b lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x37a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce4800 expref: -99 pid: 154183 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825948.646913:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825948.646920:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c7106d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663048636288:12345-192.168.203.147@tcp:104: Request processed in 110us (312us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825948.646927:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9032 00000100:00100000:0.0:1787825948.661938:0:154183:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d1b31500 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154183:1874663042760320:192.168.203.147@tcp:36:rmdir.0 00000080:00200000:0.0:1787825948.661986:0:154183:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1b59:0x0](ffff9181c837aa08) 00000100:00100000:0.0:1787825948.793388:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048637696 00000100:00100000:0.0:1787825948.793403:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048637696 00000100:00100000:0.0:1787825948.793477:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9033 00000100:00100000:0.0:1787825948.793487:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c7107480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:7728:x1874663048637696:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825948.793505:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6dd2e lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x37b:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce48b6 expref: -99 pid: 154185 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825948.793605:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825948.793611:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c7107480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:7728:x1874663048637696:12345-192.168.203.147@tcp:104: Request processed in 126us (344us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825948.793618:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9033 00000100:00100000:0.0:1787825948.798733:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825948.908400:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048639360 00000100:00100000:0.0:1787825948.908410:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048639360 00000100:00100000:0.0:1787825948.908468:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9035 00000100:00100000:0.0:1787825948.908478:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c7105c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6492:x1874663048639360:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825948.908494:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7e00/0xbef42c0221d6dd58 lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x37c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce497a expref: -99 pid: 154187 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825948.908600:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825948.908608:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c7105c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6492:x1874663048639360:12345-192.168.203.147@tcp:104: Request processed in 131us (525us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825948.909111:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9035 00000100:00100000:0.0:1787825948.915318:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000080:00200000:0.0:1787825948.939304:0:154188:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825948.939333:0:154188:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825948.939351:0:154188:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825948.939368:0:154188:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825948.939377:0:154188:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825948.939386:0:154188:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825948.939389:0:154188:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825948.939396:0:154188:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97504973b58 00010000:00010000:0.0:1787825948.939400:0:154188:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825948.939409:0:154188:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825948.939422:0:154188:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825948.939430:0:154188:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=29 00000080:00200000:0.0:1787825948.939436:0:154188:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825948.939448:0:154188:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=13 00000080:00200000:0.0:1787825948.939451:0:154188:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787825948.939463:0:154188:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787825948.939505:0:154188:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6dd66 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154188 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.939513:0:154188:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6dd66 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154188 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825948.939522:0:154188:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6dd66 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154188 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825948.939561:0:154188:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d1b33100 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154188:1874663042763776:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787825948.939684:0:154188:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8900 going to sleep for 11 seconds 00000080:00010000:0.0:1787825949.011822:0:154189:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x1b5c:0x0](ffff9181c8389988) 00000080:00010000:0.0:1787825949.011828:0:154189:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6dd7b from it ffffb9750497be08 00010000:00010000:0.0:1787825949.011833:0:154189:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6dd7b lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1b5c:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4a30 expref: -99 pid: 154189 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.011842:0:154189:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6dd7b lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1b5c:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4a30 expref: -99 pid: 154189 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787825949.011864:0:154189:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 3 00000080:00200000:0.0:1787825949.011872:0:154189:0:(namei.c:2328:ll_rmdir()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00010000:00010000:0.0:1787825949.011903:0:154189:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6dd7b lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1b5c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ce4a30 expref: -99 pid: 154189 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825949.011950:0:154189:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6dd7b lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x1b5c:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce4a30 expref: -99 pid: 154189 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825949.011963:0:154189:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787825949.011966:0:154189:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6dd7b lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x1b5c:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce4a30 expref: -99 pid: 154189 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:0.0:1787825949.012009:0:154189:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787825949.012017:0:154189:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181d1b30000 x1874663042764928/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787825949.012040:0:154189:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d1b30000 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154189:1874663042764928:192.168.203.147@tcp:36:rmdir.0 00000100:00100000:0.0:1787825949.012123:0:154189:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8a80 going to sleep for 11 seconds 00000100:00100000:0.0:1787825949.019808:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787825949.019953:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048641024 00000100:00100000:1.0:1787825949.020020:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9037 00000100:00100000:1.0:1787825949.020030:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181d1b32d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6490:x1874663048641024:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787825949.020047:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6dd82 lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x37d:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce4a3e expref: -99 pid: 154189 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825949.020126:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787825949.020133:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181d1b32d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6490:x1874663048641024:12345-192.168.203.147@tcp:104: Request processed in 105us (334us total) trans 0 rc 0/0 00000100:00100000:1.0:1787825949.020139:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9037 00000100:00100000:1.0:1787825949.025278:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787825949.025311:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048641280 00000100:00100000:1.0:1787825949.025320:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048641280 00000100:00100000:1.0:1787825949.025366:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9038 00000100:00100000:1.0:1787825949.025375:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c7106d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663048641280:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787825949.025393:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6dd89 lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x37d:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce4a4c expref: -99 pid: 154189 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825949.025469:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787825949.025477:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c7106d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663048641280:12345-192.168.203.147@tcp:104: Request processed in 103us (210us total) trans 0 rc 0/0 00000100:00100000:1.0:1787825949.025484:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9038 00000100:00100000:0.0:1787825949.154097:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787825949.154191:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048642688 00000100:00100000:1.0:1787825949.154201:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048642688 00000100:00100000:1.0:1787825949.154259:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9039 00000100:00100000:1.0:1787825949.154268:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181d1b33800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:14309:x1874663048642688:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787825949.154284:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481000/0xbef42c0221d6ddac lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x37e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce4b02 expref: -99 pid: 154191 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825949.154517:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787825949.154525:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181d1b33800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:14309:x1874663048642688:12345-192.168.203.147@tcp:104: Request processed in 258us (438us total) trans 0 rc 0/0 00000100:00100000:1.0:1787825949.154543:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9039 00000100:00100000:0.0:1787825949.160916:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787825949.161051:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048642944 00000100:00100000:1.0:1787825949.161061:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048642944 00000100:00100000:1.0:1787825949.161117:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9040 00000100:00100000:1.0:1787825949.161127:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181d1b32a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663048642944:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787825949.161144:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480600/0xbef42c0221d6ddb3 lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x37e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce4b10 expref: -99 pid: 154191 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825949.161221:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787825949.161227:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181d1b32a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663048642944:12345-192.168.203.147@tcp:104: Request processed in 102us (322us total) trans 0 rc 0/0 00000100:00100000:1.0:1787825949.161234:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9040 00000080:00200000:0.0:1787825949.232680:0:154193:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825949.232713:0:154193:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825949.232732:0:154193:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825949.232997:0:154193:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825949.233007:0:154193:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825949.233016:0:154193:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825949.233020:0:154193:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825949.233029:0:154193:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97504883c20 00010000:00010000:0.0:1787825949.233033:0:154193:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825949.233042:0:154193:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825949.233058:0:154193:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825949.233065:0:154193:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=24 00000080:00200000:0.0:1787825949.233071:0:154193:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825949.233083:0:154193:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=0 00000080:00200000:0.0:1787825949.233087:0:154193:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787825949.233101:0:154193:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787825949.233144:0:154193:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6ddcf lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154193 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.233231:0:154193:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6ddcf lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154193 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.233243:0:154193:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6ddcf lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154193 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825949.233267:0:154193:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d1b31f80 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154193:1874663042767488:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787825949.233399:0:154193:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8a80 going to sleep for 11 seconds 00000100:00100000:0.0:1787825949.264469:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787825949.264890:0:15151:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:1.0:1787825949.264909:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048644352 00000100:00100000:1.0:1787825949.264917:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048644352 00000100:00100000:1.0:1787825949.265180:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9041 00000100:00100000:1.0:1787825949.265192:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c440a850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6492:x1874663048644352:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787825949.265211:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6200/0xbef42c0221d6ddd6 lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x37f:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce4bc6 expref: -99 pid: 154193 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825949.265339:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787825949.265346:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c440a850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6492:x1874663048644352:12345-192.168.203.147@tcp:104: Request processed in 157us (887us total) trans 0 rc 0/0 00000100:00100000:1.0:1787825949.265354:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9041 00000100:00100000:0.0:1787825949.272505:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787825949.273199:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048644608 00000100:00100000:1.0:1787825949.273209:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048644608 00000100:00100000:1.0:1787825949.273265:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9042 00000100:00100000:1.0:1787825949.273274:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181d1b30700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663048644608:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787825949.273292:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6dddd lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x37f:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce4bd4 expref: -99 pid: 154193 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825949.273367:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787825949.273375:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181d1b30700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663048644608:12345-192.168.203.147@tcp:104: Request processed in 102us (880us total) trans 0 rc 0/0 00000100:00100000:1.0:1787825949.273381:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9042 00000080:00200000:0.0:1787825949.307074:0:154194:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825949.307107:0:154194:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825949.307128:0:154194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825949.307221:0:154194:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825949.307230:0:154194:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825949.307238:0:154194:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825949.307242:0:154194:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825949.307249:0:154194:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb9750493bb58 00010000:00010000:0.0:1787825949.307253:0:154194:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825949.307263:0:154194:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825949.307281:0:154194:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825949.307289:0:154194:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=29 00000080:00200000:0.0:1787825949.307295:0:154194:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825949.307306:0:154194:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=13 00000080:00200000:0.0:1787825949.307310:0:154194:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787825949.307323:0:154194:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787825949.307373:0:154194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6dde4 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.307382:0:154194:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6dde4 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.307392:0:154194:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6dde4 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154194 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825949.307422:0:154194:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d1b32d80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154194:1874663042768384:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787825949.307599:0:154194:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8900 going to sleep for 11 seconds 00000100:00100000:0.0:1787825949.313275:0:154194:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d1b32d80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154194:1874663042768384:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787825949.313296:0:154194:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6dde4 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.313320:0:154194:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6dde4 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.313326:0:154194:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6dde4 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.313333:0:154194:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6dde4 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.313341:0:154194:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6dde4 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.313347:0:154194:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0512000 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787825949.313352:0:154194:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6dde4 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.313359:0:154194:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6dde4 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.313367:0:154194:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6dde4 lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 154194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.313373:0:154194:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787825949.313379:0:154194:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6dde4 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.313386:0:154194:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6dde4 lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154194 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825949.313401:0:154194:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181d1b32d80 x1874663042768384/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787825965 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787825949.313415:0:154194:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787825949.313442:0:154194:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787825949.313508:0:154194:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825949.313523:0:154194:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825949.313544:0:154194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825949.313557:0:154194:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825949.313564:0:154194:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825949.313569:0:154194:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825949.313573:0:154194:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825949.313589:0:154194:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb9750493bbe8 00010000:00010000:0.0:1787825949.313593:0:154194:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825949.313602:0:154194:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825949.313615:0:154194:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825949.313623:0:154194:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=16408 00000080:00200000:0.0:1787825949.313628:0:154194:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825949.313641:0:154194:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=16392 00000080:00200000:0.0:1787825949.313645:0:154194:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787825949.313656:0:154194:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787825949.313690:0:154194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6ddeb lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.313708:0:154194:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6ddeb lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.313717:0:154194:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6ddeb lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154194 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825949.313736:0:154194:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d1b32d80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154194:1874663042768512:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787825949.313828:0:154194:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8900 going to sleep for 11 seconds 00000100:00100000:0.0:1787825949.320133:0:154194:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d1b32d80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154194:1874663042768512:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787825949.320171:0:154194:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6ddeb lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.320184:0:154194:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6ddeb lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.320191:0:154194:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6ddeb lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.320198:0:154194:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6ddeb lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.320207:0:154194:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6ddeb lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.320214:0:154194:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0513200 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787825949.320219:0:154194:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6ddeb lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.320227:0:154194:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6ddeb lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.320234:0:154194:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6ddeb lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 154194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.320242:0:154194:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787825949.320248:0:154194:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6ddeb lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.320255:0:154194:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6ddeb lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154194 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825949.320272:0:154194:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181d1b32d80 x1874663042768512/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787825965 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787825949.320287:0:154194:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787825949.320312:0:154194:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787825949.320390:0:154194:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825949.320405:0:154194:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825949.320414:0:154194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825949.320426:0:154194:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825949.320439:0:154194:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825949.320444:0:154194:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825949.320447:0:154194:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825949.320453:0:154194:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb9750493bb58 00010000:00010000:0.0:1787825949.320457:0:154194:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825949.320464:0:154194:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825949.320479:0:154194:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825949.320485:0:154194:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=13 00000080:00200008:0.0:1787825949.320493:0:154194:0:(file.c:6296:ll_getattr_dentry()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1787825949.320498:0:154194:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08),name=d71a.replay-single 00000002:00010000:0.0:1787825949.320509:0:154194:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x20001a9e3:0x17e0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825949.320514:0:154194:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00010000:00010000:0.0:1787825949.320562:0:154194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6ddf2 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 154194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.320569:0:154194:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6ddf2 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 154194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.320579:0:154194:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6ddf2 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 154194 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825949.320611:0:154194:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d1b32d80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154194:1874663042768640:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787825949.320686:0:154194:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8900 going to sleep for 11 seconds 00000100:00100000:0.0:1787825949.326638:0:154194:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d1b32d80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154194:1874663042768640:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787825949.326664:0:154194:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6ddf2 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4c36 expref: -99 pid: 154194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.326681:0:154194:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6ddf2 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4c36 expref: -99 pid: 154194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.326688:0:154194:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6ddf2 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4c36 expref: -99 pid: 154194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.326700:0:154194:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6ddf2 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4c36 expref: -99 pid: 154194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.326710:0:154194:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6ddf2 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4c36 expref: -99 pid: 154194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.326722:0:154194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6ddf2 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4c36 expref: -99 pid: 154194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.326729:0:154194:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6ddf2 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4c36 expref: -99 pid: 154194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.326736:0:154194:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6ddf2 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4c36 expref: -99 pid: 154194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825949.326784:0:154194:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181d1b32d80 x1874663042768640/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787825965 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787825949.326798:0:154194:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6ddf2 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4c36 expref: -99 pid: 154194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.326809:0:154194:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787825949.326816:0:154194:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1787825949.326846:0:154194:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x17e0:0x0], updating i_size 4096 00000080:00010000:0.0:1787825949.326853:0:154194:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) for lock 0xbef42c0221d6ddf2 00000080:00010000:0.0:1787825949.326861:0:154194:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00000080:00010000:0.0:1787825949.326864:0:154194:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6ddf2 from it ffffb9750493bcb8 00010000:00010000:0.0:1787825949.326867:0:154194:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6ddf2 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4c36 expref: -99 pid: 154194 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.326877:0:154194:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6ddf2 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4c36 expref: -99 pid: 154194 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:0.0:1787825949.327457:0:154194:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1787825949.327464:0:154194:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825949.327481:0:154194:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825949.327492:0:154194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825949.327503:0:154194:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825949.327512:0:154194:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825949.327517:0:154194:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825949.327520:0:154194:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825949.327525:0:154194:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb9750493bcb8 00010000:00010000:0.0:1787825949.327541:0:154194:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825949.327549:0:154194:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825949.327670:0:154194:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825949.327680:0:154194:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825949.327688:0:154194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825949.327697:0:154194:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825949.327704:0:154194:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825949.327709:0:154194:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825949.327712:0:154194:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825949.327715:0:154194:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb9750493bbf8 00010000:00010000:0.0:1787825949.327718:0:154194:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825949.327726:0:154194:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825949.327734:0:154194:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825949.327741:0:154194:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=267 00000080:00200000:0.0:1787825949.327800:0:154194:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 44 00000080:00200008:0.0:1787825949.327806:0:154194:0:(file.c:1090:ll_file_open()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00000080:00200080:0.0:1787825949.327826:0:154194:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) cmd=c03066f0 arg=7ffe141e47d0 00000080:00200000:0.0:1787825949.327838:0:154194:0:(dir.c:451:ll_dir_setdirstripe()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) name=test stripe_offset=0 stripe_count=2, hash_type=0 00010000:00010000:0.0:1787825949.327863:0:154194:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6ddf2 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ce4c36 expref: -99 pid: 154194 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825949.327877:0:154194:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6800/0xbef42c0221d64b3f lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cb9ffd expref: -99 pid: 152202 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825949.327917:0:154194:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6ddf2 lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce4c36 expref: -99 pid: 154194 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825949.327924:0:154194:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787825949.327927:0:154194:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6ddf2 lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce4c36 expref: -99 pid: 154194 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:0.0:1787825949.327946:0:154194:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787825949.327955:0:154194:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181d1b32d80 x1874663042768896/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 552/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787825949.327977:0:154194:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d1b32d80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154194:1874663042768896:192.168.203.147@tcp:36:lfs.0 00000100:00100000:0.0:1787825949.328043:0:154194:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8900 going to sleep for 11 seconds 00000100:00100000:0.0:1787825949.343958:0:154194:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d1b32d80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154194:1874663042768896:192.168.203.147@tcp:36:lfs.0 00000080:00200000:0.0:1787825949.344007:0:154194:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x20001a9e3:0x1b5f:0x0](ffff9181c8384b08) 00000080:00200000:0.0:1787825949.344014:0:154194:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1b5f:0x0], updating i_size 4096 00000080:00200008:0.0:1787825949.344050:0:154194:0:(file.c:479:ll_file_release()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00010000:00010000:0.0:1787825949.344080:0:154194:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513000/0xbef42c0221d64bee lrc: 2/0,0 mode: CR/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cba2b9 expref: -99 pid: 152210 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825949.353269:0:154195:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825949.353299:0:154195:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825949.353317:0:154195:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825949.353334:0:154195:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825949.353343:0:154195:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825949.353352:0:154195:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825949.353355:0:154195:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825949.353362:0:154195:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97504923c20 00010000:00010000:0.0:1787825949.353366:0:154195:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825949.353377:0:154195:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825949.353392:0:154195:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825949.353400:0:154195:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=24 00000080:00200000:0.0:1787825949.353406:0:154195:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825949.353418:0:154195:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=0 00000080:00200000:0.0:1787825949.353422:0:154195:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787825949.353435:0:154195:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787825949.353480:0:154195:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6ddf9 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154195 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.353488:0:154195:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6ddf9 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154195 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.353500:0:154195:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6ddf9 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154195 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825949.353523:0:154195:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d1b33800 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154195:1874663042769024:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787825949.353630:0:154195:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8900 going to sleep for 11 seconds 00000100:00100000:0.0:1787825949.357140:0:154195:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d1b33800 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154195:1874663042769024:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:0.0:1787825949.357172:0:154195:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6ddf9 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4c7c expref: -99 pid: 154195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.357191:0:154195:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6ddf9 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1b5f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4c7c expref: -99 pid: 154195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.357200:0:154195:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6ddf9 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1b5f:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4c7c expref: -99 pid: 154195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.357210:0:154195:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6ddf9 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1b5f:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4c7c expref: -99 pid: 154195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.357220:0:154195:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6ddf9 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1b5f:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4c7c expref: -99 pid: 154195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.357230:0:154195:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6ddf9 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1b5f:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4c7c expref: -99 pid: 154195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.357238:0:154195:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6ddf9 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1b5f:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4c7c expref: -99 pid: 154195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.357252:0:154195:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6ddf9 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1b5f:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4c7c expref: -99 pid: 154195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825949.357262:0:154195:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181d1b33800 x1874663042769024/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/672 e 0 to 0 dl 1787825965 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787825949.357276:0:154195:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6ddf9 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1b5f:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4c7c expref: -99 pid: 154195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.357285:0:154195:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/7007 (0 0) 00000080:00200000:0.0:1787825949.357291:0:154195:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00000002:00010000:0.0:1787825949.357344:0:154195:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200025d90:0x380:0x0]) in obj [0x200025d90:0x380:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825949.357380:0:154195:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6de00 lrc: 3/1,0 mode: --/CR res: [0x200025d90:0x380:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 154195 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.357387:0:154195:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6de00 lrc: 3/1,0 mode: --/CR res: [0x200025d90:0x380:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154195 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.357395:0:154195:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6de00 lrc: 3/1,0 mode: --/CR res: [0x200025d90:0x380:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154195 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825949.357410:0:154195:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d1b32a00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154195:1874663042769152:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787825949.357472:0:154195:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8900 going to sleep for 11 seconds 00000100:00100000:0.0:1787825949.360615:0:154195:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d1b32a00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154195:1874663042769152:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:0.0:1787825949.360644:0:154195:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6de00 lrc: 4/1,0 mode: --/CR res: [0x200025d90:0x380:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4c8a expref: -99 pid: 154195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.360661:0:154195:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6de00 lrc: 4/1,0 mode: --/PR res: [0x200025d90:0x380:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4c8a expref: -99 pid: 154195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.360673:0:154195:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6de00 lrc: 4/1,0 mode: PR/PR res: [0x200025d90:0x380:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4c8a expref: -99 pid: 154195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.360684:0:154195:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6de00 lrc: 4/1,0 mode: PR/PR res: [0x200025d90:0x380:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4c8a expref: -99 pid: 154195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.360708:0:154195:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6de00 lrc: 5/2,0 mode: PR/PR res: [0x200025d90:0x380:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4c8a expref: -99 pid: 154195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.360717:0:154195:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6de00 lrc: 5/2,0 mode: PR/PR res: [0x200025d90:0x380:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4c8a expref: -99 pid: 154195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.360724:0:154195:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6de00 lrc: 4/1,0 mode: PR/PR res: [0x200025d90:0x380:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4c8a expref: -99 pid: 154195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825949.360736:0:154195:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181d1b32a00 x1874663042769152/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787825965 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787825949.360771:0:154195:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6de00 lrc: 3/1,0 mode: PR/PR res: [0x200025d90:0x380:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4c8a expref: -99 pid: 154195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.360784:0:154195:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590089616/896 (0 0) 00010000:00010000:0.0:1787825949.360792:0:154195:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6de00 lrc: 3/1,0 mode: PR/PR res: [0x200025d90:0x380:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4c8a expref: -99 pid: 154195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.360806:0:154195:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6de00 lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x380:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4c8a expref: -99 pid: 154195 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000002:00010000:0.0:1787825949.360831:0:154195:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x24000178a:0x380:0x0]) in obj [0x24000178a:0x380:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825949.360876:0:154195:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6de07 lrc: 3/1,0 mode: --/CR res: [0x24000178a:0x380:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 154195 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.360884:0:154195:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6de07 lrc: 3/1,0 mode: --/CR res: [0x24000178a:0x380:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154195 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.360894:0:154195:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6de07 lrc: 3/1,0 mode: --/CR res: [0x24000178a:0x380:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154195 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825949.360936:0:154195:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d1b32a00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154195:1874663042769280:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787825949.360999:0:154195:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8900 going to sleep for 11 seconds 00000100:00100000:0.0:1787825949.368863:0:154195:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d1b32a00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154195:1874663042769280:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:0.0:1787825949.368890:0:154195:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6de07 lrc: 4/1,0 mode: --/CR res: [0x24000178a:0x380:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4c98 expref: -99 pid: 154195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.368902:0:154195:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6de07 lrc: 4/1,0 mode: --/PR res: [0x24000178a:0x380:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4c98 expref: -99 pid: 154195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.368913:0:154195:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6de07 lrc: 4/1,0 mode: PR/PR res: [0x24000178a:0x380:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4c98 expref: -99 pid: 154195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.368923:0:154195:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6de07 lrc: 4/1,0 mode: PR/PR res: [0x24000178a:0x380:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4c98 expref: -99 pid: 154195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.368941:0:154195:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6de07 lrc: 5/2,0 mode: PR/PR res: [0x24000178a:0x380:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4c98 expref: -99 pid: 154195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.368950:0:154195:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6de07 lrc: 5/2,0 mode: PR/PR res: [0x24000178a:0x380:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4c98 expref: -99 pid: 154195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.368957:0:154195:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6de07 lrc: 4/1,0 mode: PR/PR res: [0x24000178a:0x380:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4c98 expref: -99 pid: 154195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825949.368968:0:154195:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181d1b32a00 x1874663042769280/t0(0) o101->lustre-MDT0001-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787825965 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787825949.368982:0:154195:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6de07 lrc: 3/1,0 mode: PR/PR res: [0x24000178a:0x380:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4c98 expref: -99 pid: 154195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.368995:0:154195:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d0f54800 type 13 mode 4 res 9663682442/896 (0 0) 00010000:00010000:0.0:1787825949.369004:0:154195:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6de07 lrc: 3/1,0 mode: PR/PR res: [0x24000178a:0x380:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4c98 expref: -99 pid: 154195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.369013:0:154195:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6de07 lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x380:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4c98 expref: -99 pid: 154195 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787825949.369048:0:154195:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1b5f:0x0], updating i_size 8192 00000080:00200000:0.0:1787825949.369052:0:154195:0:(namei.c:147:ll_iget()) got inode: [0x20001a9e3:0x1b5f:0x0](ffff9181c8384b08): rc = 0 00000080:00010000:0.0:1787825949.369061:0:154195:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x1b5f:0x0](ffff9181c8384b08) for lock 0xbef42c0221d6ddf9 00000080:00010000:0.0:1787825949.369078:0:154195:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x1b5f:0x0](ffff9181c8384b08) 00000080:00010000:0.0:1787825949.369082:0:154195:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6ddf9 from it ffffb97504923e08 00010000:00010000:0.0:1787825949.369086:0:154195:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6ddf9 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1b5f:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4c7c expref: -99 pid: 154195 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.369094:0:154195:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6ddf9 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1b5f:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4c7c expref: -99 pid: 154195 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787825949.369114:0:154195:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 3 00000080:00200000:0.0:1787825949.369122:0:154195:0:(namei.c:2328:ll_rmdir()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00010000:00010000:0.0:1787825949.369163:0:154195:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6ddf9 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1b5f:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ce4c7c expref: -99 pid: 154195 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825949.369210:0:154195:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6ddf9 lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x1b5f:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce4c7c expref: -99 pid: 154195 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825949.369217:0:154195:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787825949.369220:0:154195:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6ddf9 lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x1b5f:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce4c7c expref: -99 pid: 154195 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:0.0:1787825949.369244:0:154195:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787825949.369251:0:154195:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181d1b33800 x1874663042769536/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787825949.369271:0:154195:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d1b33800 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154195:1874663042769536:192.168.203.147@tcp:36:rmdir.0 00000100:00100000:0.0:1787825949.369343:0:154195:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8900 going to sleep for 11 seconds 00000100:00100000:1.0:1787825949.372649:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825949.373431:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048646016 00000100:00100000:0.0:1787825949.373444:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048646016 00000100:00100000:0.0:1787825949.373552:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9043 00000100:00100000:0.0:1787825949.373563:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c7105c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6490:x1874663048646016:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825949.373595:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6de00 lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x380:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce4c8a expref: -99 pid: 154195 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825949.373690:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825949.374443:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c7105c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6490:x1874663048646016:12345-192.168.203.147@tcp:104: Request processed in 136us (1059us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825949.374452:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9043 00000100:00100000:1.0:1787825949.379459:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825949.379551:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048646272 00000100:00100000:0.0:1787825949.379560:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048646272 00000100:00100000:0.0:1787825949.379631:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9044 00000100:00100000:0.0:1787825949.379642:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c7106d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663048646272:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825949.379659:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6de07 lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x380:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce4c98 expref: -99 pid: 154195 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825949.379741:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825949.379770:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c7106d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663048646272:12345-192.168.203.147@tcp:104: Request processed in 130us (323us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825949.379778:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9044 00000100:00100000:0.0:1787825949.396339:0:154195:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d1b33800 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154195:1874663042769536:192.168.203.147@tcp:36:rmdir.0 00000080:00200000:0.0:1787825949.396387:0:154195:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1b5f:0x0](ffff9181c8384b08) 00000080:00200000:0.0:1787825949.408603:0:154196:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825949.408634:0:154196:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825949.408652:0:154196:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825949.408668:0:154196:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825949.408676:0:154196:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825949.408684:0:154196:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825949.408688:0:154196:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825949.408704:0:154196:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb9750498bb58 00010000:00010000:0.0:1787825949.408708:0:154196:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825949.408717:0:154196:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825949.408733:0:154196:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825949.408740:0:154196:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=29 00000080:00200000:0.0:1787825949.409282:0:154196:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825949.409297:0:154196:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=13 00000080:00200000:0.0:1787825949.409302:0:154196:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787825949.409316:0:154196:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787825949.409359:0:154196:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6de0e lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154196 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.409367:0:154196:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6de0e lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154196 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.409377:0:154196:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6de0e lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154196 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825949.409401:0:154196:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d1b32a00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154196:1874663042769920:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787825949.409540:0:154196:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8900 going to sleep for 11 seconds 00000100:00100000:0.0:1787825949.413329:0:154196:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d1b32a00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154196:1874663042769920:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787825949.413352:0:154196:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6de0e lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154196 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.413364:0:154196:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6de0e lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154196 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.413371:0:154196:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6de0e lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154196 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.413378:0:154196:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6de0e lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154196 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.413386:0:154196:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6de0e lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154196 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.413393:0:154196:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0513400 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787825949.413398:0:154196:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6de0e lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154196 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.413404:0:154196:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6de0e lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154196 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.413411:0:154196:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6de0e lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 154196 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.413417:0:154196:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787825949.413423:0:154196:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6de0e lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154196 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.413430:0:154196:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6de0e lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154196 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825949.413446:0:154196:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181d1b32a00 x1874663042769920/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787825965 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787825949.413459:0:154196:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787825949.413484:0:154196:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787825949.413568:0:154196:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825949.413589:0:154196:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825949.413598:0:154196:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825949.413608:0:154196:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825949.413614:0:154196:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825949.413619:0:154196:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825949.413623:0:154196:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825949.413627:0:154196:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb9750498bbe8 00010000:00010000:0.0:1787825949.413631:0:154196:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825949.413639:0:154196:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825949.413652:0:154196:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825949.413659:0:154196:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=16408 00000080:00200000:0.0:1787825949.413665:0:154196:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825949.413678:0:154196:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=16392 00000080:00200000:0.0:1787825949.413682:0:154196:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787825949.413693:0:154196:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787825949.413784:0:154196:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6de15 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154196 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.413792:0:154196:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6de15 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154196 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.413802:0:154196:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6de15 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154196 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825949.413821:0:154196:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d1b32a00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154196:1874663042770048:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787825949.413879:0:154196:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8900 going to sleep for 11 seconds 00000100:00100000:0.0:1787825949.417169:0:154196:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d1b32a00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154196:1874663042770048:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787825949.417195:0:154196:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6de15 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154196 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.417209:0:154196:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6de15 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154196 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.417216:0:154196:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6de15 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154196 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.417225:0:154196:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6de15 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154196 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.417233:0:154196:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6de15 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154196 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.417240:0:154196:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0512000 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787825949.417245:0:154196:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6de15 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154196 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.417253:0:154196:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6de15 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154196 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.417261:0:154196:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6de15 lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 154196 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.417268:0:154196:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787825949.417274:0:154196:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6de15 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154196 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.417281:0:154196:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6de15 lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154196 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825949.417300:0:154196:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181d1b32a00 x1874663042770048/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787825965 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787825949.417315:0:154196:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787825949.417341:0:154196:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787825949.417413:0:154196:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825949.417426:0:154196:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825949.417435:0:154196:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825949.417446:0:154196:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825949.417453:0:154196:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825949.417458:0:154196:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825949.417462:0:154196:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825949.417468:0:154196:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb9750498bb58 00010000:00010000:0.0:1787825949.417472:0:154196:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825949.417480:0:154196:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825949.417492:0:154196:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825949.417499:0:154196:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=13 00000080:00200008:0.0:1787825949.417508:0:154196:0:(file.c:6296:ll_getattr_dentry()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1787825949.417513:0:154196:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08),name=d71a.replay-single 00000002:00010000:0.0:1787825949.417524:0:154196:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x20001a9e3:0x17e0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825949.417541:0:154196:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00010000:00010000:0.0:1787825949.417576:0:154196:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6de1c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 154196 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.417869:0:154196:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6de1c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 154196 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.417881:0:154196:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6de1c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 154196 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787825949.418155:0:154196:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d1b32a00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154196:1874663042770176:192.168.203.147@tcp:101:lfs.0 00000100:00100000:1.0:1787825949.418239:0:154196:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b59c00 going to sleep for 11 seconds 00000080:00200000:1.0:1787825949.446434:0:154197:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787825949.446466:0:154197:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787825949.446484:0:154197:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825949.446500:0:154197:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825949.446508:0:154197:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787825949.446517:0:154197:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787825949.446521:0:154197:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787825949.446540:0:154197:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97504993c20 00010000:00010000:1.0:1787825949.446544:0:154197:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825949.446554:0:154197:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825949.446570:0:154197:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787825949.446654:0:154197:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=24 00000080:00200000:1.0:1787825949.446662:0:154197:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:1.0:1787825949.446675:0:154197:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=0 00000080:00200000:1.0:1787825949.446680:0:154197:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:1.0:1787825949.446693:0:154197:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787825949.446736:0:154197:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6de23 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154197 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825949.446896:0:154197:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6de23 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154197 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825949.446908:0:154197:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6de23 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154197 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787825949.446934:0:154197:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c7107480 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154197:1874663042770560:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:1.0:1787825949.447082:0:154197:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b59180 going to sleep for 11 seconds 00000100:00100000:1.0:1787825949.473645:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825949.473786:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048647680 00000100:00100000:0.0:1787825949.473796:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048647680 00000100:00100000:0.0:1787825949.473844:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9045 00000100:00100000:0.0:1787825949.473854:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c7107480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:41299:x1874663048647680:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825949.473871:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480600/0xbef42c0221d6de2a lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x381:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce4d4e expref: -99 pid: 154197 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825949.473957:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825949.473964:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c7107480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:41299:x1874663048647680:12345-192.168.203.147@tcp:104: Request processed in 112us (332us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825949.473971:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9045 00000100:00100000:0.0:1787825949.480510:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787825949.480617:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048647936 00000100:00100000:1.0:1787825949.480627:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048647936 00000100:00100000:1.0:1787825949.480684:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9046 00000100:00100000:1.0:1787825949.480705:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181d1b31f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663048647936:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787825949.480724:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481000/0xbef42c0221d6de31 lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x381:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce4d5c expref: -99 pid: 154197 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825949.480839:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787825949.480847:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181d1b31f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663048647936:12345-192.168.203.147@tcp:104: Request processed in 153us (345us total) trans 0 rc 0/0 00000100:00100000:1.0:1787825949.480855:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9046 00000080:00200000:0.0:1787825949.551277:0:154199:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825949.551308:0:154199:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825949.551327:0:154199:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825949.551343:0:154199:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825949.551351:0:154199:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825949.551360:0:154199:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825949.551363:0:154199:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825949.551370:0:154199:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb975049a3c20 00010000:00010000:0.0:1787825949.551374:0:154199:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825949.551382:0:154199:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825949.551397:0:154199:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825949.551404:0:154199:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=24 00000080:00200000:0.0:1787825949.551410:0:154199:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825949.551422:0:154199:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=0 00000080:00200000:0.0:1787825949.551426:0:154199:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787825949.551438:0:154199:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787825949.551485:0:154199:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6de4d lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154199 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.551493:0:154199:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6de4d lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154199 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.551501:0:154199:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6de4d lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154199 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825949.551536:0:154199:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d1b33100 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154199:1874663042772096:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787825949.551914:0:154199:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787825949.555553:0:154199:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d1b33100 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154199:1874663042772096:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:0.0:1787825949.555592:0:154199:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6de4d lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4e04 expref: -99 pid: 154199 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.555614:0:154199:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6de4d lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1b61:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4e04 expref: -99 pid: 154199 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.555623:0:154199:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6de4d lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1b61:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4e04 expref: -99 pid: 154199 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.555633:0:154199:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6de4d lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1b61:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4e04 expref: -99 pid: 154199 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.555642:0:154199:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6de4d lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1b61:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4e04 expref: -99 pid: 154199 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.555655:0:154199:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6de4d lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1b61:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4e04 expref: -99 pid: 154199 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.555662:0:154199:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6de4d lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1b61:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4e04 expref: -99 pid: 154199 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.555669:0:154199:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6de4d lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1b61:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4e04 expref: -99 pid: 154199 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825949.555680:0:154199:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181d1b33100 x1874663042772096/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/672 e 0 to 0 dl 1787825965 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787825949.555693:0:154199:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6de4d lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1b61:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4e04 expref: -99 pid: 154199 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.555715:0:154199:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/7009 (0 0) 00000080:00200000:0.0:1787825949.555721:0:154199:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00000002:00010000:0.0:1787825949.555813:0:154199:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200025d90:0x382:0x0]) in obj [0x200025d90:0x382:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825949.555861:0:154199:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6de54 lrc: 3/1,0 mode: --/CR res: [0x200025d90:0x382:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 154199 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.555869:0:154199:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6de54 lrc: 3/1,0 mode: --/CR res: [0x200025d90:0x382:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154199 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.555878:0:154199:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6de54 lrc: 3/1,0 mode: --/CR res: [0x200025d90:0x382:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154199 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825949.555897:0:154199:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d1b32680 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154199:1874663042772224:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787825949.555957:0:154199:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787825949.559346:0:154199:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d1b32680 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154199:1874663042772224:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:0.0:1787825949.559370:0:154199:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6de54 lrc: 4/1,0 mode: --/CR res: [0x200025d90:0x382:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4e12 expref: -99 pid: 154199 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.559381:0:154199:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6de54 lrc: 4/1,0 mode: --/PR res: [0x200025d90:0x382:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4e12 expref: -99 pid: 154199 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.559391:0:154199:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6de54 lrc: 4/1,0 mode: PR/PR res: [0x200025d90:0x382:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4e12 expref: -99 pid: 154199 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.559400:0:154199:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6de54 lrc: 4/1,0 mode: PR/PR res: [0x200025d90:0x382:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4e12 expref: -99 pid: 154199 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.559413:0:154199:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6de54 lrc: 5/2,0 mode: PR/PR res: [0x200025d90:0x382:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4e12 expref: -99 pid: 154199 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.559422:0:154199:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6de54 lrc: 5/2,0 mode: PR/PR res: [0x200025d90:0x382:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4e12 expref: -99 pid: 154199 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.559429:0:154199:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6de54 lrc: 4/1,0 mode: PR/PR res: [0x200025d90:0x382:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4e12 expref: -99 pid: 154199 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825949.559439:0:154199:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181d1b32680 x1874663042772224/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787825965 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787825949.559452:0:154199:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6de54 lrc: 3/1,0 mode: PR/PR res: [0x200025d90:0x382:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4e12 expref: -99 pid: 154199 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.559470:0:154199:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590089616/898 (0 0) 00010000:00010000:0.0:1787825949.559476:0:154199:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6de54 lrc: 3/1,0 mode: PR/PR res: [0x200025d90:0x382:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4e12 expref: -99 pid: 154199 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.559484:0:154199:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6de54 lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x382:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4e12 expref: -99 pid: 154199 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000002:00010000:0.0:1787825949.559509:0:154199:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x24000178a:0x382:0x0]) in obj [0x24000178a:0x382:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825949.559568:0:154199:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6de5b lrc: 3/1,0 mode: --/CR res: [0x24000178a:0x382:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 154199 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.559574:0:154199:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6de5b lrc: 3/1,0 mode: --/CR res: [0x24000178a:0x382:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154199 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.559592:0:154199:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6de5b lrc: 3/1,0 mode: --/CR res: [0x24000178a:0x382:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154199 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825949.559609:0:154199:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d1b32680 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154199:1874663042772352:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787825949.559676:0:154199:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787825949.567211:0:154199:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d1b32680 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154199:1874663042772352:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:0.0:1787825949.567236:0:154199:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6de5b lrc: 4/1,0 mode: --/CR res: [0x24000178a:0x382:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4e20 expref: -99 pid: 154199 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.567247:0:154199:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6de5b lrc: 4/1,0 mode: --/PR res: [0x24000178a:0x382:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4e20 expref: -99 pid: 154199 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.567259:0:154199:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6de5b lrc: 4/1,0 mode: PR/PR res: [0x24000178a:0x382:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4e20 expref: -99 pid: 154199 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.567271:0:154199:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6de5b lrc: 4/1,0 mode: PR/PR res: [0x24000178a:0x382:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4e20 expref: -99 pid: 154199 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.567283:0:154199:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6de5b lrc: 5/2,0 mode: PR/PR res: [0x24000178a:0x382:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4e20 expref: -99 pid: 154199 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.567291:0:154199:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6de5b lrc: 5/2,0 mode: PR/PR res: [0x24000178a:0x382:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4e20 expref: -99 pid: 154199 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.567298:0:154199:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6de5b lrc: 4/1,0 mode: PR/PR res: [0x24000178a:0x382:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4e20 expref: -99 pid: 154199 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825949.567309:0:154199:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181d1b32680 x1874663042772352/t0(0) o101->lustre-MDT0001-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787825965 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787825949.567324:0:154199:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6de5b lrc: 3/1,0 mode: PR/PR res: [0x24000178a:0x382:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4e20 expref: -99 pid: 154199 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.567336:0:154199:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d0f54800 type 13 mode 4 res 9663682442/898 (0 0) 00010000:00010000:0.0:1787825949.567345:0:154199:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6de5b lrc: 3/1,0 mode: PR/PR res: [0x24000178a:0x382:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4e20 expref: -99 pid: 154199 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.567353:0:154199:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6de5b lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x382:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4e20 expref: -99 pid: 154199 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787825949.567381:0:154199:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1b61:0x0], updating i_size 8192 00000080:00200000:0.0:1787825949.567385:0:154199:0:(namei.c:147:ll_iget()) got inode: [0x20001a9e3:0x1b61:0x0](ffff9181c838aa08): rc = 0 00000080:00010000:0.0:1787825949.567395:0:154199:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x1b61:0x0](ffff9181c838aa08) for lock 0xbef42c0221d6de4d 00000080:00010000:0.0:1787825949.567412:0:154199:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x1b61:0x0](ffff9181c838aa08) 00000080:00010000:0.0:1787825949.567415:0:154199:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6de4d from it ffffb975049a3e08 00010000:00010000:0.0:1787825949.567419:0:154199:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6de4d lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1b61:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4e04 expref: -99 pid: 154199 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.567427:0:154199:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6de4d lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1b61:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4e04 expref: -99 pid: 154199 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787825949.567446:0:154199:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 3 00000080:00200000:0.0:1787825949.567454:0:154199:0:(namei.c:2328:ll_rmdir()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00010000:00010000:0.0:1787825949.567480:0:154199:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6de4d lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1b61:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ce4e04 expref: -99 pid: 154199 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825949.567523:0:154199:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6de4d lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x1b61:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce4e04 expref: -99 pid: 154199 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825949.567545:0:154199:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787825949.567548:0:154199:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6de4d lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x1b61:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce4e04 expref: -99 pid: 154199 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:0.0:1787825949.567572:0:154199:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787825949.567589:0:154199:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181d1b33100 x1874663042772608/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787825949.567612:0:154199:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d1b33100 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154199:1874663042772608:192.168.203.147@tcp:36:rmdir.0 00000100:00100000:0.0:1787825949.567683:0:154199:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787825949.571243:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825949.571360:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048649344 00000100:00100000:0.0:1787825949.571369:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048649344 00000100:00100000:0.0:1787825949.571417:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9047 00000100:00100000:0.0:1787825949.571426:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c7105500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6492:x1874663048649344:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825949.571442:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6de54 lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x382:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce4e12 expref: -99 pid: 154199 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825949.571513:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825949.571520:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c7105500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6492:x1874663048649344:12345-192.168.203.147@tcp:104: Request processed in 96us (288us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825949.571538:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9047 00000100:00100000:1.0:1787825949.577786:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825949.577872:0:15151:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:0.0:1787825949.577891:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048649600 00000100:00100000:0.0:1787825949.577899:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048649600 00000100:00100000:0.0:1787825949.577957:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9048 00000100:00100000:0.0:1787825949.577968:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c440a050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8414:x1874663048649600:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825949.577984:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6de5b lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x382:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce4e20 expref: -99 pid: 154199 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825949.578057:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825949.578064:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c440a050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8414:x1874663048649600:12345-192.168.203.147@tcp:104: Request processed in 98us (285us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825949.578072:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9048 00000100:00100000:0.0:1787825949.596225:0:154199:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d1b33100 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154199:1874663042772608:192.168.203.147@tcp:36:rmdir.0 00000080:00200000:0.0:1787825949.596272:0:154199:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1b61:0x0](ffff9181c838aa08) 00000080:00200000:0.0:1787825949.609446:0:154200:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825949.609477:0:154200:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825949.609495:0:154200:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825949.609511:0:154200:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825949.609518:0:154200:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825949.609535:0:154200:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825949.609539:0:154200:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825949.609548:0:154200:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97504933b58 00010000:00010000:0.0:1787825949.609552:0:154200:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825949.609560:0:154200:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825949.609574:0:154200:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825949.609706:0:154200:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=29 00000080:00200000:0.0:1787825949.609711:0:154200:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825949.609724:0:154200:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=13 00000080:00200000:0.0:1787825949.609729:0:154200:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787825949.609819:0:154200:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787825949.609865:0:154200:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6de62 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154200 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.609873:0:154200:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6de62 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154200 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.609883:0:154200:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6de62 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154200 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825949.609913:0:154200:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d1b30700 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154200:1874663042772992:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787825949.610049:0:154200:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787825949.613498:0:154200:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d1b30700 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154200:1874663042772992:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787825949.613517:0:154200:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6de62 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154200 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.613539:0:154200:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6de62 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154200 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.613547:0:154200:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6de62 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154200 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.613553:0:154200:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6de62 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154200 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.613561:0:154200:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6de62 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154200 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.613567:0:154200:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0513a00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787825949.613571:0:154200:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6de62 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154200 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.613589:0:154200:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6de62 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154200 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.613596:0:154200:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6de62 lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 154200 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.613603:0:154200:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787825949.613610:0:154200:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6de62 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154200 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.613616:0:154200:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6de62 lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154200 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825949.613634:0:154200:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181d1b30700 x1874663042772992/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787825965 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787825949.613648:0:154200:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787825949.613674:0:154200:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787825949.613741:0:154200:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825949.613904:0:154200:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825949.613914:0:154200:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825949.613924:0:154200:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825949.613931:0:154200:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825949.613936:0:154200:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825949.613940:0:154200:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825949.613945:0:154200:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97504933be8 00010000:00010000:0.0:1787825949.613949:0:154200:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825949.613958:0:154200:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825949.613972:0:154200:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825949.613979:0:154200:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=16408 00000080:00200000:0.0:1787825949.613984:0:154200:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825949.613995:0:154200:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=16392 00000080:00200000:0.0:1787825949.613999:0:154200:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787825949.614012:0:154200:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787825949.614051:0:154200:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6de69 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154200 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.614059:0:154200:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6de69 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154200 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.614068:0:154200:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6de69 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154200 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825949.614087:0:154200:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d1b30700 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154200:1874663042773120:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787825949.614162:0:154200:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787825949.618958:0:154200:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d1b30700 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154200:1874663042773120:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787825949.618979:0:154200:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6de69 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154200 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.618991:0:154200:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6de69 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154200 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.618999:0:154200:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6de69 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154200 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.619006:0:154200:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6de69 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154200 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.619014:0:154200:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6de69 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154200 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.619021:0:154200:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0512e00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787825949.619026:0:154200:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6de69 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154200 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.619034:0:154200:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6de69 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154200 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.619041:0:154200:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6de69 lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 154200 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.619049:0:154200:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787825949.619054:0:154200:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6de69 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154200 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.619061:0:154200:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6de69 lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154200 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825949.619078:0:154200:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181d1b30700 x1874663042773120/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787825965 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787825949.619090:0:154200:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787825949.619112:0:154200:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787825949.619188:0:154200:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825949.619200:0:154200:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825949.619208:0:154200:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825949.619220:0:154200:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825949.619231:0:154200:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825949.619235:0:154200:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825949.619238:0:154200:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825949.619242:0:154200:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97504933b58 00010000:00010000:0.0:1787825949.619245:0:154200:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825949.619252:0:154200:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825949.619263:0:154200:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825949.619269:0:154200:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=13 00000080:00200008:0.0:1787825949.619276:0:154200:0:(file.c:6296:ll_getattr_dentry()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1787825949.619280:0:154200:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08),name=d71a.replay-single 00000002:00010000:0.0:1787825949.619289:0:154200:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x20001a9e3:0x17e0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825949.619293:0:154200:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00010000:00010000:0.0:1787825949.619322:0:154200:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6de70 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 154200 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.619328:0:154200:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6de70 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 154200 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.619337:0:154200:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6de70 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 154200 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825949.619355:0:154200:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d1b30700 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154200:1874663042773248:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787825949.619412:0:154200:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787825949.623302:0:154200:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d1b30700 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154200:1874663042773248:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787825949.623332:0:154200:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6de70 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4e82 expref: -99 pid: 154200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.623350:0:154200:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6de70 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4e82 expref: -99 pid: 154200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.623357:0:154200:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6de70 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4e82 expref: -99 pid: 154200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.623369:0:154200:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6de70 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4e82 expref: -99 pid: 154200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.623379:0:154200:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6de70 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4e82 expref: -99 pid: 154200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.623392:0:154200:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6de70 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4e82 expref: -99 pid: 154200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.623399:0:154200:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6de70 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4e82 expref: -99 pid: 154200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.623407:0:154200:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6de70 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4e82 expref: -99 pid: 154200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825949.623420:0:154200:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181d1b30700 x1874663042773248/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787825965 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787825949.623433:0:154200:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6de70 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4e82 expref: -99 pid: 154200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.623445:0:154200:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787825949.623451:0:154200:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1787825949.623477:0:154200:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x17e0:0x0], updating i_size 4096 00000080:00010000:0.0:1787825949.623484:0:154200:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) for lock 0xbef42c0221d6de70 00000080:00010000:0.0:1787825949.623491:0:154200:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00000080:00010000:0.0:1787825949.623494:0:154200:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6de70 from it ffffb97504933cb8 00010000:00010000:0.0:1787825949.623497:0:154200:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6de70 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce4e82 expref: -99 pid: 154200 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.623506:0:154200:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6de70 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce4e82 expref: -99 pid: 154200 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:0.0:1787825949.625034:0:154200:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1787825949.625044:0:154200:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825949.625065:0:154200:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825949.625079:0:154200:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825949.625093:0:154200:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825949.625101:0:154200:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825949.625107:0:154200:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825949.625111:0:154200:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825949.625116:0:154200:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97504933cb8 00010000:00010000:0.0:1787825949.625119:0:154200:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825949.625128:0:154200:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825949.625362:0:154200:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825949.625372:0:154200:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825949.625379:0:154200:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825949.625389:0:154200:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825949.625395:0:154200:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825949.625400:0:154200:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825949.625403:0:154200:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825949.625407:0:154200:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97504933bf8 00010000:00010000:0.0:1787825949.625410:0:154200:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825949.625417:0:154200:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825949.625427:0:154200:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825949.625434:0:154200:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=267 00000080:00200000:0.0:1787825949.625440:0:154200:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 44 00000080:00200008:0.0:1787825949.625446:0:154200:0:(file.c:1090:ll_file_open()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00000080:00200080:0.0:1787825949.625473:0:154200:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) cmd=c03066f0 arg=7fffbf95af30 00000080:00200000:0.0:1787825949.625490:0:154200:0:(dir.c:451:ll_dir_setdirstripe()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) name=test stripe_offset=0 stripe_count=2, hash_type=0 00010000:00010000:0.0:1787825949.625520:0:154200:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6de70 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ce4e82 expref: -99 pid: 154200 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825949.625548:0:154200:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6800/0xbef42c0221d64b3f lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cb9ffd expref: -99 pid: 152202 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825949.625978:0:154200:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6de70 lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce4e82 expref: -99 pid: 154200 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825949.625988:0:154200:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787825949.625990:0:154200:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6de70 lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce4e82 expref: -99 pid: 154200 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:0.0:1787825949.626009:0:154200:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787825949.626021:0:154200:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181d1b30700 x1874663042773504/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 552/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787825949.626047:0:154200:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d1b30700 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154200:1874663042773504:192.168.203.147@tcp:36:lfs.0 00000100:00100000:0.0:1787825949.626131:0:154200:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787825949.634597:0:154200:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d1b30700 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154200:1874663042773504:192.168.203.147@tcp:36:lfs.0 00000080:00200000:0.0:1787825949.634661:0:154200:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x20001a9e3:0x1b62:0x0](ffff9181c8386c08) 00000080:00200000:0.0:1787825949.634669:0:154200:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1b62:0x0], updating i_size 4096 00000080:00200008:0.0:1787825949.634719:0:154200:0:(file.c:479:ll_file_release()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00010000:00010000:0.0:1787825949.634791:0:154200:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513000/0xbef42c0221d64bee lrc: 2/0,0 mode: CR/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cba2b9 expref: -99 pid: 152210 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000100:00100000:1.0:1787825949.666431:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825949.666634:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048651008 00000100:00100000:0.0:1787825949.666644:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048651008 00000100:00100000:0.0:1787825949.666694:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9049 00000100:00100000:0.0:1787825949.666711:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c7107100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6490:x1874663048651008:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825949.666729:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480400/0xbef42c0221d6de7e lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x383:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce4ed6 expref: -99 pid: 154201 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825949.666826:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825949.666834:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c7107100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6490:x1874663048651008:12345-192.168.203.147@tcp:104: Request processed in 125us (413us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825949.666841:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9049 00000100:00100000:1.0:1787825949.674105:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825949.674237:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048651264 00000100:00100000:0.0:1787825949.674247:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048651264 00000100:00100000:0.0:1787825949.674296:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9050 00000100:00100000:0.0:1787825949.674306:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c7105c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663048651264:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825949.674323:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f7600/0xbef42c0221d6de85 lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x383:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce4ee4 expref: -99 pid: 154201 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825949.674394:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825949.674401:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c7105c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663048651264:12345-192.168.203.147@tcp:104: Request processed in 97us (304us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825949.674408:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9050 00000100:00100000:1.0:1787825949.783831:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825949.783991:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048652672 00000100:00100000:0.0:1787825949.784000:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048652672 00000100:00100000:0.0:1787825949.784055:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9051 00000100:00100000:0.0:1787825949.784065:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c7106d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6491:x1874663048652672:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825949.784083:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6dea8 lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x384:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce4f9a expref: -99 pid: 154203 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825949.784180:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825949.784186:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c7106d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6491:x1874663048652672:12345-192.168.203.147@tcp:104: Request processed in 123us (365us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825949.784194:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9051 00000100:00100000:1.0:1787825949.791041:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825949.794381:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048652928 00000100:00100000:0.0:1787825949.794397:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048652928 00000100:00100000:0.0:1787825949.794475:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9052 00000100:00100000:0.0:1787825949.794486:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c7107480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663048652928:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825949.794505:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6deaf lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x384:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce4fa8 expref: -99 pid: 154203 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825949.794609:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825949.794616:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c7107480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663048652928:12345-192.168.203.147@tcp:104: Request processed in 132us (3585us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825949.794623:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9052 00000080:00200000:1.0:1787825949.826569:0:154203:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1b63:0x0](ffff9181c838cb08) 00000080:00200000:0.0:1787825949.850686:0:154204:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825949.850725:0:154204:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825949.851228:0:154204:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825949.851246:0:154204:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825949.851255:0:154204:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825949.851265:0:154204:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825949.851268:0:154204:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825949.851276:0:154204:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97504923b58 00010000:00010000:0.0:1787825949.851280:0:154204:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825949.851289:0:154204:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825949.851305:0:154204:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825949.851314:0:154204:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=29 00000080:00200000:0.0:1787825949.851321:0:154204:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825949.851335:0:154204:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=13 00000080:00200000:0.0:1787825949.851339:0:154204:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787825949.851355:0:154204:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787825949.851397:0:154204:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512a00/0xbef42c0221d6deb6 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154204 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.851406:0:154204:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512a00/0xbef42c0221d6deb6 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154204 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.851414:0:154204:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512a00/0xbef42c0221d6deb6 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154204 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825949.851438:0:154204:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d1b31f80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154204:1874663042776064:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787825949.852018:0:154204:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8480 going to sleep for 11 seconds 00000100:00100000:0.0:1787825949.868104:0:154204:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d1b31f80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154204:1874663042776064:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787825949.868135:0:154204:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512a00/0xbef42c0221d6deb6 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154204 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.868230:0:154204:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512a00/0xbef42c0221d6deb6 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154204 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.868240:0:154204:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512a00/0xbef42c0221d6deb6 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154204 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.868247:0:154204:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512a00/0xbef42c0221d6deb6 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154204 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.868256:0:154204:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512a00/0xbef42c0221d6deb6 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154204 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.868264:0:154204:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0512a00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787825949.868269:0:154204:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512a00/0xbef42c0221d6deb6 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154204 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.868276:0:154204:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512a00/0xbef42c0221d6deb6 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154204 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.868284:0:154204:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512a00/0xbef42c0221d6deb6 lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 154204 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.868290:0:154204:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787825949.868297:0:154204:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512a00/0xbef42c0221d6deb6 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154204 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825949.868303:0:154204:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512a00/0xbef42c0221d6deb6 lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154204 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825949.868325:0:154204:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181d1b31f80 x1874663042776064/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787825965 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787825949.868340:0:154204:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:1.0:1787825949.873065:0:154204:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:1.0:1787825949.873555:0:154204:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787825949.873576:0:154204:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787825949.873588:0:154204:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825949.873601:0:154204:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825949.873609:0:154204:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787825949.873614:0:154204:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787825949.873618:0:154204:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787825949.873625:0:154204:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97504923be8 00010000:00010000:1.0:1787825949.873629:0:154204:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825949.873637:0:154204:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825949.873651:0:154204:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787825949.873658:0:154204:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=16408 00000080:00200000:1.0:1787825949.873663:0:154204:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000100:00100000:1.0:1787825949.968857:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825949.968957:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048654336 00000100:00100000:0.0:1787825949.968967:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048654336 00000100:00100000:0.0:1787825949.969023:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9053 00000100:00100000:0.0:1787825949.969034:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c7105500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6492:x1874663048654336:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825949.969052:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6ded2 lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x385:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce505e expref: -99 pid: 154205 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825949.969128:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825949.969134:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c7105500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6492:x1874663048654336:12345-192.168.203.147@tcp:104: Request processed in 103us (287us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825949.969140:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9053 00010000:00010000:0.0:1787825949.969378:0:15161:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6ded2 lrc: 2/0,0 mode: --/PR res: [0x200025d90:0x385:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ce505e expref: -99 pid: 154205 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825949.969385:0:15161:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787825949.969406:0:15161:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6ded2 lrc: 1/0,0 mode: --/PR res: [0x200025d90:0x385:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ce505e expref: -99 pid: 154205 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825949.969414:0:15161:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6ded2 lrc: 0/0,0 mode: --/PR res: [0x200025d90:0x385:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ce505e expref: -99 pid: 154205 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825949.969436:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787825949.969438:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787825949.969439:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787825949.969454:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d1b32680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042777344:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787825949.978434:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d1b32680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042777344:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787825949.992459:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825949.993281:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048654592 00000100:00100000:0.0:1787825949.993292:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048654592 00000100:00100000:0.0:1787825949.993402:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9054 00000100:00100000:0.0:1787825949.993414:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c33fca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663048654592:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825949.993432:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480400/0xbef42c0221d6ded9 lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x385:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce506c expref: -99 pid: 154205 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825949.993531:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825949.993537:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c33fca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663048654592:12345-192.168.203.147@tcp:104: Request processed in 126us (1088us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825949.993544:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9054 00000080:00200000:1.0:1787825950.041064:0:154206:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787825950.041095:0:154206:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787825950.041114:0:154206:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825950.041130:0:154206:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825950.041139:0:154206:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787825950.041273:0:154206:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787825950.041277:0:154206:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787825950.041286:0:154206:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb975049b3b58 00010000:00010000:1.0:1787825950.041291:0:154206:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825950.041308:0:154206:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825950.041324:0:154206:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787825950.041332:0:154206:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=29 00000080:00200000:1.0:1787825950.041338:0:154206:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:1.0:1787825950.041351:0:154206:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=13 00000080:00200000:1.0:1787825950.041357:0:154206:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:1.0:1787825950.041370:0:154206:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787825950.041418:0:154206:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6dee0 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154206 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825950.041426:0:154206:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6dee0 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154206 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825950.041436:0:154206:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6dee0 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154206 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787825950.041463:0:154206:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c49a6680 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154206:1874663042777600:192.168.203.147@tcp:101:lfs.0 00000100:00100000:1.0:1787825950.041714:0:154206:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b59600 going to sleep for 11 seconds 00000080:00200000:0.0:1787825950.103364:0:154207:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825950.103400:0:154207:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825950.103420:0:154207:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825950.103436:0:154207:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825950.103444:0:154207:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825950.103456:0:154207:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825950.103459:0:154207:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825950.103468:0:154207:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb975049bbc20 00010000:00010000:0.0:1787825950.103472:0:154207:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825950.103481:0:154207:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825950.103498:0:154207:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825950.103508:0:154207:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=24 00000080:00200000:0.0:1787825950.103515:0:154207:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825950.103543:0:154207:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=0 00000080:00200000:0.0:1787825950.103547:0:154207:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787825950.103558:0:154207:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787825950.103614:0:154207:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6def5 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154207 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.103623:0:154207:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6def5 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154207 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.103633:0:154207:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6def5 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154207 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825950.103659:0:154207:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d1b30e00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154207:1874663042778240:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787825950.103790:0:154207:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c83c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787825950.109541:0:154207:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d1b30e00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154207:1874663042778240:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:0.0:1787825950.109565:0:154207:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6def5 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5114 expref: -99 pid: 154207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.109597:0:154207:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6def5 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1b65:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5114 expref: -99 pid: 154207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.109605:0:154207:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6def5 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1b65:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5114 expref: -99 pid: 154207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.109615:0:154207:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6def5 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1b65:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5114 expref: -99 pid: 154207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.109630:0:154207:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6def5 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1b65:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5114 expref: -99 pid: 154207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.109643:0:154207:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6def5 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1b65:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5114 expref: -99 pid: 154207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.109651:0:154207:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6def5 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1b65:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5114 expref: -99 pid: 154207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.109658:0:154207:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6def5 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1b65:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5114 expref: -99 pid: 154207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825950.109669:0:154207:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181d1b30e00 x1874663042778240/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/672 e 0 to 0 dl 1787825966 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787825950.109683:0:154207:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6def5 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1b65:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5114 expref: -99 pid: 154207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.109692:0:154207:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/7013 (0 0) 00000080:00200000:0.0:1787825950.109698:0:154207:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00000002:00010000:0.0:1787825950.109793:0:154207:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200025d90:0x386:0x0]) in obj [0x200025d90:0x386:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825950.109851:0:154207:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6defc lrc: 3/1,0 mode: --/CR res: [0x200025d90:0x386:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 154207 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.109859:0:154207:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6defc lrc: 3/1,0 mode: --/CR res: [0x200025d90:0x386:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154207 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.109867:0:154207:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6defc lrc: 3/1,0 mode: --/CR res: [0x200025d90:0x386:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154207 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825950.109886:0:154207:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d1b32680 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154207:1874663042778368:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787825950.109953:0:154207:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c83c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787825950.125628:0:154207:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d1b32680 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154207:1874663042778368:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:0.0:1787825950.125656:0:154207:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6defc lrc: 4/1,0 mode: --/CR res: [0x200025d90:0x386:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5122 expref: -99 pid: 154207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.125670:0:154207:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6defc lrc: 4/1,0 mode: --/PR res: [0x200025d90:0x386:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5122 expref: -99 pid: 154207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.125683:0:154207:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6defc lrc: 4/1,0 mode: PR/PR res: [0x200025d90:0x386:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5122 expref: -99 pid: 154207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.125695:0:154207:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6defc lrc: 4/1,0 mode: PR/PR res: [0x200025d90:0x386:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5122 expref: -99 pid: 154207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.125708:0:154207:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6defc lrc: 5/2,0 mode: PR/PR res: [0x200025d90:0x386:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5122 expref: -99 pid: 154207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.125715:0:154207:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6defc lrc: 5/2,0 mode: PR/PR res: [0x200025d90:0x386:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5122 expref: -99 pid: 154207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.125723:0:154207:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6defc lrc: 4/1,0 mode: PR/PR res: [0x200025d90:0x386:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5122 expref: -99 pid: 154207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825950.125735:0:154207:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181d1b32680 x1874663042778368/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787825966 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787825950.125791:0:154207:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6defc lrc: 3/1,0 mode: PR/PR res: [0x200025d90:0x386:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5122 expref: -99 pid: 154207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.125815:0:154207:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590089616/902 (0 0) 00010000:00010000:0.0:1787825950.125824:0:154207:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6defc lrc: 3/1,0 mode: PR/PR res: [0x200025d90:0x386:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5122 expref: -99 pid: 154207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.125832:0:154207:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6defc lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x386:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5122 expref: -99 pid: 154207 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000002:00010000:0.0:1787825950.125860:0:154207:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x24000178a:0x386:0x0]) in obj [0x24000178a:0x386:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825950.125908:0:154207:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6df03 lrc: 3/1,0 mode: --/CR res: [0x24000178a:0x386:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 154207 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.125916:0:154207:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6df03 lrc: 3/1,0 mode: --/CR res: [0x24000178a:0x386:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154207 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.125925:0:154207:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6df03 lrc: 3/1,0 mode: --/CR res: [0x24000178a:0x386:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154207 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825950.125949:0:154207:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d1b32680 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154207:1874663042778496:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787825950.126010:0:154207:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c83c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787825950.140216:0:154207:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d1b32680 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154207:1874663042778496:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:0.0:1787825950.140242:0:154207:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6df03 lrc: 4/1,0 mode: --/CR res: [0x24000178a:0x386:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5130 expref: -99 pid: 154207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.140254:0:154207:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6df03 lrc: 4/1,0 mode: --/PR res: [0x24000178a:0x386:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5130 expref: -99 pid: 154207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.140266:0:154207:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6df03 lrc: 4/1,0 mode: PR/PR res: [0x24000178a:0x386:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5130 expref: -99 pid: 154207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.140277:0:154207:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6df03 lrc: 4/1,0 mode: PR/PR res: [0x24000178a:0x386:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5130 expref: -99 pid: 154207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.140289:0:154207:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6df03 lrc: 5/2,0 mode: PR/PR res: [0x24000178a:0x386:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5130 expref: -99 pid: 154207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.140296:0:154207:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6df03 lrc: 5/2,0 mode: PR/PR res: [0x24000178a:0x386:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5130 expref: -99 pid: 154207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.140303:0:154207:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6df03 lrc: 4/1,0 mode: PR/PR res: [0x24000178a:0x386:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5130 expref: -99 pid: 154207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825950.140314:0:154207:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181d1b32680 x1874663042778496/t0(0) o101->lustre-MDT0001-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787825966 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787825950.140327:0:154207:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6df03 lrc: 3/1,0 mode: PR/PR res: [0x24000178a:0x386:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5130 expref: -99 pid: 154207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.140339:0:154207:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d0f54800 type 13 mode 4 res 9663682442/902 (0 0) 00010000:00010000:0.0:1787825950.140347:0:154207:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6df03 lrc: 3/1,0 mode: PR/PR res: [0x24000178a:0x386:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5130 expref: -99 pid: 154207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.140355:0:154207:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6df03 lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x386:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5130 expref: -99 pid: 154207 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787825950.140394:0:154207:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1b65:0x0], updating i_size 8192 00000080:00200000:0.0:1787825950.140398:0:154207:0:(namei.c:147:ll_iget()) got inode: [0x20001a9e3:0x1b65:0x0](ffff9181c838c2c8): rc = 0 00000080:00010000:0.0:1787825950.140407:0:154207:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x1b65:0x0](ffff9181c838c2c8) for lock 0xbef42c0221d6def5 00000080:00010000:0.0:1787825950.140425:0:154207:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x1b65:0x0](ffff9181c838c2c8) 00000080:00010000:0.0:1787825950.140428:0:154207:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6def5 from it ffffb975049bbe08 00010000:00010000:0.0:1787825950.140432:0:154207:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6def5 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1b65:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5114 expref: -99 pid: 154207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.140441:0:154207:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6def5 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1b65:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5114 expref: -99 pid: 154207 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787825950.140462:0:154207:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 3 00000080:00200000:0.0:1787825950.140471:0:154207:0:(namei.c:2328:ll_rmdir()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00010000:00010000:0.0:1787825950.140500:0:154207:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6def5 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1b65:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ce5114 expref: -99 pid: 154207 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825950.140554:0:154207:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6def5 lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x1b65:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce5114 expref: -99 pid: 154207 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825950.140562:0:154207:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787825950.140566:0:154207:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6def5 lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x1b65:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce5114 expref: -99 pid: 154207 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:0.0:1787825950.141336:0:154207:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787825950.141348:0:154207:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181d1b30e00 x1874663042778752/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787825950.147898:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825950.148023:0:15150:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:0.0:1787825950.148042:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048656000 00000100:00100000:0.0:1787825950.148052:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048656000 00000100:00100000:0.0:1787825950.148111:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9055 00000100:00100000:0.0:1787825950.148122:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c32bdc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6490:x1874663048656000:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825950.148140:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6defc lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x386:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce5122 expref: -99 pid: 154207 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825950.148210:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825950.148216:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c32bdc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6490:x1874663048656000:12345-192.168.203.147@tcp:104: Request processed in 97us (328us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825950.148223:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9055 00000100:00100000:1.0:1787825950.168277:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825950.168449:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048656256 00000100:00100000:0.0:1787825950.168459:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048656256 00000100:00100000:0.0:1787825950.168519:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9056 00000100:00100000:0.0:1787825950.168541:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c49a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663048656256:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825950.168559:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6df03 lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x386:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce5130 expref: -99 pid: 154207 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825950.168630:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825950.168637:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c49a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663048656256:12345-192.168.203.147@tcp:104: Request processed in 99us (369us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825950.168645:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9056 00000080:00200000:0.0:1787825950.231410:0:154208:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825950.231446:0:154208:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825950.231465:0:154208:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825950.231485:0:154208:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825950.231492:0:154208:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825950.231501:0:154208:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825950.231505:0:154208:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825950.231513:0:154208:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb975049c3b58 00010000:00010000:0.0:1787825950.231517:0:154208:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825950.231535:0:154208:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825950.231548:0:154208:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825950.231554:0:154208:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=29 00000080:00200000:0.0:1787825950.231560:0:154208:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825950.231570:0:154208:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=13 00000080:00200000:0.0:1787825950.231574:0:154208:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787825950.231586:0:154208:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787825950.231631:0:154208:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6df0a lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154208 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.231640:0:154208:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6df0a lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154208 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.231650:0:154208:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6df0a lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154208 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825950.231678:0:154208:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d1b32680 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154208:1874663042779136:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787825950.232238:0:154208:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8300 going to sleep for 11 seconds 00000100:00100000:0.0:1787825950.242092:0:154208:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d1b32680 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154208:1874663042779136:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787825950.242115:0:154208:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6df0a lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154208 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.242129:0:154208:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6df0a lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154208 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.242136:0:154208:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6df0a lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154208 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.242143:0:154208:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6df0a lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154208 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.242152:0:154208:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6df0a lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154208 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.242158:0:154208:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0513e00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787825950.242162:0:154208:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6df0a lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154208 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.242169:0:154208:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6df0a lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154208 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.242176:0:154208:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6df0a lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 154208 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.242183:0:154208:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787825950.242189:0:154208:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6df0a lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154208 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.242196:0:154208:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6df0a lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154208 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825950.242228:0:154208:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181d1b32680 x1874663042779136/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787825966 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787825950.242242:0:154208:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787825950.242266:0:154208:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787825950.242335:0:154208:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825950.242350:0:154208:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825950.242359:0:154208:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825950.242371:0:154208:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825950.242377:0:154208:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825950.242383:0:154208:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825950.242386:0:154208:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825950.242391:0:154208:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb975049c3be8 00010000:00010000:0.0:1787825950.242395:0:154208:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825950.242402:0:154208:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825950.242415:0:154208:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825950.242423:0:154208:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=16408 00000080:00200000:0.0:1787825950.242428:0:154208:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825950.242441:0:154208:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=16392 00000080:00200000:0.0:1787825950.242445:0:154208:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787825950.242457:0:154208:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787825950.242492:0:154208:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6df11 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154208 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.242499:0:154208:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6df11 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154208 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.242508:0:154208:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6df11 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154208 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825950.242539:0:154208:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d1b32680 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154208:1874663042779264:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787825950.242605:0:154208:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8300 going to sleep for 11 seconds 00000100:00100000:0.0:1787825950.248975:0:154208:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d1b32680 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154208:1874663042779264:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787825950.248998:0:154208:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6df11 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154208 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.249012:0:154208:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6df11 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154208 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.249019:0:154208:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6df11 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154208 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.249026:0:154208:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6df11 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154208 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.249034:0:154208:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6df11 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154208 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.249040:0:154208:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0513c00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787825950.249045:0:154208:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6df11 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154208 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.249051:0:154208:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6df11 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154208 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.249058:0:154208:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6df11 lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 154208 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.249066:0:154208:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787825950.249072:0:154208:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6df11 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154208 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.249078:0:154208:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6df11 lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154208 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825950.249095:0:154208:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181d1b32680 x1874663042779264/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787825966 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787825950.249109:0:154208:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787825950.249135:0:154208:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787825950.249281:0:154208:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825950.249297:0:154208:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825950.249306:0:154208:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825950.249318:0:154208:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825950.249330:0:154208:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825950.249335:0:154208:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825950.249339:0:154208:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825950.249344:0:154208:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb975049c3b58 00010000:00010000:0.0:1787825950.249347:0:154208:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825950.249354:0:154208:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825950.249367:0:154208:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825950.249374:0:154208:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=13 00000080:00200008:0.0:1787825950.249383:0:154208:0:(file.c:6296:ll_getattr_dentry()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1787825950.249388:0:154208:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08),name=d71a.replay-single 00000002:00010000:0.0:1787825950.249398:0:154208:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x20001a9e3:0x17e0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825950.249403:0:154208:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00010000:00010000:0.0:1787825950.249441:0:154208:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6df18 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 154208 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.249448:0:154208:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6df18 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 154208 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.249457:0:154208:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6df18 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 154208 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825950.253348:0:154208:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d1b32680 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154208:1874663042779392:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787825950.253487:0:154208:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8300 going to sleep for 11 seconds 00000100:00100000:0.0:1787825950.256029:0:154208:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d1b32680 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154208:1874663042779392:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787825950.256057:0:154208:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6df18 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5192 expref: -99 pid: 154208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.256075:0:154208:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6df18 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5192 expref: -99 pid: 154208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.256084:0:154208:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6df18 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5192 expref: -99 pid: 154208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.256096:0:154208:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6df18 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5192 expref: -99 pid: 154208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.256106:0:154208:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6df18 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5192 expref: -99 pid: 154208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.256118:0:154208:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6df18 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5192 expref: -99 pid: 154208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.256126:0:154208:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6df18 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5192 expref: -99 pid: 154208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.256133:0:154208:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6df18 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5192 expref: -99 pid: 154208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825950.256265:0:154208:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181d1b32680 x1874663042779392/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787825966 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787825950.256282:0:154208:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6df18 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5192 expref: -99 pid: 154208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.256293:0:154208:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787825950.256299:0:154208:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1787825950.256327:0:154208:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x17e0:0x0], updating i_size 4096 00000080:00010000:0.0:1787825950.256333:0:154208:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) for lock 0xbef42c0221d6df18 00000080:00010000:0.0:1787825950.256341:0:154208:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00000080:00010000:0.0:1787825950.256344:0:154208:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6df18 from it ffffb975049c3cb8 00010000:00010000:0.0:1787825950.256347:0:154208:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6df18 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5192 expref: -99 pid: 154208 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.256356:0:154208:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6df18 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5192 expref: -99 pid: 154208 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:1.0:1787825950.262102:0:154208:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), request_mask 0, flags 0, foreign 0 00000080:00200000:1.0:1787825950.262112:0:154208:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787825950.262136:0:154208:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787825950.262371:0:154208:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825950.262393:0:154208:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825950.262401:0:154208:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787825950.262410:0:154208:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787825950.262413:0:154208:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787825950.262421:0:154208:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb975049c3cb8 00010000:00010000:1.0:1787825950.262425:0:154208:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825950.262433:0:154208:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825950.263139:0:154208:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787825950.263220:0:154208:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787825950.263233:0:154208:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825950.263243:0:154208:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825950.263249:0:154208:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787825950.263254:0:154208:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787825950.263257:0:154208:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787825950.263261:0:154208:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb975049c3bf8 00010000:00010000:1.0:1787825950.263264:0:154208:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825950.263272:0:154208:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825950.263283:0:154208:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787825950.263290:0:154208:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=267 00000080:00200000:1.0:1787825950.263297:0:154208:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 44 00000080:00200008:1.0:1787825950.263305:0:154208:0:(file.c:1090:ll_file_open()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00000080:00200080:1.0:1787825950.263330:0:154208:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) cmd=c03066f0 arg=7ffebaa2ef50 00000080:00200000:1.0:1787825950.263343:0:154208:0:(dir.c:451:ll_dir_setdirstripe()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) name=test stripe_offset=0 stripe_count=2, hash_type=0 00010000:00010000:1.0:1787825950.263368:0:154208:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6df18 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ce5192 expref: -99 pid: 154208 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825950.263382:0:154208:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6800/0xbef42c0221d64b3f lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cb9ffd expref: -99 pid: 152202 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825950.263423:0:154208:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6df18 lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce5192 expref: -99 pid: 154208 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825950.263429:0:154208:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787825950.263432:0:154208:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6df18 lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce5192 expref: -99 pid: 154208 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:1.0:1787825950.263449:0:154208:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787825950.263462:0:154208:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c49a6680 x1874663042779648/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 552/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787825950.263482:0:154208:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c49a6680 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154208:1874663042779648:192.168.203.147@tcp:36:lfs.0 00000100:00100000:1.0:1787825950.263693:0:154208:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b593c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787825950.286705:0:154208:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c49a6680 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154208:1874663042779648:192.168.203.147@tcp:36:lfs.0 00000080:00200000:0.0:1787825950.286801:0:154208:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x20001a9e3:0x1b66:0x0](ffff9181c837aa08) 00000080:00200000:0.0:1787825950.286809:0:154208:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1b66:0x0], updating i_size 4096 00000080:00200008:0.0:1787825950.286851:0:154208:0:(file.c:479:ll_file_release()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00010000:00010000:0.0:1787825950.286882:0:154208:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513000/0xbef42c0221d64bee lrc: 2/0,0 mode: CR/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cba2b9 expref: -99 pid: 152210 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000100:00100000:1.0:1787825950.359906:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825950.364138:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048657664 00000100:00100000:0.0:1787825950.364654:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048657664 00000100:00100000:0.0:1787825950.364829:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9057 00000100:00100000:0.0:1787825950.364841:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b3800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:14309:x1874663048657664:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825950.364859:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6200/0xbef42c0221d6df26 lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x387:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce51e6 expref: -99 pid: 154209 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825950.364957:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825950.364964:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b3800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:14309:x1874663048657664:12345-192.168.203.147@tcp:104: Request processed in 126us (5069us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825950.364971:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9057 00000100:00100000:1.0:1787825950.377472:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825950.377561:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048657920 00000100:00100000:0.0:1787825950.377572:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048657920 00000100:00100000:0.0:1787825950.377629:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9058 00000100:00100000:0.0:1787825950.377639:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663048657920:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825950.377656:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f7200/0xbef42c0221d6df2d lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x387:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce51f4 expref: -99 pid: 154209 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825950.377724:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825950.377731:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663048657920:12345-192.168.203.147@tcp:104: Request processed in 94us (269us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825950.377738:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9058 00000080:00200000:1.0:1787825950.414884:0:154210:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787825950.414918:0:154210:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787825950.414937:0:154210:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825950.414953:0:154210:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825950.414962:0:154210:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787825950.414970:0:154210:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787825950.414973:0:154210:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787825950.414981:0:154210:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb975049d3b58 00010000:00010000:1.0:1787825950.414984:0:154210:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825950.414993:0:154210:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825950.415009:0:154210:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787825950.415017:0:154210:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=29 00000080:00200000:1.0:1787825950.415023:0:154210:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:1.0:1787825950.415034:0:154210:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=13 00000080:00200000:1.0:1787825950.415039:0:154210:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:1.0:1787825950.415052:0:154210:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787825950.415098:0:154210:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4e00/0xbef42c0221d6df34 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154210 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825950.415107:0:154210:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4e00/0xbef42c0221d6df34 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154210 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825950.415117:0:154210:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4e00/0xbef42c0221d6df34 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154210 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787825950.415141:0:154210:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154210:1874663042780672:192.168.203.147@tcp:101:lfs.0 00000100:00100000:1.0:1787825950.415237:0:154210:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b59240 going to sleep for 11 seconds 00000080:00200000:0.0:1787825950.477215:0:154211:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825950.477246:0:154211:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825950.477263:0:154211:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825950.477279:0:154211:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825950.477286:0:154211:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825950.477295:0:154211:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825950.477299:0:154211:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825950.477307:0:154211:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb975049dbc20 00010000:00010000:0.0:1787825950.477312:0:154211:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825950.477321:0:154211:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825950.477335:0:154211:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825950.477343:0:154211:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=24 00000080:00200000:0.0:1787825950.477348:0:154211:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825950.477360:0:154211:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=0 00000080:00200000:0.0:1787825950.477364:0:154211:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787825950.477376:0:154211:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787825950.477420:0:154211:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6df49 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154211 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.477428:0:154211:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6df49 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154211 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.477437:0:154211:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6df49 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154211 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825950.477460:0:154211:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d1b32680 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154211:1874663042781312:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787825950.477565:0:154211:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8180 going to sleep for 11 seconds 00000100:00100000:1.0:1787825950.480882:0:154211:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d1b32680 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154211:1874663042781312:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:1.0:1787825950.480907:0:154211:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6df49 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce529c expref: -99 pid: 154211 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825950.480928:0:154211:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6df49 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1b67:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce529c expref: -99 pid: 154211 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825950.480936:0:154211:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6df49 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1b67:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce529c expref: -99 pid: 154211 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825950.480946:0:154211:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6df49 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1b67:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce529c expref: -99 pid: 154211 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825950.480957:0:154211:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6df49 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1b67:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce529c expref: -99 pid: 154211 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825950.480969:0:154211:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6df49 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1b67:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce529c expref: -99 pid: 154211 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825950.480983:0:154211:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6df49 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1b67:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce529c expref: -99 pid: 154211 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825950.480990:0:154211:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6df49 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1b67:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce529c expref: -99 pid: 154211 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787825950.481001:0:154211:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181d1b32680 x1874663042781312/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/672 e 0 to 0 dl 1787825966 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787825950.481014:0:154211:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6df49 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1b67:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce529c expref: -99 pid: 154211 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825950.481023:0:154211:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/7015 (0 0) 00000080:00200000:1.0:1787825950.481029:0:154211:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00000002:00010000:1.0:1787825950.481092:0:154211:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200025d90:0x388:0x0]) in obj [0x200025d90:0x388:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787825950.481138:0:154211:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5600/0xbef42c0221d6df50 lrc: 3/1,0 mode: --/CR res: [0x200025d90:0x388:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 154211 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825950.481160:0:154211:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5600/0xbef42c0221d6df50 lrc: 3/1,0 mode: --/CR res: [0x200025d90:0x388:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154211 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825950.481168:0:154211:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5600/0xbef42c0221d6df50 lrc: 3/1,0 mode: --/CR res: [0x200025d90:0x388:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154211 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787825950.481186:0:154211:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154211:1874663042781440:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:1.0:1787825950.481263:0:154211:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b59480 going to sleep for 11 seconds 00000100:00100000:1.0:1787825950.490946:0:154211:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154211:1874663042781440:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:1.0:1787825950.490977:0:154211:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5600/0xbef42c0221d6df50 lrc: 4/1,0 mode: --/CR res: [0x200025d90:0x388:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce52aa expref: -99 pid: 154211 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825950.490990:0:154211:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5600/0xbef42c0221d6df50 lrc: 4/1,0 mode: --/PR res: [0x200025d90:0x388:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce52aa expref: -99 pid: 154211 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825950.491001:0:154211:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5600/0xbef42c0221d6df50 lrc: 4/1,0 mode: PR/PR res: [0x200025d90:0x388:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce52aa expref: -99 pid: 154211 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825950.491012:0:154211:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5600/0xbef42c0221d6df50 lrc: 4/1,0 mode: PR/PR res: [0x200025d90:0x388:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce52aa expref: -99 pid: 154211 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825950.491025:0:154211:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5600/0xbef42c0221d6df50 lrc: 5/2,0 mode: PR/PR res: [0x200025d90:0x388:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce52aa expref: -99 pid: 154211 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825950.491033:0:154211:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5600/0xbef42c0221d6df50 lrc: 5/2,0 mode: PR/PR res: [0x200025d90:0x388:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce52aa expref: -99 pid: 154211 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825950.491040:0:154211:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5600/0xbef42c0221d6df50 lrc: 4/1,0 mode: PR/PR res: [0x200025d90:0x388:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce52aa expref: -99 pid: 154211 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787825950.491052:0:154211:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c35b3480 x1874663042781440/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787825966 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787825950.491066:0:154211:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5600/0xbef42c0221d6df50 lrc: 3/1,0 mode: PR/PR res: [0x200025d90:0x388:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce52aa expref: -99 pid: 154211 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825950.491077:0:154211:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590089616/904 (0 0) 00010000:00010000:1.0:1787825950.491085:0:154211:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5600/0xbef42c0221d6df50 lrc: 3/1,0 mode: PR/PR res: [0x200025d90:0x388:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce52aa expref: -99 pid: 154211 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825950.491093:0:154211:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5600/0xbef42c0221d6df50 lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x388:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce52aa expref: -99 pid: 154211 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000002:00010000:1.0:1787825950.491118:0:154211:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x24000178a:0x388:0x0]) in obj [0x24000178a:0x388:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787825950.491176:0:154211:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6df57 lrc: 3/1,0 mode: --/CR res: [0x24000178a:0x388:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 154211 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825950.491184:0:154211:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6df57 lrc: 3/1,0 mode: --/CR res: [0x24000178a:0x388:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154211 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825950.491192:0:154211:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6df57 lrc: 3/1,0 mode: --/CR res: [0x24000178a:0x388:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154211 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787825950.491238:0:154211:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154211:1874663042781568:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:1.0:1787825950.491312:0:154211:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b59480 going to sleep for 11 seconds 00000100:00100000:1.0:1787825950.497819:0:154211:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154211:1874663042781568:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:1.0:1787825950.497846:0:154211:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6df57 lrc: 4/1,0 mode: --/CR res: [0x24000178a:0x388:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce52b8 expref: -99 pid: 154211 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825950.497857:0:154211:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6df57 lrc: 4/1,0 mode: --/PR res: [0x24000178a:0x388:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce52b8 expref: -99 pid: 154211 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825950.497868:0:154211:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6df57 lrc: 4/1,0 mode: PR/PR res: [0x24000178a:0x388:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce52b8 expref: -99 pid: 154211 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825950.497879:0:154211:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6df57 lrc: 4/1,0 mode: PR/PR res: [0x24000178a:0x388:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce52b8 expref: -99 pid: 154211 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825950.497891:0:154211:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6df57 lrc: 5/2,0 mode: PR/PR res: [0x24000178a:0x388:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce52b8 expref: -99 pid: 154211 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825950.497899:0:154211:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6df57 lrc: 5/2,0 mode: PR/PR res: [0x24000178a:0x388:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce52b8 expref: -99 pid: 154211 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825950.497907:0:154211:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6df57 lrc: 4/1,0 mode: PR/PR res: [0x24000178a:0x388:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce52b8 expref: -99 pid: 154211 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787825950.497919:0:154211:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c35b3480 x1874663042781568/t0(0) o101->lustre-MDT0001-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787825966 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787825950.497932:0:154211:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6df57 lrc: 3/1,0 mode: PR/PR res: [0x24000178a:0x388:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce52b8 expref: -99 pid: 154211 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825950.497944:0:154211:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d0f54800 type 13 mode 4 res 9663682442/904 (0 0) 00010000:00010000:1.0:1787825950.497951:0:154211:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6df57 lrc: 3/1,0 mode: PR/PR res: [0x24000178a:0x388:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce52b8 expref: -99 pid: 154211 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825950.497958:0:154211:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6df57 lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x388:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce52b8 expref: -99 pid: 154211 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787825950.497990:0:154211:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1b67:0x0], updating i_size 8192 00000080:00200000:1.0:1787825950.497994:0:154211:0:(namei.c:147:ll_iget()) got inode: [0x20001a9e3:0x1b67:0x0](ffff9181c838ec08): rc = 0 00000080:00010000:1.0:1787825950.498003:0:154211:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x1b67:0x0](ffff9181c838ec08) for lock 0xbef42c0221d6df49 00000080:00010000:1.0:1787825950.498020:0:154211:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x1b67:0x0](ffff9181c838ec08) 00000080:00010000:1.0:1787825950.498023:0:154211:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6df49 from it ffffb975049dbe08 00010000:00010000:1.0:1787825950.498027:0:154211:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6df49 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1b67:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce529c expref: -99 pid: 154211 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825950.498035:0:154211:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6df49 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1b67:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce529c expref: -99 pid: 154211 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787825950.498055:0:154211:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 3 00000080:00200000:1.0:1787825950.498062:0:154211:0:(namei.c:2328:ll_rmdir()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00010000:00010000:1.0:1787825950.498086:0:154211:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6df49 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1b67:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ce529c expref: -99 pid: 154211 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825950.498127:0:154211:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6df49 lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x1b67:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce529c expref: -99 pid: 154211 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825950.498134:0:154211:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787825950.498137:0:154211:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6df49 lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x1b67:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce529c expref: -99 pid: 154211 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:1.0:1787825950.498162:0:154211:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787825950.498168:0:154211:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c35b3480 x1874663042781824/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787825950.498187:0:154211:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154211:1874663042781824:192.168.203.147@tcp:36:rmdir.0 00000100:00100000:1.0:1787825950.498247:0:154211:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b59480 going to sleep for 11 seconds 00000100:00100000:0.0:1787825950.501984:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787825950.502072:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048659328 00000100:00100000:1.0:1787825950.502082:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048659328 00000100:00100000:1.0:1787825950.502140:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9059 00000100:00100000:1.0:1787825950.502158:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181d1b31f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:7728:x1874663048659328:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787825950.502178:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5600/0xbef42c0221d6df50 lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x388:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce52aa expref: -99 pid: 154211 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825950.502253:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787825950.502261:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181d1b31f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:7728:x1874663048659328:12345-192.168.203.147@tcp:104: Request processed in 113us (286us total) trans 0 rc 0/0 00000100:00100000:1.0:1787825950.502268:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9059 00000100:00100000:0.0:1787825950.506861:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787825950.507067:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048659584 00000100:00100000:1.0:1787825950.507074:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048659584 00000100:00100000:1.0:1787825950.507118:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9060 00000100:00100000:1.0:1787825950.507127:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181d1b32a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663048659584:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787825950.507152:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6df57 lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x388:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce52b8 expref: -99 pid: 154211 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825950.507229:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787825950.507236:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181d1b32a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663048659584:12345-192.168.203.147@tcp:104: Request processed in 110us (386us total) trans 0 rc 0/0 00000100:00100000:1.0:1787825950.507243:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9060 00000100:00100000:1.0:1787825950.521941:0:154211:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154211:1874663042781824:192.168.203.147@tcp:36:rmdir.0 00000080:00200000:1.0:1787825950.521986:0:154211:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1b67:0x0](ffff9181c838ec08) 00000100:00100000:1.0:1787825950.587547:0:154213:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154213:1874663042782848:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:1.0:1787825950.587574:0:154213:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6df73 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5360 expref: -99 pid: 154213 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825950.587598:0:154213:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6df73 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1b68:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5360 expref: -99 pid: 154213 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825950.587606:0:154213:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6df73 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1b68:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5360 expref: -99 pid: 154213 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825950.587617:0:154213:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6df73 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1b68:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5360 expref: -99 pid: 154213 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825950.587628:0:154213:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6df73 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1b68:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5360 expref: -99 pid: 154213 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825950.587641:0:154213:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6df73 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1b68:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5360 expref: -99 pid: 154213 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825950.587650:0:154213:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6df73 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1b68:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5360 expref: -99 pid: 154213 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825950.587657:0:154213:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6df73 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1b68:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5360 expref: -99 pid: 154213 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787825950.587669:0:154213:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c5635500 x1874663042782848/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/672 e 0 to 0 dl 1787825966 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787825950.587682:0:154213:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6df73 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1b68:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5360 expref: -99 pid: 154213 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825950.587692:0:154213:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/7016 (0 0) 00000080:00200000:1.0:1787825950.587698:0:154213:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00000002:00010000:1.0:1787825950.587938:0:154213:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200025d90:0x389:0x0]) in obj [0x200025d90:0x389:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787825950.587987:0:154213:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6df7a lrc: 3/1,0 mode: --/CR res: [0x200025d90:0x389:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 154213 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825950.587995:0:154213:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6df7a lrc: 3/1,0 mode: --/CR res: [0x200025d90:0x389:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154213 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825950.588003:0:154213:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6df7a lrc: 3/1,0 mode: --/CR res: [0x200025d90:0x389:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154213 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787825950.588021:0:154213:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154213:1874663042782976:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:1.0:1787825950.588104:0:154213:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b596c0 going to sleep for 11 seconds 00000002:00010000:0.0:1787825950.599563:0:154213:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x24000178a:0x389:0x0]) in obj [0x24000178a:0x389:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825950.599618:0:154213:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6df81 lrc: 3/1,0 mode: --/CR res: [0x24000178a:0x389:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 154213 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.599630:0:154213:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6df81 lrc: 3/1,0 mode: --/CR res: [0x24000178a:0x389:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154213 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.599639:0:154213:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6df81 lrc: 3/1,0 mode: --/CR res: [0x24000178a:0x389:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154213 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825950.599660:0:154213:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d1b32680 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154213:1874663042783104:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787825950.599884:0:154213:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8540 going to sleep for 11 seconds 00000100:00100000:0.0:1787825950.612234:0:154213:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d1b32680 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154213:1874663042783104:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:0.0:1787825950.612260:0:154213:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6df81 lrc: 4/1,0 mode: --/CR res: [0x24000178a:0x389:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce537c expref: -99 pid: 154213 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.612272:0:154213:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6df81 lrc: 4/1,0 mode: --/PR res: [0x24000178a:0x389:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce537c expref: -99 pid: 154213 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.612283:0:154213:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6df81 lrc: 4/1,0 mode: PR/PR res: [0x24000178a:0x389:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce537c expref: -99 pid: 154213 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.612294:0:154213:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6df81 lrc: 4/1,0 mode: PR/PR res: [0x24000178a:0x389:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce537c expref: -99 pid: 154213 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.612306:0:154213:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6df81 lrc: 5/2,0 mode: PR/PR res: [0x24000178a:0x389:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce537c expref: -99 pid: 154213 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.612315:0:154213:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6df81 lrc: 5/2,0 mode: PR/PR res: [0x24000178a:0x389:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce537c expref: -99 pid: 154213 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.612329:0:154213:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6df81 lrc: 4/1,0 mode: PR/PR res: [0x24000178a:0x389:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce537c expref: -99 pid: 154213 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825950.612342:0:154213:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181d1b32680 x1874663042783104/t0(0) o101->lustre-MDT0001-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787825966 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787825950.612354:0:154213:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6df81 lrc: 3/1,0 mode: PR/PR res: [0x24000178a:0x389:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce537c expref: -99 pid: 154213 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.612367:0:154213:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d0f54800 type 13 mode 4 res 9663682442/905 (0 0) 00010000:00010000:0.0:1787825950.612382:0:154213:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6df81 lrc: 3/1,0 mode: PR/PR res: [0x24000178a:0x389:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce537c expref: -99 pid: 154213 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.612391:0:154213:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6df81 lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x389:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce537c expref: -99 pid: 154213 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787825950.612424:0:154213:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1b68:0x0], updating i_size 8192 00000080:00200000:0.0:1787825950.612429:0:154213:0:(namei.c:147:ll_iget()) got inode: [0x20001a9e3:0x1b68:0x0](ffff9181c83880c8): rc = 0 00000080:00010000:0.0:1787825950.612438:0:154213:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x1b68:0x0](ffff9181c83880c8) for lock 0xbef42c0221d6df73 00000080:00010000:0.0:1787825950.612455:0:154213:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x1b68:0x0](ffff9181c83880c8) 00000080:00010000:0.0:1787825950.612459:0:154213:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6df73 from it ffffb975049dbe08 00010000:00010000:0.0:1787825950.612463:0:154213:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6df73 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1b68:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5360 expref: -99 pid: 154213 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.612472:0:154213:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6df73 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1b68:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5360 expref: -99 pid: 154213 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787825950.612499:0:154213:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 3 00000080:00200000:0.0:1787825950.612507:0:154213:0:(namei.c:2328:ll_rmdir()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00010000:00010000:0.0:1787825950.612546:0:154213:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6df73 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1b68:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ce5360 expref: -99 pid: 154213 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825950.612593:0:154213:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6df73 lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x1b68:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce5360 expref: -99 pid: 154213 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825950.612601:0:154213:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787825950.612604:0:154213:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6df73 lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x1b68:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce5360 expref: -99 pid: 154213 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:0.0:1787825950.612629:0:154213:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787825950.612636:0:154213:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181d1b32680 x1874663042783360/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787825950.612657:0:154213:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d1b32680 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154213:1874663042783360:192.168.203.147@tcp:36:rmdir.0 00000100:00100000:0.0:1787825950.614074:0:154213:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8540 going to sleep for 11 seconds 00000100:00100000:1.0:1787825950.617367:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825950.617479:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048660992 00000100:00100000:0.0:1787825950.617489:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048660992 00000100:00100000:0.0:1787825950.617551:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9061 00000100:00100000:0.0:1787825950.617562:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b1180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6490:x1874663048660992:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825950.617580:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6df7a lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x389:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce536e expref: -99 pid: 154213 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825950.617708:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825950.617717:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b1180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6490:x1874663048660992:12345-192.168.203.147@tcp:104: Request processed in 157us (360us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825950.617725:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9061 00000100:00100000:1.0:1787825950.630322:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825950.630422:0:33333:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:0.0:1787825950.630440:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048661248 00000100:00100000:0.0:1787825950.630449:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048661248 00000100:00100000:0.0:1787825950.630507:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9062 00000100:00100000:0.0:1787825950.630528:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c32bc450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663048661248:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825950.630547:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6df81 lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x389:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce537c expref: -99 pid: 154213 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825950.630626:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825950.630633:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c32bc450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663048661248:12345-192.168.203.147@tcp:104: Request processed in 118us (322us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825950.630641:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9062 00000100:00100000:0.0:1787825950.651065:0:154213:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d1b32680 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154213:1874663042783360:192.168.203.147@tcp:36:rmdir.0 00000080:00200000:0.0:1787825950.651108:0:154213:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1b68:0x0](ffff9181c83880c8) 00000100:00100000:1.0:1787825950.766619:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825950.766774:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048662656 00000100:00100000:0.0:1787825950.766784:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048662656 00000100:00100000:0.0:1787825950.766831:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9063 00000100:00100000:0.0:1787825950.766842:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b1880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:14309:x1874663048662656:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825950.766857:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6200/0xbef42c0221d6dfa4 lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x38a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce5432 expref: -99 pid: 154215 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825950.766935:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825950.766942:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b1880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:14309:x1874663048662656:12345-192.168.203.147@tcp:104: Request processed in 103us (334us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825950.766950:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9063 00000100:00100000:0.0:1787825950.776306:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787825950.776495:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048662912 00000100:00100000:1.0:1787825950.776502:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048662912 00000100:00100000:1.0:1787825950.776562:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9064 00000100:00100000:1.0:1787825950.776581:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181d1b31f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663048662912:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787825950.776599:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6600/0xbef42c0221d6dfab lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x38a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce5440 expref: -99 pid: 154215 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825950.776674:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787825950.776682:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181d1b31f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663048662912:12345-192.168.203.147@tcp:104: Request processed in 111us (386us total) trans 0 rc 0/0 00000100:00100000:1.0:1787825950.776689:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9064 00000080:00200000:0.0:1787825950.821563:0:154216:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825950.822301:0:154216:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825950.822319:0:154216:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825950.822333:0:154216:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825950.822340:0:154216:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825950.822348:0:154216:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825950.822352:0:154216:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825950.822359:0:154216:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb975049ebb58 00010000:00010000:0.0:1787825950.822363:0:154216:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825950.822371:0:154216:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825950.822386:0:154216:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825950.822394:0:154216:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=29 00000080:00200000:0.0:1787825950.822400:0:154216:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825950.822412:0:154216:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=13 00000080:00200000:0.0:1787825950.822416:0:154216:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787825950.822428:0:154216:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787825950.822476:0:154216:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6dfb2 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154216 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.822484:0:154216:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6dfb2 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154216 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.822493:0:154216:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6dfb2 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154216 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825950.822528:0:154216:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d1b32a00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154216:1874663042785280:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787825950.823307:0:154216:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8840 going to sleep for 11 seconds 00000100:00100000:0.0:1787825950.829232:0:154216:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d1b32a00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154216:1874663042785280:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787825950.829260:0:154216:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6dfb2 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154216 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.829275:0:154216:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6dfb2 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154216 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.829284:0:154216:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6dfb2 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154216 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.829294:0:154216:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6dfb2 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154216 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.829303:0:154216:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6dfb2 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154216 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.829309:0:154216:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0513600 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787825950.829314:0:154216:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6dfb2 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154216 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.829322:0:154216:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6dfb2 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154216 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.829331:0:154216:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6dfb2 lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 154216 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.829339:0:154216:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787825950.829346:0:154216:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6dfb2 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154216 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.829353:0:154216:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6dfb2 lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154216 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825950.829372:0:154216:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181d1b32a00 x1874663042785280/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787825966 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787825950.829387:0:154216:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787825950.829415:0:154216:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787825950.829482:0:154216:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825950.829498:0:154216:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825950.829508:0:154216:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825950.829668:0:154216:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825950.829677:0:154216:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825950.829682:0:154216:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825950.829685:0:154216:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825950.829691:0:154216:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb975049ebbe8 00010000:00010000:0.0:1787825950.829694:0:154216:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825950.829703:0:154216:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825950.829717:0:154216:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825950.829724:0:154216:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=16408 00000080:00200000:0.0:1787825950.829729:0:154216:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825950.829741:0:154216:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=16392 00000080:00200000:0.0:1787825950.829784:0:154216:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787825950.829796:0:154216:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787825950.829834:0:154216:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6dfb9 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154216 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.829842:0:154216:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6dfb9 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154216 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.829850:0:154216:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6dfb9 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154216 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825950.829870:0:154216:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d1b32a00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154216:1874663042785408:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787825950.829938:0:154216:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8840 going to sleep for 11 seconds 00000100:00100000:0.0:1787825950.840073:0:154216:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d1b32a00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154216:1874663042785408:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787825950.840095:0:154216:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6dfb9 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154216 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.840109:0:154216:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6dfb9 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154216 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.840120:0:154216:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6dfb9 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154216 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.840127:0:154216:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6dfb9 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154216 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.840134:0:154216:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6dfb9 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154216 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.840151:0:154216:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0512600 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787825950.840157:0:154216:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6dfb9 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154216 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.840163:0:154216:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6dfb9 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154216 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.840170:0:154216:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6dfb9 lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 154216 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.840177:0:154216:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787825950.840184:0:154216:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6dfb9 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154216 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.840190:0:154216:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6dfb9 lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154216 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825950.840209:0:154216:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181d1b32a00 x1874663042785408/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787825966 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787825950.840223:0:154216:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787825950.840251:0:154216:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787825950.840321:0:154216:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825950.840336:0:154216:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825950.840345:0:154216:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825950.840355:0:154216:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825950.840361:0:154216:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825950.840366:0:154216:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825950.840368:0:154216:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825950.840373:0:154216:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb975049ebb58 00010000:00010000:0.0:1787825950.840376:0:154216:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825950.840384:0:154216:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825950.840396:0:154216:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825950.840402:0:154216:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=13 00000080:00200008:0.0:1787825950.840411:0:154216:0:(file.c:6296:ll_getattr_dentry()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1787825950.840416:0:154216:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08),name=d71a.replay-single 00000002:00010000:0.0:1787825950.840428:0:154216:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x20001a9e3:0x17e0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825950.840433:0:154216:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00010000:00010000:0.0:1787825950.840465:0:154216:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6dfc0 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 154216 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.840476:0:154216:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6dfc0 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 154216 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.840485:0:154216:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6dfc0 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 154216 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825950.840503:0:154216:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d1b32a00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154216:1874663042785536:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787825950.840596:0:154216:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8840 going to sleep for 11 seconds 00000100:00100000:0.0:1787825950.852469:0:154216:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d1b32a00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154216:1874663042785536:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787825950.852497:0:154216:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6dfc0 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce54a2 expref: -99 pid: 154216 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.852526:0:154216:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6dfc0 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce54a2 expref: -99 pid: 154216 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.852534:0:154216:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6dfc0 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce54a2 expref: -99 pid: 154216 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.852545:0:154216:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6dfc0 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce54a2 expref: -99 pid: 154216 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.852556:0:154216:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6dfc0 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce54a2 expref: -99 pid: 154216 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.852579:0:154216:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6dfc0 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce54a2 expref: -99 pid: 154216 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.852594:0:154216:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6dfc0 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce54a2 expref: -99 pid: 154216 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.852602:0:154216:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6dfc0 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce54a2 expref: -99 pid: 154216 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825950.852616:0:154216:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181d1b32a00 x1874663042785536/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787825966 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787825950.852630:0:154216:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6dfc0 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce54a2 expref: -99 pid: 154216 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.852641:0:154216:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787825950.852648:0:154216:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1787825950.852683:0:154216:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x17e0:0x0], updating i_size 4096 00000080:00010000:0.0:1787825950.852691:0:154216:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) for lock 0xbef42c0221d6dfc0 00000080:00010000:0.0:1787825950.852700:0:154216:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00000080:00010000:0.0:1787825950.852703:0:154216:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6dfc0 from it ffffb975049ebcb8 00010000:00010000:0.0:1787825950.852707:0:154216:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6dfc0 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce54a2 expref: -99 pid: 154216 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825950.852716:0:154216:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6dfc0 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce54a2 expref: -99 pid: 154216 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:0.0:1787825950.853348:0:154216:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1787825950.853355:0:154216:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825950.853371:0:154216:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825950.853385:0:154216:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825950.853397:0:154216:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825950.853404:0:154216:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825950.853409:0:154216:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825950.853412:0:154216:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825950.853417:0:154216:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb975049ebcb8 00010000:00010000:0.0:1787825950.853420:0:154216:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825950.853428:0:154216:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825950.853569:0:154216:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825950.853579:0:154216:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825950.853586:0:154216:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825950.853595:0:154216:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825950.853601:0:154216:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825950.853605:0:154216:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825950.853607:0:154216:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825950.853611:0:154216:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb975049ebbf8 00010000:00010000:0.0:1787825950.853615:0:154216:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825950.853622:0:154216:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825950.853632:0:154216:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825950.853638:0:154216:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=267 00000080:00200000:0.0:1787825950.853642:0:154216:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 44 00000080:00200008:0.0:1787825950.853647:0:154216:0:(file.c:1090:ll_file_open()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00000080:00200080:0.0:1787825950.853669:0:154216:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) cmd=c03066f0 arg=7fffaa719c80 00000080:00200000:0.0:1787825950.853682:0:154216:0:(dir.c:451:ll_dir_setdirstripe()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) name=test stripe_offset=0 stripe_count=2, hash_type=0 00010000:00010000:0.0:1787825950.853708:0:154216:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6dfc0 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ce54a2 expref: -99 pid: 154216 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825950.853723:0:154216:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6800/0xbef42c0221d64b3f lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cb9ffd expref: -99 pid: 152202 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825950.853816:0:154216:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6dfc0 lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce54a2 expref: -99 pid: 154216 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825950.853823:0:154216:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787825950.853826:0:154216:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6dfc0 lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce54a2 expref: -99 pid: 154216 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:0.0:1787825950.853842:0:154216:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787825950.853850:0:154216:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181d1b32a00 x1874663042785792/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 552/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787825950.853877:0:154216:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d1b32a00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154216:1874663042785792:192.168.203.147@tcp:36:lfs.0 00000100:00100000:0.0:1787825950.853950:0:154216:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8840 going to sleep for 11 seconds 00000100:00100000:0.0:1787825950.872929:0:154216:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d1b32a00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154216:1874663042785792:192.168.203.147@tcp:36:lfs.0 00000080:00200000:0.0:1787825950.872988:0:154216:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x20001a9e3:0x1b6a:0x0](ffff9181c837aa08) 00000080:00200000:0.0:1787825950.872995:0:154216:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1b6a:0x0], updating i_size 4096 00000080:00200008:0.0:1787825950.873030:0:154216:0:(file.c:479:ll_file_release()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00010000:00010000:0.0:1787825950.873060:0:154216:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513000/0xbef42c0221d64bee lrc: 2/0,0 mode: CR/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cba2b9 expref: -99 pid: 152210 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825950.877902:0:154217:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787825950.877933:0:154217:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787825950.877951:0:154217:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825950.877967:0:154217:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825950.877975:0:154217:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787825950.877982:0:154217:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787825950.877986:0:154217:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787825950.877994:0:154217:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97504913c20 00010000:00010000:1.0:1787825950.877998:0:154217:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825950.878007:0:154217:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825950.878022:0:154217:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787825950.878029:0:154217:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=24 00000080:00200000:1.0:1787825950.878035:0:154217:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:1.0:1787825950.878046:0:154217:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=0 00000080:00200000:1.0:1787825950.878050:0:154217:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:1.0:1787825950.878062:0:154217:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787825950.878108:0:154217:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6dfc7 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154217 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825950.878116:0:154217:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6dfc7 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154217 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825950.878125:0:154217:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6dfc7 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154217 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787825950.878155:0:154217:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154217:1874663042785920:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:1.0:1787825950.878248:0:154217:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b59e40 going to sleep for 11 seconds 00000100:00100000:1.0:1787825950.923732:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825950.923968:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048664448 00000100:00100000:0.0:1787825950.923978:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048664448 00000100:00100000:0.0:1787825950.924036:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9065 00000100:00100000:0.0:1787825950.924046:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:14309:x1874663048664448:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825950.924063:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480200/0xbef42c0221d6dfce lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x38b:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce54f6 expref: -99 pid: 154217 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825950.924140:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825950.924146:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:14309:x1874663048664448:12345-192.168.203.147@tcp:104: Request processed in 103us (425us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825950.924153:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9065 00000100:00100000:1.0:1787825950.937213:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825950.938271:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048664704 00000100:00100000:0.0:1787825950.938284:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048664704 00000100:00100000:0.0:1787825950.938353:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9066 00000100:00100000:0.0:1787825950.938364:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663048664704:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825950.938382:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481c00/0xbef42c0221d6dfd5 lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x38b:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce5504 expref: -99 pid: 154217 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825950.938466:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825950.938474:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663048664704:12345-192.168.203.147@tcp:104: Request processed in 112us (1270us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825950.938481:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9066 00000100:00100000:0.0:1787825951.064805:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787825951.064896:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048666112 00000100:00100000:1.0:1787825951.064905:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048666112 00000100:00100000:1.0:1787825951.064959:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9067 00000100:00100000:1.0:1787825951.064970:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c49a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:41299:x1874663048666112:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787825951.064987:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480200/0xbef42c0221d6dff8 lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x38c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce55ba expref: -99 pid: 154219 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825951.065060:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787825951.065067:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c49a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:41299:x1874663048666112:12345-192.168.203.147@tcp:104: Request processed in 99us (272us total) trans 0 rc 0/0 00000100:00100000:1.0:1787825951.065074:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9067 00000100:00100000:0.0:1787825951.072670:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787825951.072905:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048666368 00000100:00100000:1.0:1787825951.072915:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048666368 00000100:00100000:1.0:1787825951.072968:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9068 00000100:00100000:1.0:1787825951.072978:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c7107100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663048666368:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787825951.072995:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480400/0xbef42c0221d6dfff lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x38c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce55c8 expref: -99 pid: 154219 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825951.073070:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787825951.073077:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c7107100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663048666368:12345-192.168.203.147@tcp:104: Request processed in 100us (418us total) trans 0 rc 0/0 00000100:00100000:1.0:1787825951.073088:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9068 00000100:00100000:0.0:1787825951.169796:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787825951.169893:0:15151:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:1.0:1787825951.169912:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048667776 00000100:00100000:1.0:1787825951.169920:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048667776 00000100:00100000:1.0:1787825951.169975:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9069 00000100:00100000:1.0:1787825951.169986:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c32bd850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:7728:x1874663048667776:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787825951.170002:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480400/0xbef42c0221d6e022 lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x38d:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce567e expref: -99 pid: 154221 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825951.170069:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787825951.170075:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c32bd850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:7728:x1874663048667776:12345-192.168.203.147@tcp:104: Request processed in 92us (288us total) trans 0 rc 0/0 00000100:00100000:1.0:1787825951.170082:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9069 00000100:00100000:1.0:1787825951.177020:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825951.177100:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048668032 00000100:00100000:0.0:1787825951.177111:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048668032 00000100:00100000:0.0:1787825951.177168:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9070 00000100:00100000:0.0:1787825951.177177:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b1880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663048668032:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825951.177195:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480200/0xbef42c0221d6e029 lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x38d:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce568c expref: -99 pid: 154221 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825951.177266:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825951.177273:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b1880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663048668032:12345-192.168.203.147@tcp:104: Request processed in 97us (263us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825951.177279:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9070 00010000:00010000:0.0:1787825951.177300:0:33350:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787825951.177306:0:33350:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480200/0xbef42c0221d6e029 lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x38d:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce568c expref: -99 pid: 154221 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825951.177314:0:33350:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3480200 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787825951.177319:0:33350:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480200/0xbef42c0221d6e029 lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x38d:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00ce568c expref: -99 pid: 154221 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825951.177332:0:33350:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480200/0xbef42c0221d6e029 lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x38d:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00ce568c expref: -99 pid: 154221 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825951.177340:0:33350:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480200/0xbef42c0221d6e029 lrc: 3/0,0 mode: PR/PR res: [0x24000178a:0x38d:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x408400000000 nid: local remote: 0x878a5eea00ce568c expref: -99 pid: 154221 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825951.177390:0:33350:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480200/0xbef42c0221d6e029 lrc: 2/0,0 mode: --/PR res: [0x24000178a:0x38d:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ce568c expref: -99 pid: 154221 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825951.177397:0:33350:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787825951.177413:0:33350:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480200/0xbef42c0221d6e029 lrc: 1/0,0 mode: --/PR res: [0x24000178a:0x38d:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ce568c expref: -99 pid: 154221 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825951.177421:0:33350:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480200/0xbef42c0221d6e029 lrc: 0/0,0 mode: --/PR res: [0x24000178a:0x38d:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ce568c expref: -99 pid: 154221 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825951.177443:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787825951.177445:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787825951.177447:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787825951.177459:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c7105c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042789760:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1787825951.183454:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c7105c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042789760:192.168.203.147@tcp:103:ldlm_bl.0 00000080:00200000:1.0:1787825951.209111:0:154222:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787825951.209412:0:154222:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787825951.209431:0:154222:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825951.209448:0:154222:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825951.209455:0:154222:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787825951.209463:0:154222:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787825951.209466:0:154222:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787825951.209474:0:154222:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97504a0bb58 00010000:00010000:1.0:1787825951.209479:0:154222:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825951.209488:0:154222:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825951.209502:0:154222:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787825951.209509:0:154222:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=29 00000080:00200000:1.0:1787825951.209526:0:154222:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:1.0:1787825951.209538:0:154222:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=13 00000080:00200000:1.0:1787825951.209543:0:154222:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:1.0:1787825951.209557:0:154222:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787825951.209685:0:154222:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6e030 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154222 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825951.209693:0:154222:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6e030 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154222 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825951.209781:0:154222:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6e030 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154222 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787825951.209806:0:154222:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154222:1874663042789888:192.168.203.147@tcp:101:lfs.0 00000100:00100000:1.0:1787825951.209966:0:154222:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b59e40 going to sleep for 11 seconds 00000100:00100000:1.0:1787825951.215302:0:154222:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154222:1874663042789888:192.168.203.147@tcp:101:lfs.0 00010000:00010000:1.0:1787825951.215326:0:154222:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6e030 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154222 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825951.215339:0:154222:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6e030 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154222 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825951.215351:0:154222:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6e030 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154222 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825951.215358:0:154222:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6e030 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154222 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825951.215368:0:154222:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6e030 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154222 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825951.215375:0:154222:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4a00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:1.0:1787825951.215379:0:154222:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6e030 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154222 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825951.215386:0:154222:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6e030 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154222 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825951.215393:0:154222:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6e030 lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 154222 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825951.215400:0:154222:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787825951.215407:0:154222:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6e030 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154222 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825951.215413:0:154222:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6e030 lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154222 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787825951.215432:0:154222:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c35b2a00 x1874663042789888/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787825967 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787825951.215446:0:154222:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:1.0:1787825951.215475:0:154222:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:1.0:1787825951.215560:0:154222:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787825951.215652:0:154222:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787825951.215663:0:154222:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825951.215675:0:154222:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825951.215681:0:154222:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787825951.215686:0:154222:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787825951.215689:0:154222:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787825951.216059:0:154222:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97504a0bbe8 00010000:00010000:1.0:1787825951.216066:0:154222:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825951.216075:0:154222:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825951.216090:0:154222:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787825951.216097:0:154222:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=16408 00000080:00200000:1.0:1787825951.216103:0:154222:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:1.0:1787825951.216116:0:154222:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=16392 00000080:00200000:1.0:1787825951.216119:0:154222:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:1.0:1787825951.216131:0:154222:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787825951.216247:0:154222:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5800/0xbef42c0221d6e037 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154222 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825951.216255:0:154222:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5800/0xbef42c0221d6e037 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154222 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825951.216264:0:154222:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5800/0xbef42c0221d6e037 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154222 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787825951.216282:0:154222:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154222:1874663042790016:192.168.203.147@tcp:101:lfs.0 00000100:00100000:1.0:1787825951.216398:0:154222:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b59e40 going to sleep for 11 seconds 00000100:00100000:0.0:1787825951.246622:0:154223:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154223:1874663042790528:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:0.0:1787825951.246648:0:154223:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7600/0xbef42c0221d6e045 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5734 expref: -99 pid: 154223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.246675:0:154223:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7600/0xbef42c0221d6e045 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1b6d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5734 expref: -99 pid: 154223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.246684:0:154223:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7600/0xbef42c0221d6e045 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1b6d:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5734 expref: -99 pid: 154223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.246695:0:154223:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7600/0xbef42c0221d6e045 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1b6d:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5734 expref: -99 pid: 154223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.246706:0:154223:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7600/0xbef42c0221d6e045 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1b6d:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5734 expref: -99 pid: 154223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.246719:0:154223:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7600/0xbef42c0221d6e045 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1b6d:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5734 expref: -99 pid: 154223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.246728:0:154223:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7600/0xbef42c0221d6e045 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1b6d:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5734 expref: -99 pid: 154223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.246736:0:154223:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7600/0xbef42c0221d6e045 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1b6d:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5734 expref: -99 pid: 154223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825951.246768:0:154223:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c5634e00 x1874663042790528/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/672 e 0 to 0 dl 1787825967 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787825951.246783:0:154223:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7600/0xbef42c0221d6e045 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1b6d:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5734 expref: -99 pid: 154223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.246799:0:154223:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/7021 (0 0) 00000080:00200000:0.0:1787825951.246804:0:154223:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00000002:00010000:0.0:1787825951.246888:0:154223:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200025d90:0x38e:0x0]) in obj [0x200025d90:0x38e:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825951.246935:0:154223:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e04c lrc: 3/1,0 mode: --/CR res: [0x200025d90:0x38e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 154223 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.246943:0:154223:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e04c lrc: 3/1,0 mode: --/CR res: [0x200025d90:0x38e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154223 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.246951:0:154223:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e04c lrc: 3/1,0 mode: --/CR res: [0x200025d90:0x38e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154223 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825951.246972:0:154223:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c7105c00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154223:1874663042790656:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787825951.247059:0:154223:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8d80 going to sleep for 11 seconds 00000100:00100000:0.0:1787825951.260733:0:154223:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c7105c00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154223:1874663042790656:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:0.0:1787825951.260799:0:154223:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e04c lrc: 4/1,0 mode: --/CR res: [0x200025d90:0x38e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5742 expref: -99 pid: 154223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.260811:0:154223:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e04c lrc: 4/1,0 mode: --/PR res: [0x200025d90:0x38e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5742 expref: -99 pid: 154223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.260823:0:154223:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e04c lrc: 4/1,0 mode: PR/PR res: [0x200025d90:0x38e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5742 expref: -99 pid: 154223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.260834:0:154223:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e04c lrc: 4/1,0 mode: PR/PR res: [0x200025d90:0x38e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5742 expref: -99 pid: 154223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.260851:0:154223:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e04c lrc: 5/2,0 mode: PR/PR res: [0x200025d90:0x38e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5742 expref: -99 pid: 154223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.260859:0:154223:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e04c lrc: 5/2,0 mode: PR/PR res: [0x200025d90:0x38e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5742 expref: -99 pid: 154223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.260867:0:154223:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e04c lrc: 4/1,0 mode: PR/PR res: [0x200025d90:0x38e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5742 expref: -99 pid: 154223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825951.260880:0:154223:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c7105c00 x1874663042790656/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787825967 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787825951.260893:0:154223:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e04c lrc: 3/1,0 mode: PR/PR res: [0x200025d90:0x38e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5742 expref: -99 pid: 154223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.260904:0:154223:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590089616/910 (0 0) 00010000:00010000:0.0:1787825951.260911:0:154223:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e04c lrc: 3/1,0 mode: PR/PR res: [0x200025d90:0x38e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5742 expref: -99 pid: 154223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.260920:0:154223:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e04c lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x38e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5742 expref: -99 pid: 154223 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000002:00010000:0.0:1787825951.260954:0:154223:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x24000178a:0x38e:0x0]) in obj [0x24000178a:0x38e:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825951.260999:0:154223:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e053 lrc: 3/1,0 mode: --/CR res: [0x24000178a:0x38e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 154223 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.261006:0:154223:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e053 lrc: 3/1,0 mode: --/CR res: [0x24000178a:0x38e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154223 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.261015:0:154223:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e053 lrc: 3/1,0 mode: --/CR res: [0x24000178a:0x38e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154223 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825951.261033:0:154223:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c7105c00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154223:1874663042790784:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787825951.261098:0:154223:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8d80 going to sleep for 11 seconds 00000100:00100000:1.0:1787825951.288601:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825951.288712:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048669440 00000100:00100000:0.0:1787825951.288723:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048669440 00000100:00100000:0.0:1787825951.288815:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9071 00000100:00100000:0.0:1787825951.288826:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:14309:x1874663048669440:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825951.288842:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e04c lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x38e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce5742 expref: -99 pid: 154223 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825951.288913:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825951.288921:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:14309:x1874663048669440:12345-192.168.203.147@tcp:104: Request processed in 97us (330us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825951.288932:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9071 00000100:00100000:1.0:1787825951.299358:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825951.299559:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048669696 00000100:00100000:0.0:1787825951.299575:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048669696 00000100:00100000:0.0:1787825951.299626:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9072 00000100:00100000:0.0:1787825951.299637:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b1500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8414:x1874663048669696:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825951.299654:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e053 lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x38e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce5750 expref: -99 pid: 154223 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825951.299725:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825951.299732:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b1500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8414:x1874663048669696:12345-192.168.203.147@tcp:104: Request processed in 97us (383us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825951.299774:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9072 00000100:00100000:1.0:1787825951.438935:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825951.439375:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048671104 00000100:00100000:0.0:1787825951.439386:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048671104 00000100:00100000:0.0:1787825951.439536:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9073 00000100:00100000:0.0:1787825951.439547:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b3100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:41299:x1874663048671104:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825951.439623:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480400/0xbef42c0221d6e076 lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x38f:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce5806 expref: -99 pid: 154226 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825951.439773:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825951.439780:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b3100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:41299:x1874663048671104:12345-192.168.203.147@tcp:104: Request processed in 235us (853us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825951.439788:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9073 00000100:00100000:1.0:1787825951.446165:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825951.446468:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048671360 00000100:00100000:0.0:1787825951.446478:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048671360 00000100:00100000:0.0:1787825951.446546:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9074 00000100:00100000:0.0:1787825951.446557:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663048671360:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825951.446585:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481c00/0xbef42c0221d6e07d lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x38f:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce5814 expref: -99 pid: 154226 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825951.446665:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825951.446672:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663048671360:12345-192.168.203.147@tcp:104: Request processed in 117us (514us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825951.446680:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9074 00000080:00200000:0.0:1787825951.497399:0:154227:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x17e0:0x0], updating i_size 4096 00000080:00010000:0.0:1787825951.497408:0:154227:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) for lock 0xbef42c0221d6e092 00000080:00010000:0.0:1787825951.497416:0:154227:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00000080:00010000:0.0:1787825951.497420:0:154227:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6e092 from it ffffb97504a23cb8 00010000:00010000:0.0:1787825951.497424:0:154227:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480c00/0xbef42c0221d6e092 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5876 expref: -99 pid: 154227 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.497433:0:154227:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480c00/0xbef42c0221d6e092 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5876 expref: -99 pid: 154227 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:0.0:1787825951.499133:0:154227:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1787825951.499217:0:154227:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825951.499239:0:154227:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825951.499254:0:154227:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825951.499269:0:154227:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825951.499276:0:154227:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825951.499284:0:154227:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825951.499287:0:154227:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825951.499294:0:154227:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97504a23cb8 00010000:00010000:0.0:1787825951.499298:0:154227:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825951.499306:0:154227:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825951.499461:0:154227:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825951.499470:0:154227:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825951.499479:0:154227:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825951.499488:0:154227:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825951.499495:0:154227:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825951.499499:0:154227:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825951.499501:0:154227:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825951.499505:0:154227:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97504a23bf8 00010000:00010000:0.0:1787825951.499508:0:154227:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825951.499526:0:154227:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825951.499535:0:154227:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825951.499542:0:154227:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=267 00000080:00200000:0.0:1787825951.499548:0:154227:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 44 00000080:00200008:0.0:1787825951.499555:0:154227:0:(file.c:1090:ll_file_open()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00000080:00200080:0.0:1787825951.499919:0:154227:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) cmd=c03066f0 arg=7ffc03e13ea0 00000080:00200000:0.0:1787825951.499934:0:154227:0:(dir.c:451:ll_dir_setdirstripe()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) name=test stripe_offset=0 stripe_count=2, hash_type=0 00010000:00010000:0.0:1787825951.499961:0:154227:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480c00/0xbef42c0221d6e092 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ce5876 expref: -99 pid: 154227 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825951.499978:0:154227:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6800/0xbef42c0221d64b3f lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cb9ffd expref: -99 pid: 152202 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825951.500041:0:154227:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480c00/0xbef42c0221d6e092 lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce5876 expref: -99 pid: 154227 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825951.500048:0:154227:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787825951.500051:0:154227:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480c00/0xbef42c0221d6e092 lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce5876 expref: -99 pid: 154227 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:0.0:1787825951.500070:0:154227:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787825951.500083:0:154227:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c7106d80 x1874663042793472/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 552/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787825951.500106:0:154227:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c7106d80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154227:1874663042793472:192.168.203.147@tcp:36:lfs.0 00000100:00100000:0.0:1787825951.500197:0:154227:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787825951.509388:0:154227:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c7106d80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154227:1874663042793472:192.168.203.147@tcp:36:lfs.0 00000080:00200000:0.0:1787825951.509440:0:154227:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x20001a9e3:0x1b6f:0x0](ffff9181c83a4b08) 00000080:00200000:0.0:1787825951.509448:0:154227:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1b6f:0x0], updating i_size 4096 00000080:00200008:0.0:1787825951.509483:0:154227:0:(file.c:479:ll_file_release()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00010000:00010000:0.0:1787825951.509524:0:154227:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513000/0xbef42c0221d64bee lrc: 2/0,0 mode: CR/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cba2b9 expref: -99 pid: 152210 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000100:00100000:1.0:1787825951.539672:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825951.539933:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048672768 00000100:00100000:0.0:1787825951.539946:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048672768 00000100:00100000:1.0:1787825951.540090:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9075 00000100:00100000:1.0:1787825951.540100:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6490:x1874663048672768:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787825951.540116:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7c00/0xbef42c0221d6e0a0 lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x390:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce58ca expref: -99 pid: 154228 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825951.540205:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787825951.540212:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6490:x1874663048672768:12345-192.168.203.147@tcp:104: Request processed in 114us (551us total) trans 0 rc 0/0 00000100:00100000:1.0:1787825951.540218:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9075 00000100:00100000:1.0:1787825951.547160:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825951.547254:0:15151:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:0.0:1787825951.547276:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048673024 00000100:00100000:0.0:1787825951.547285:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048673024 00000100:00100000:0.0:1787825951.547342:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9076 00000100:00100000:0.0:1787825951.547352:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c440bc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8414:x1874663048673024:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825951.547371:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f7400/0xbef42c0221d6e0a7 lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x390:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce58d8 expref: -99 pid: 154228 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825951.547448:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825951.547455:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c440bc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8414:x1874663048673024:12345-192.168.203.147@tcp:104: Request processed in 105us (307us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825951.547463:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9076 00000100:00100000:0.0:1787825951.679001:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787825951.679115:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048674432 00000100:00100000:1.0:1787825951.679126:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048674432 00000100:00100000:1.0:1787825951.679185:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9077 00000100:00100000:1.0:1787825951.679195:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c7107480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:7728:x1874663048674432:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787825951.679211:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7e00/0xbef42c0221d6e0ca lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x391:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce598e expref: -99 pid: 154230 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825951.679286:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787825951.679299:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c7107480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:7728:x1874663048674432:12345-192.168.203.147@tcp:104: Request processed in 106us (309us total) trans 0 rc 0/0 00000100:00100000:1.0:1787825951.679306:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9077 00000100:00100000:0.0:1787825951.692198:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787825951.692277:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048674688 00000100:00100000:1.0:1787825951.692284:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048674688 00000100:00100000:1.0:1787825951.692339:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9078 00000100:00100000:1.0:1787825951.692348:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c7105500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663048674688:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787825951.692364:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f7c00/0xbef42c0221d6e0d1 lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x391:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce599c expref: -99 pid: 154230 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825951.692435:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787825951.692441:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c7105500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663048674688:12345-192.168.203.147@tcp:104: Request processed in 94us (253us total) trans 0 rc 0/0 00000100:00100000:1.0:1787825951.692448:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9078 00000100:00100000:1.0:1787825951.812082:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825951.812246:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048676096 00000100:00100000:0.0:1787825951.812258:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048676096 00000100:00100000:0.0:1787825951.812353:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9079 00000100:00100000:0.0:1787825951.812364:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b2680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6491:x1874663048676096:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825951.812381:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481c00/0xbef42c0221d6e0f4 lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x392:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce5a52 expref: -99 pid: 154232 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825951.812461:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825951.812468:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b2680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6491:x1874663048676096:12345-192.168.203.147@tcp:104: Request processed in 107us (397us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825951.812476:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9079 00000100:00100000:0.0:1787825951.820371:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787825951.820480:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048676352 00000100:00100000:1.0:1787825951.820490:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048676352 00000100:00100000:1.0:1787825951.820569:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9080 00000100:00100000:1.0:1787825951.820580:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c7105c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8414:x1874663048676352:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787825951.820597:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f7c00/0xbef42c0221d6e0fb lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x392:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce5a60 expref: -99 pid: 154232 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825951.820677:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787825951.820684:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c7105c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8414:x1874663048676352:12345-192.168.203.147@tcp:104: Request processed in 106us (323us total) trans 0 rc 0/0 00000100:00100000:1.0:1787825951.820692:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9080 00000100:00100000:0.0:1787825951.852033:0:154233:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636300 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154233:1874663042797568:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787825951.852054:0:154233:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7c00/0xbef42c0221d6e102 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154233 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.852067:0:154233:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7c00/0xbef42c0221d6e102 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154233 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.852075:0:154233:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7c00/0xbef42c0221d6e102 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154233 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.852082:0:154233:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7c00/0xbef42c0221d6e102 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154233 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.852090:0:154233:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7c00/0xbef42c0221d6e102 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154233 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.852096:0:154233:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e06f7c00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787825951.852100:0:154233:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7c00/0xbef42c0221d6e102 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154233 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.852108:0:154233:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7c00/0xbef42c0221d6e102 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154233 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.852115:0:154233:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7c00/0xbef42c0221d6e102 lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 154233 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.852122:0:154233:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787825951.852129:0:154233:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7c00/0xbef42c0221d6e102 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154233 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.852146:0:154233:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7c00/0xbef42c0221d6e102 lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154233 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825951.852166:0:154233:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c5636300 x1874663042797568/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787825967 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787825951.852180:0:154233:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787825951.852205:0:154233:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787825951.852274:0:154233:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825951.852288:0:154233:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825951.852296:0:154233:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825951.852306:0:154233:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825951.852313:0:154233:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825951.852319:0:154233:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825951.852322:0:154233:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825951.852327:0:154233:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97504a33be8 00010000:00010000:0.0:1787825951.852330:0:154233:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825951.852338:0:154233:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825951.852351:0:154233:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825951.852358:0:154233:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=16408 00000080:00200000:0.0:1787825951.852364:0:154233:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825951.852376:0:154233:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=16392 00000080:00200000:0.0:1787825951.852380:0:154233:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787825951.852392:0:154233:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787825951.852428:0:154233:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e109 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154233 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.852436:0:154233:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e109 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154233 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.852445:0:154233:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e109 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154233 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825951.852462:0:154233:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c7107100 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154233:1874663042797696:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787825951.852557:0:154233:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8900 going to sleep for 11 seconds 00000100:00100000:0.0:1787825951.856056:0:154233:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c7107100 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154233:1874663042797696:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787825951.856079:0:154233:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e109 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154233 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.856091:0:154233:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e109 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154233 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.856098:0:154233:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e109 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154233 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.856105:0:154233:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e109 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154233 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.856115:0:154233:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e109 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154233 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.856120:0:154233:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0512800 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787825951.856125:0:154233:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e109 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154233 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.856131:0:154233:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e109 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154233 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.856148:0:154233:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e109 lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 154233 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.856156:0:154233:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787825951.856163:0:154233:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e109 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154233 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.856169:0:154233:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e109 lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154233 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825951.856187:0:154233:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c7107100 x1874663042797696/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787825967 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787825951.856202:0:154233:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787825951.856227:0:154233:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787825951.856297:0:154233:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825951.856311:0:154233:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825951.856320:0:154233:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825951.856330:0:154233:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825951.856337:0:154233:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825951.856342:0:154233:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825951.856345:0:154233:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825951.856350:0:154233:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97504a33b58 00010000:00010000:0.0:1787825951.856354:0:154233:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825951.856362:0:154233:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825951.856375:0:154233:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825951.856383:0:154233:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=13 00000080:00200008:0.0:1787825951.856391:0:154233:0:(file.c:6296:ll_getattr_dentry()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1787825951.856395:0:154233:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08),name=d71a.replay-single 00000002:00010000:0.0:1787825951.856405:0:154233:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x20001a9e3:0x17e0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825951.856410:0:154233:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00010000:00010000:0.0:1787825951.856441:0:154233:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e110 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 154233 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.856447:0:154233:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e110 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 154233 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.856456:0:154233:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e110 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 154233 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825951.856474:0:154233:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c7107100 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154233:1874663042797824:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787825951.856544:0:154233:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8900 going to sleep for 11 seconds 00000100:00100000:0.0:1787825951.861053:0:154233:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c7107100 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154233:1874663042797824:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787825951.861079:0:154233:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e110 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5ac2 expref: -99 pid: 154233 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.861095:0:154233:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e110 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5ac2 expref: -99 pid: 154233 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.861103:0:154233:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e110 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5ac2 expref: -99 pid: 154233 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.861118:0:154233:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e110 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5ac2 expref: -99 pid: 154233 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.861128:0:154233:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e110 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5ac2 expref: -99 pid: 154233 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.861153:0:154233:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e110 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5ac2 expref: -99 pid: 154233 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.861160:0:154233:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e110 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5ac2 expref: -99 pid: 154233 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.861166:0:154233:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e110 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5ac2 expref: -99 pid: 154233 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825951.861177:0:154233:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c7107100 x1874663042797824/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787825967 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787825951.861190:0:154233:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e110 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5ac2 expref: -99 pid: 154233 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.861200:0:154233:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787825951.861207:0:154233:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1787825951.861235:0:154233:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x17e0:0x0], updating i_size 4096 00000080:00010000:0.0:1787825951.861242:0:154233:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) for lock 0xbef42c0221d6e110 00000080:00010000:0.0:1787825951.861250:0:154233:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00000080:00010000:0.0:1787825951.861253:0:154233:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6e110 from it ffffb97504a33cb8 00010000:00010000:0.0:1787825951.861257:0:154233:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e110 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5ac2 expref: -99 pid: 154233 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.861265:0:154233:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e110 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5ac2 expref: -99 pid: 154233 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:0.0:1787825951.862010:0:154233:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1787825951.862019:0:154233:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825951.862037:0:154233:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825951.862050:0:154233:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825951.862063:0:154233:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825951.862071:0:154233:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825951.862076:0:154233:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825951.862080:0:154233:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825951.862084:0:154233:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97504a33cb8 00010000:00010000:0.0:1787825951.862088:0:154233:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825951.862096:0:154233:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825951.862226:0:154233:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825951.862234:0:154233:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825951.862241:0:154233:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825951.862250:0:154233:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825951.862255:0:154233:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825951.862259:0:154233:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825951.862261:0:154233:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825951.862265:0:154233:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97504a33bf8 00010000:00010000:0.0:1787825951.862267:0:154233:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825951.862274:0:154233:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825951.862282:0:154233:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825951.862288:0:154233:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=267 00000080:00200000:0.0:1787825951.862292:0:154233:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 44 00000080:00200008:0.0:1787825951.862298:0:154233:0:(file.c:1090:ll_file_open()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00000080:00200080:0.0:1787825951.862317:0:154233:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) cmd=c03066f0 arg=7ffee81f60f0 00000080:00200000:0.0:1787825951.862328:0:154233:0:(dir.c:451:ll_dir_setdirstripe()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) name=test stripe_offset=0 stripe_count=2, hash_type=0 00010000:00010000:0.0:1787825951.862351:0:154233:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e110 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ce5ac2 expref: -99 pid: 154233 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825951.862366:0:154233:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6800/0xbef42c0221d64b3f lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cb9ffd expref: -99 pid: 152202 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825951.862406:0:154233:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e110 lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce5ac2 expref: -99 pid: 154233 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825951.862415:0:154233:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787825951.862417:0:154233:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e110 lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce5ac2 expref: -99 pid: 154233 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:0.0:1787825951.862435:0:154233:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787825951.862445:0:154233:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c7107100 x1874663042798080/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 552/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787825951.862467:0:154233:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c7107100 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154233:1874663042798080:192.168.203.147@tcp:36:lfs.0 00000100:00100000:0.0:1787825951.862555:0:154233:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8900 going to sleep for 11 seconds 00000100:00100000:0.0:1787825951.874505:0:154233:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c7107100 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154233:1874663042798080:192.168.203.147@tcp:36:lfs.0 00000080:00200000:0.0:1787825951.874582:0:154233:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x20001a9e3:0x1b72:0x0](ffff9181c83a0908) 00000080:00200000:0.0:1787825951.874589:0:154233:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1b72:0x0], updating i_size 4096 00000080:00200008:0.0:1787825951.874625:0:154233:0:(file.c:479:ll_file_release()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00010000:00010000:0.0:1787825951.874654:0:154233:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513000/0xbef42c0221d64bee lrc: 2/0,0 mode: CR/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cba2b9 expref: -99 pid: 152210 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000100:00100000:1.0:1787825951.905664:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825951.905740:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048677760 00000100:00100000:0.0:1787825951.905786:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048677760 00000100:00100000:0.0:1787825951.905833:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9081 00000100:00100000:0.0:1787825951.905843:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b0380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6490:x1874663048677760:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825951.905860:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7e00/0xbef42c0221d6e11e lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x393:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce5b16 expref: -99 pid: 154234 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825951.905934:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825951.905940:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b0380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6490:x1874663048677760:12345-192.168.203.147@tcp:104: Request processed in 99us (285us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825951.905948:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9081 00000100:00100000:1.0:1787825951.911108:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825951.911232:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048678016 00000100:00100000:0.0:1787825951.911242:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048678016 00000100:00100000:0.0:1787825951.911301:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9082 00000100:00100000:0.0:1787825951.911311:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663048678016:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825951.911328:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f7400/0xbef42c0221d6e125 lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x393:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce5b24 expref: -99 pid: 154234 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825951.911401:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825951.911407:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663048678016:12345-192.168.203.147@tcp:104: Request processed in 98us (309us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825951.911415:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9082 00000100:00100000:0.0:1787825951.991814:0:154236:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c4687b80 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154236:1874663042799744:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:0.0:1787825951.991842:0:154236:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481c00/0xbef42c0221d6e141 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5bcc expref: -99 pid: 154236 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.991869:0:154236:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481c00/0xbef42c0221d6e141 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1b73:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5bcc expref: -99 pid: 154236 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.991877:0:154236:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481c00/0xbef42c0221d6e141 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1b73:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5bcc expref: -99 pid: 154236 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.991901:0:154236:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481c00/0xbef42c0221d6e141 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1b73:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5bcc expref: -99 pid: 154236 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.991912:0:154236:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481c00/0xbef42c0221d6e141 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1b73:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5bcc expref: -99 pid: 154236 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.991925:0:154236:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481c00/0xbef42c0221d6e141 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1b73:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5bcc expref: -99 pid: 154236 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.991933:0:154236:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481c00/0xbef42c0221d6e141 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1b73:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5bcc expref: -99 pid: 154236 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.991941:0:154236:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481c00/0xbef42c0221d6e141 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1b73:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5bcc expref: -99 pid: 154236 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825951.991953:0:154236:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c4687b80 x1874663042799744/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/672 e 0 to 0 dl 1787825967 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787825951.991966:0:154236:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481c00/0xbef42c0221d6e141 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1b73:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5bcc expref: -99 pid: 154236 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.991975:0:154236:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/7027 (0 0) 00000080:00200000:0.0:1787825951.991981:0:154236:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00000002:00010000:0.0:1787825951.992034:0:154236:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200025d90:0x394:0x0]) in obj [0x200025d90:0x394:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825951.992081:0:154236:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e148 lrc: 3/1,0 mode: --/CR res: [0x200025d90:0x394:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 154236 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.992089:0:154236:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e148 lrc: 3/1,0 mode: --/CR res: [0x200025d90:0x394:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154236 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825951.992098:0:154236:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e148 lrc: 3/1,0 mode: --/CR res: [0x200025d90:0x394:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154236 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825951.992116:0:154236:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c33fca80 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154236:1874663042799872:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787825951.992208:0:154236:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8000 going to sleep for 11 seconds 00000100:00100000:0.0:1787825952.000415:0:154236:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c33fca80 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154236:1874663042799872:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:0.0:1787825952.000442:0:154236:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e148 lrc: 4/1,0 mode: --/CR res: [0x200025d90:0x394:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5bda expref: -99 pid: 154236 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825952.000454:0:154236:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e148 lrc: 4/1,0 mode: --/PR res: [0x200025d90:0x394:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5bda expref: -99 pid: 154236 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825952.000467:0:154236:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e148 lrc: 4/1,0 mode: PR/PR res: [0x200025d90:0x394:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5bda expref: -99 pid: 154236 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825952.000478:0:154236:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e148 lrc: 4/1,0 mode: PR/PR res: [0x200025d90:0x394:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5bda expref: -99 pid: 154236 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825952.000491:0:154236:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e148 lrc: 5/2,0 mode: PR/PR res: [0x200025d90:0x394:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5bda expref: -99 pid: 154236 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825952.000498:0:154236:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e148 lrc: 5/2,0 mode: PR/PR res: [0x200025d90:0x394:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5bda expref: -99 pid: 154236 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825952.000505:0:154236:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e148 lrc: 4/1,0 mode: PR/PR res: [0x200025d90:0x394:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5bda expref: -99 pid: 154236 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825952.000528:0:154236:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c33fca80 x1874663042799872/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787825967 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787825952.000541:0:154236:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e148 lrc: 3/1,0 mode: PR/PR res: [0x200025d90:0x394:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5bda expref: -99 pid: 154236 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825952.000688:0:154236:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590089616/916 (0 0) 00010000:00010000:0.0:1787825952.000700:0:154236:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e148 lrc: 3/1,0 mode: PR/PR res: [0x200025d90:0x394:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5bda expref: -99 pid: 154236 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825952.000708:0:154236:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e148 lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x394:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5bda expref: -99 pid: 154236 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000002:00010000:0.0:1787825952.000738:0:154236:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x24000178a:0x394:0x0]) in obj [0x24000178a:0x394:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825952.000989:0:154236:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e14f lrc: 3/1,0 mode: --/CR res: [0x24000178a:0x394:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 154236 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825952.000997:0:154236:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e14f lrc: 3/1,0 mode: --/CR res: [0x24000178a:0x394:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154236 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825952.001006:0:154236:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e14f lrc: 3/1,0 mode: --/CR res: [0x24000178a:0x394:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154236 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825952.001025:0:154236:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c33fca80 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154236:1874663042800000:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787825952.001163:0:154236:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8000 going to sleep for 10 seconds 00000080:00010000:0.0:1787825952.019334:0:154236:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x1b73:0x0](ffff9181c83aaa08) 00000080:00010000:0.0:1787825952.019341:0:154236:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6e141 from it ffffb97504a5be08 00010000:00010000:0.0:1787825952.019346:0:154236:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481c00/0xbef42c0221d6e141 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1b73:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5bcc expref: -99 pid: 154236 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825952.019355:0:154236:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481c00/0xbef42c0221d6e141 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1b73:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5bcc expref: -99 pid: 154236 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787825952.019388:0:154236:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 3 00000080:00200000:0.0:1787825952.019396:0:154236:0:(namei.c:2328:ll_rmdir()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00010000:00010000:0.0:1787825952.019425:0:154236:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481c00/0xbef42c0221d6e141 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1b73:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ce5bcc expref: -99 pid: 154236 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825952.019471:0:154236:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481c00/0xbef42c0221d6e141 lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x1b73:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce5bcc expref: -99 pid: 154236 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825952.019478:0:154236:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787825952.019481:0:154236:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481c00/0xbef42c0221d6e141 lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x1b73:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce5bcc expref: -99 pid: 154236 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:0.0:1787825952.019506:0:154236:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787825952.019524:0:154236:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c33fca80 x1874663042800256/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787825952.019547:0:154236:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c33fca80 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154236:1874663042800256:192.168.203.147@tcp:36:rmdir.0 00000100:00100000:0.0:1787825952.019664:0:154236:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8a80 going to sleep for 11 seconds 00000100:00100000:0.0:1787825952.027517:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787825952.027599:0:15150:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:1.0:1787825952.027619:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048679424 00000100:00100000:1.0:1787825952.027628:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048679424 00000100:00100000:1.0:1787825952.027710:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9083 00000100:00100000:1.0:1787825952.027721:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c5cfd850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:14309:x1874663048679424:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787825952.027738:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e148 lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x394:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce5bda expref: -99 pid: 154236 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825952.027848:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787825952.027855:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c5cfd850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:14309:x1874663048679424:12345-192.168.203.147@tcp:104: Request processed in 136us (358us total) trans 0 rc 0/0 00000100:00100000:1.0:1787825952.027862:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9083 00000100:00100000:1.0:1787825952.037696:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825952.037952:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048679680 00000100:00100000:0.0:1787825952.037962:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048679680 00000100:00100000:0.0:1787825952.038095:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9084 00000100:00100000:0.0:1787825952.038106:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b2680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8414:x1874663048679680:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825952.038120:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e14f lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x394:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce5be8 expref: -99 pid: 154236 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825952.038199:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825952.038207:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b2680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8414:x1874663048679680:12345-192.168.203.147@tcp:104: Request processed in 104us (517us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825952.038213:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9084 00000100:00100000:0.0:1787825952.053610:0:154236:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c33fca80 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154236:1874663042800256:192.168.203.147@tcp:36:rmdir.0 00000080:00200000:0.0:1787825952.053658:0:154236:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1b73:0x0](ffff9181c83aaa08) 00000080:00200000:1.0:1787825952.071921:0:154237:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787825952.071951:0:154237:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787825952.071970:0:154237:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825952.071986:0:154237:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825952.071995:0:154237:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787825952.072004:0:154237:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787825952.072007:0:154237:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787825952.072015:0:154237:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97504913b58 00010000:00010000:1.0:1787825952.072019:0:154237:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825952.072028:0:154237:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825952.072045:0:154237:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787825952.072053:0:154237:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=29 00000080:00200000:1.0:1787825952.072058:0:154237:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:1.0:1787825952.072069:0:154237:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=13 00000080:00200000:1.0:1787825952.072074:0:154237:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:1.0:1787825952.072086:0:154237:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787825952.072129:0:154237:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5000/0xbef42c0221d6e156 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154237 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825952.072137:0:154237:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5000/0xbef42c0221d6e156 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154237 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825952.072146:0:154237:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5000/0xbef42c0221d6e156 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154237 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787825952.072169:0:154237:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154237:1874663042800640:192.168.203.147@tcp:101:lfs.0 00000100:00100000:1.0:1787825952.072832:0:154237:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b593c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787825952.190674:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825952.190877:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048681088 00000100:00100000:0.0:1787825952.190889:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048681088 00000100:00100000:0.0:1787825952.190947:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9085 00000100:00100000:0.0:1787825952.190957:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:41299:x1874663048681088:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825952.190974:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481600/0xbef42c0221d6e172 lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x395:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce5c9e expref: -99 pid: 154238 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825952.191060:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825952.191067:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:41299:x1874663048681088:12345-192.168.203.147@tcp:104: Request processed in 112us (403us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825952.191075:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9085 00000100:00100000:1.0:1787825952.203882:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825952.203985:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048681344 00000100:00100000:0.0:1787825952.203995:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048681344 00000100:00100000:0.0:1787825952.204052:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9086 00000100:00100000:0.0:1787825952.204060:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b0700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663048681344:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825952.204077:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6e179 lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x395:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce5cac expref: -99 pid: 154238 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825952.204157:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825952.204164:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b0700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663048681344:12345-192.168.203.147@tcp:104: Request processed in 105us (291us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825952.204174:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9086 00000100:00100000:0.0:1787825952.354463:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787825952.355008:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048682752 00000100:00100000:1.0:1787825952.355023:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048682752 00000100:00100000:1.0:1787825952.355095:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9087 00000100:00100000:1.0:1787825952.355108:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a5180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6492:x1874663048682752:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787825952.355131:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7400/0xbef42c0221d6e19c lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x396:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce5d62 expref: -99 pid: 154240 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825952.355227:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787825952.355234:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a5180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6492:x1874663048682752:12345-192.168.203.147@tcp:104: Request processed in 129us (789us total) trans 0 rc 0/0 00000100:00100000:1.0:1787825952.355242:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9087 00000100:00100000:0.0:1787825952.372202:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787825952.372608:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048683008 00000100:00100000:1.0:1787825952.372623:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048683008 00000100:00100000:1.0:1787825952.372706:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9088 00000100:00100000:1.0:1787825952.372719:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663048683008:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787825952.372787:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f7a00/0xbef42c0221d6e1a3 lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x396:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce5d70 expref: -99 pid: 154240 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825952.372893:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787825952.373475:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663048683008:12345-192.168.203.147@tcp:104: Request processed in 184us (716us total) trans 0 rc 0/0 00000100:00100000:1.0:1787825952.373484:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9088 00000100:00100000:1.0:1787825952.405957:0:154241:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d81a99c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787825952.420562:0:154241:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636300 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154241:1874663042803712:192.168.203.147@tcp:101:lfs.0 00010000:00010000:1.0:1787825952.420593:0:154241:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7600/0xbef42c0221d6e1aa lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154241 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825952.420610:0:154241:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7600/0xbef42c0221d6e1aa lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154241 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825952.420625:0:154241:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7600/0xbef42c0221d6e1aa lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154241 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825952.420632:0:154241:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7600/0xbef42c0221d6e1aa lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154241 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825952.420643:0:154241:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7600/0xbef42c0221d6e1aa lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154241 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825952.420650:0:154241:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e06f7600 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:1.0:1787825952.420656:0:154241:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7600/0xbef42c0221d6e1aa lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154241 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825952.420664:0:154241:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7600/0xbef42c0221d6e1aa lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154241 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825952.420672:0:154241:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7600/0xbef42c0221d6e1aa lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 154241 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825952.420680:0:154241:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787825952.420689:0:154241:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7600/0xbef42c0221d6e1aa lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154241 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825952.420702:0:154241:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7600/0xbef42c0221d6e1aa lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154241 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787825952.420723:0:154241:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c5636300 x1874663042803712/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787825968 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787825952.420740:0:154241:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:1.0:1787825952.423390:0:154241:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:1.0:1787825952.423477:0:154241:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787825952.423497:0:154241:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787825952.423524:0:154241:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825952.423541:0:154241:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825952.423550:0:154241:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787825952.423557:0:154241:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787825952.423561:0:154241:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787825952.423570:0:154241:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97504a13be8 00010000:00010000:1.0:1787825952.423575:0:154241:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825952.423586:0:154241:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825952.423606:0:154241:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787825952.423617:0:154241:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=16408 00000080:00200000:1.0:1787825952.423624:0:154241:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:1.0:1787825952.423636:0:154241:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=16392 00000080:00200000:1.0:1787825952.423641:0:154241:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:1.0:1787825952.423656:0:154241:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787825952.423716:0:154241:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4e00/0xbef42c0221d6e1b1 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154241 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825952.423725:0:154241:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4e00/0xbef42c0221d6e1b1 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154241 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825952.423738:0:154241:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4e00/0xbef42c0221d6e1b1 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154241 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787825952.423789:0:154241:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154241:1874663042803840:192.168.203.147@tcp:101:lfs.0 00000100:00100000:1.0:1787825952.423901:0:154241:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b59c00 going to sleep for 11 seconds 00000100:00100000:1.0:1787825952.433421:0:154241:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154241:1874663042803840:192.168.203.147@tcp:101:lfs.0 00010000:00010000:1.0:1787825952.433452:0:154241:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4e00/0xbef42c0221d6e1b1 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154241 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825952.433470:0:154241:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4e00/0xbef42c0221d6e1b1 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154241 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825952.433479:0:154241:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4e00/0xbef42c0221d6e1b1 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154241 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825952.433486:0:154241:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4e00/0xbef42c0221d6e1b1 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154241 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825952.433497:0:154241:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4e00/0xbef42c0221d6e1b1 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154241 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825952.433504:0:154241:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4e00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:1.0:1787825952.433521:0:154241:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4e00/0xbef42c0221d6e1b1 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154241 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825952.433529:0:154241:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4e00/0xbef42c0221d6e1b1 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154241 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825952.433545:0:154241:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4e00/0xbef42c0221d6e1b1 lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 154241 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825952.433554:0:154241:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787825952.433562:0:154241:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4e00/0xbef42c0221d6e1b1 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154241 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825952.433570:0:154241:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4e00/0xbef42c0221d6e1b1 lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154241 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787825952.433591:0:154241:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c35b0380 x1874663042803840/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787825968 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787825952.433608:0:154241:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:1.0:1787825952.433643:0:154241:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:1.0:1787825952.433731:0:154241:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787825952.433775:0:154241:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787825952.433787:0:154241:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825952.433801:0:154241:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825952.433809:0:154241:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787825952.433814:0:154241:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787825952.433818:0:154241:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787825952.433824:0:154241:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97504a13b58 00010000:00010000:1.0:1787825952.433827:0:154241:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825952.433835:0:154241:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825952.433852:0:154241:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787825952.433861:0:154241:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=13 00000080:00200008:1.0:1787825952.433872:0:154241:0:(file.c:6296:ll_getattr_dentry()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), request_mask 0, flags 0, foreign 0 00000080:00200000:1.0:1787825952.433878:0:154241:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08),name=d71a.replay-single 00000002:00010000:1.0:1787825952.433892:0:154241:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x20001a9e3:0x17e0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787825952.433897:0:154241:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00010000:00010000:1.0:1787825952.433945:0:154241:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6e1b8 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 154241 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825952.433952:0:154241:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6e1b8 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 154241 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825952.433963:0:154241:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6e1b8 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 154241 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787825952.433987:0:154241:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154241:1874663042803968:192.168.203.147@tcp:101:lfs.0 00000100:00100000:1.0:1787825952.434073:0:154241:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b59c00 going to sleep for 11 seconds 00000100:00100000:1.0:1787825952.502568:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787825952.502614:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048685952 00000100:00100000:1.0:1787825952.502626:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048685952 00000100:00100000:1.0:1787825952.502682:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9089 00000100:00100000:1.0:1787825952.502693:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b1880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6490:x1874663048685952:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787825952.502715:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480000/0xbef42c0221d6e1c6 lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x397:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce5e26 expref: -99 pid: 154246 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825952.502822:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787825952.502829:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b1880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6490:x1874663048685952:12345-192.168.203.147@tcp:104: Request processed in 138us (275us total) trans 0 rc 0/0 00000100:00100000:1.0:1787825952.502837:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9089 00000100:00100000:1.0:1787825952.513377:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825952.513575:0:15151:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:0.0:1787825952.513598:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048686208 00000100:00100000:0.0:1787825952.513609:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048686208 00000100:00100000:0.0:1787825952.513675:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9090 00000100:00100000:0.0:1787825952.513688:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c440a450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663048686208:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825952.513711:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481400/0xbef42c0221d6e1cd lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x397:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce5e34 expref: -99 pid: 154246 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825952.513961:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825952.513971:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c440a450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663048686208:12345-192.168.203.147@tcp:104: Request processed in 285us (611us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825952.513980:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9090 00010000:00010000:0.0:1787825952.515356:0:33329:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787825952.515359:0:33329:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787825952.515361:0:33329:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200000:0.0:1787825952.585690:0:154259:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825952.585734:0:154259:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825952.585775:0:154259:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825952.585797:0:154259:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825952.585806:0:154259:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825952.585817:0:154259:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825952.585822:0:154259:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825952.585831:0:154259:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97504a93b58 00010000:00010000:0.0:1787825952.585836:0:154259:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825952.585845:0:154259:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825952.585864:0:154259:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825952.585873:0:154259:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=29 00000080:00200000:0.0:1787825952.585881:0:154259:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825952.585894:0:154259:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=13 00000080:00200000:0.0:1787825952.585898:0:154259:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787825952.585913:0:154259:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787825952.585974:0:154259:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6e1d4 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154259 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825952.585983:0:154259:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6e1d4 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154259 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825952.585994:0:154259:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6e1d4 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154259 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825952.586028:0:154259:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154259:1874663042805248:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787825952.586138:0:154259:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8f00 going to sleep for 11 seconds 00000100:00100000:0.0:1787825952.592066:0:154259:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154259:1874663042805248:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787825952.592102:0:154259:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6e1d4 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154259 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825952.592123:0:154259:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6e1d4 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154259 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825952.592132:0:154259:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6e1d4 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154259 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825952.592140:0:154259:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6e1d4 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154259 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825952.592149:0:154259:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6e1d4 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154259 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825952.592155:0:154259:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0513800 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787825952.592160:0:154259:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6e1d4 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154259 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825952.592168:0:154259:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6e1d4 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154259 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825952.592177:0:154259:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6e1d4 lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 154259 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825952.592186:0:154259:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787825952.592194:0:154259:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6e1d4 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154259 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825952.592203:0:154259:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6e1d4 lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154259 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825952.592222:0:154259:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c55a5180 x1874663042805248/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787825968 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787825952.592241:0:154259:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787825952.592277:0:154259:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787825952.592368:0:154259:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825952.592387:0:154259:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825952.592396:0:154259:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825952.592409:0:154259:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825952.592415:0:154259:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825952.592420:0:154259:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825952.592424:0:154259:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825952.592429:0:154259:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97504a93be8 00010000:00010000:0.0:1787825952.592433:0:154259:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825952.592441:0:154259:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825952.592460:0:154259:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825952.592468:0:154259:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=16408 00000080:00200000:0.0:1787825952.592474:0:154259:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825952.592487:0:154259:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=16392 00000080:00200000:0.0:1787825952.592491:0:154259:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787825952.592515:0:154259:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787825952.592562:0:154259:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6e1db lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154259 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825952.592570:0:154259:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6e1db lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154259 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825952.592580:0:154259:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6e1db lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154259 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825952.592603:0:154259:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154259:1874663042805376:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787825952.592699:0:154259:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8f00 going to sleep for 11 seconds 00000100:00100000:0.0:1787825952.608016:0:154259:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154259:1874663042805376:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787825952.608051:0:154259:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6e1db lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154259 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825952.608070:0:154259:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6e1db lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154259 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825952.608079:0:154259:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6e1db lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154259 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825952.608086:0:154259:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6e1db lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154259 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825952.608096:0:154259:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6e1db lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154259 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825952.608103:0:154259:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0512e00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787825952.608108:0:154259:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6e1db lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154259 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825952.608116:0:154259:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6e1db lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154259 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825952.608124:0:154259:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6e1db lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 154259 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825952.608132:0:154259:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787825952.608139:0:154259:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6e1db lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154259 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825952.608145:0:154259:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6e1db lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154259 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825952.608167:0:154259:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c55a5180 x1874663042805376/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787825968 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787825952.608184:0:154259:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787825952.608219:0:154259:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787825952.608320:0:154259:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825952.608340:0:154259:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825952.608351:0:154259:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825952.608365:0:154259:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825952.608372:0:154259:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825952.608378:0:154259:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825952.608382:0:154259:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825952.608388:0:154259:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97504a93b58 00010000:00010000:0.0:1787825952.608392:0:154259:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825952.608402:0:154259:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825952.608420:0:154259:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825952.608431:0:154259:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=13 00000080:00200008:0.0:1787825952.608443:0:154259:0:(file.c:6296:ll_getattr_dentry()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1787825952.608448:0:154259:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08),name=d71a.replay-single 00000002:00010000:0.0:1787825952.608463:0:154259:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x20001a9e3:0x17e0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825952.608469:0:154259:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00010000:00010000:0.0:1787825952.608530:0:154259:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e1e2 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 154259 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825952.608538:0:154259:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e1e2 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 154259 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825952.608550:0:154259:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e1e2 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 154259 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825952.608577:0:154259:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154259:1874663042805504:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787825952.608675:0:154259:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8f00 going to sleep for 11 seconds 00010000:00010000:1.0:1787825952.610054:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787825952.619972:0:154259:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154259:1874663042805504:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787825952.620006:0:154259:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e1e2 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5e96 expref: -99 pid: 154259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825952.620025:0:154259:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e1e2 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5e96 expref: -99 pid: 154259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825952.620039:0:154259:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e1e2 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5e96 expref: -99 pid: 154259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825952.620051:0:154259:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e1e2 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5e96 expref: -99 pid: 154259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825952.620063:0:154259:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e1e2 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5e96 expref: -99 pid: 154259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825952.620077:0:154259:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e1e2 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5e96 expref: -99 pid: 154259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825952.620085:0:154259:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e1e2 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5e96 expref: -99 pid: 154259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825952.620092:0:154259:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e1e2 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5e96 expref: -99 pid: 154259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825952.620105:0:154259:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c55a5180 x1874663042805504/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787825968 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787825952.620121:0:154259:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e1e2 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5e96 expref: -99 pid: 154259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825952.620142:0:154259:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787825952.620150:0:154259:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1787825952.620186:0:154259:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x17e0:0x0], updating i_size 4096 00000080:00010000:0.0:1787825952.620195:0:154259:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) for lock 0xbef42c0221d6e1e2 00000080:00010000:0.0:1787825952.620204:0:154259:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00000080:00010000:0.0:1787825952.620208:0:154259:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6e1e2 from it ffffb97504a93cb8 00010000:00010000:0.0:1787825952.620211:0:154259:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e1e2 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5e96 expref: -99 pid: 154259 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825952.620219:0:154259:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e1e2 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5e96 expref: -99 pid: 154259 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:0.0:1787825952.622204:0:154259:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1787825952.622218:0:154259:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825952.622248:0:154259:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825952.622270:0:154259:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825952.622290:0:154259:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825952.622298:0:154259:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825952.622307:0:154259:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825952.622311:0:154259:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825952.622319:0:154259:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97504a93cb8 00010000:00010000:0.0:1787825952.622323:0:154259:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825952.622332:0:154259:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825952.622499:0:154259:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825952.622519:0:154259:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825952.622528:0:154259:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825952.622538:0:154259:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825952.622545:0:154259:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825952.622549:0:154259:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825952.622552:0:154259:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825952.623939:0:154259:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97504a93bf8 00010000:00010000:0.0:1787825952.623953:0:154259:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825952.623964:0:154259:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825952.623981:0:154259:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825952.623993:0:154259:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=267 00000080:00200000:0.0:1787825952.624004:0:154259:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 44 00000080:00200008:0.0:1787825952.624012:0:154259:0:(file.c:1090:ll_file_open()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00000080:00200080:0.0:1787825952.624049:0:154259:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) cmd=c03066f0 arg=7ffd31298ba0 00000080:00200000:0.0:1787825952.624066:0:154259:0:(dir.c:451:ll_dir_setdirstripe()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) name=test stripe_offset=0 stripe_count=2, hash_type=0 00010000:00010000:0.0:1787825952.624103:0:154259:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e1e2 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ce5e96 expref: -99 pid: 154259 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825952.624200:0:154259:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6800/0xbef42c0221d64b3f lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cb9ffd expref: -99 pid: 152202 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825952.624265:0:154259:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e1e2 lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce5e96 expref: -99 pid: 154259 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825952.624273:0:154259:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787825952.624276:0:154259:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e1e2 lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce5e96 expref: -99 pid: 154259 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:0.0:1787825952.624298:0:154259:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787825952.624313:0:154259:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c55a5180 x1874663042805760/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 552/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787825952.624342:0:154259:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154259:1874663042805760:192.168.203.147@tcp:36:lfs.0 00000100:00100000:0.0:1787825952.624493:0:154259:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8f00 going to sleep for 11 seconds 00000100:00100000:1.0:1787825952.741103:0:154270:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d805d880 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154270:1874663042806144:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:1.0:1787825952.741138:0:154270:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481200/0xbef42c0221d6e1f7 lrc: 4/1,0 mode: --/CR res: [0x24000178a:0x398:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5ef8 expref: -99 pid: 154270 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825952.741153:0:154270:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481200/0xbef42c0221d6e1f7 lrc: 4/1,0 mode: --/PR res: [0x24000178a:0x398:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5ef8 expref: -99 pid: 154270 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825952.741166:0:154270:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481200/0xbef42c0221d6e1f7 lrc: 4/1,0 mode: PR/PR res: [0x24000178a:0x398:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5ef8 expref: -99 pid: 154270 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825952.741178:0:154270:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481200/0xbef42c0221d6e1f7 lrc: 4/1,0 mode: PR/PR res: [0x24000178a:0x398:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5ef8 expref: -99 pid: 154270 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825952.741194:0:154270:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481200/0xbef42c0221d6e1f7 lrc: 5/2,0 mode: PR/PR res: [0x24000178a:0x398:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5ef8 expref: -99 pid: 154270 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825952.741203:0:154270:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481200/0xbef42c0221d6e1f7 lrc: 5/2,0 mode: PR/PR res: [0x24000178a:0x398:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5ef8 expref: -99 pid: 154270 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825952.741211:0:154270:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481200/0xbef42c0221d6e1f7 lrc: 4/1,0 mode: PR/PR res: [0x24000178a:0x398:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5ef8 expref: -99 pid: 154270 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787825952.741226:0:154270:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181d805d880 x1874663042806144/t0(0) o101->lustre-MDT0001-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787825968 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787825952.741244:0:154270:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481200/0xbef42c0221d6e1f7 lrc: 3/1,0 mode: PR/PR res: [0x24000178a:0x398:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5ef8 expref: -99 pid: 154270 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825952.741260:0:154270:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d0f54800 type 13 mode 4 res 9663682442/920 (0 0) 00010000:00010000:1.0:1787825952.741270:0:154270:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481200/0xbef42c0221d6e1f7 lrc: 3/1,0 mode: PR/PR res: [0x24000178a:0x398:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5ef8 expref: -99 pid: 154270 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825952.741280:0:154270:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481200/0xbef42c0221d6e1f7 lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x398:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5ef8 expref: -99 pid: 154270 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787825952.741325:0:154270:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1b77:0x0], updating i_size 8192 00000080:00200000:1.0:1787825952.741330:0:154270:0:(namei.c:147:ll_iget()) got inode: [0x20001a9e3:0x1b77:0x0](ffff9181c83aec08): rc = 0 00000080:00010000:1.0:1787825952.741342:0:154270:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x1b77:0x0](ffff9181c83aec08) for lock 0xbef42c0221d6e1e9 00000080:00010000:1.0:1787825952.741363:0:154270:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x1b77:0x0](ffff9181c83aec08) 00000080:00010000:1.0:1787825952.741368:0:154270:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6e1e9 from it ffffb97504acbe08 00010000:00010000:1.0:1787825952.741372:0:154270:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481400/0xbef42c0221d6e1e9 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1b77:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5edc expref: -99 pid: 154270 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825952.741382:0:154270:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481400/0xbef42c0221d6e1e9 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1b77:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5edc expref: -99 pid: 154270 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787825952.741407:0:154270:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 3 00000080:00200000:1.0:1787825952.741416:0:154270:0:(namei.c:2328:ll_rmdir()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00010000:00010000:1.0:1787825952.741455:0:154270:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481400/0xbef42c0221d6e1e9 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1b77:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ce5edc expref: -99 pid: 154270 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825952.741529:0:154270:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481400/0xbef42c0221d6e1e9 lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x1b77:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce5edc expref: -99 pid: 154270 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825952.741537:0:154270:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787825952.741540:0:154270:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481400/0xbef42c0221d6e1e9 lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x1b77:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce5edc expref: -99 pid: 154270 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:1.0:1787825952.741578:0:154270:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787825952.741585:0:154270:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c35b0000 x1874663042806400/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787825952.741612:0:154270:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154270:1874663042806400:192.168.203.147@tcp:36:rmdir.0 00000100:00100000:1.0:1787825952.744887:0:154270:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b596c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787825952.755496:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787825952.755633:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048687616 00000100:00100000:1.0:1787825952.755646:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048687616 00000100:00100000:1.0:1787825952.755720:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9091 00000100:00100000:1.0:1787825952.755733:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6491:x1874663048687616:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787825952.755780:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480000/0xbef42c0221d6e1f0 lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x398:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce5eea expref: -99 pid: 154270 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825952.755875:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787825952.755882:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6491:x1874663048687616:12345-192.168.203.147@tcp:104: Request processed in 152us (400us total) trans 0 rc 0/0 00000100:00100000:1.0:1787825952.755891:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9091 00000100:00100000:0.0:1787825952.771013:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825952.771053:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048687872 00000100:00100000:0.0:1787825952.771065:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048687872 00000100:00100000:0.0:1787825952.771130:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9092 00000100:00100000:0.0:1787825952.771141:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8414:x1874663048687872:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825952.771164:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481200/0xbef42c0221d6e1f7 lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x398:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce5ef8 expref: -99 pid: 154270 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825952.771251:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825952.771259:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8414:x1874663048687872:12345-192.168.203.147@tcp:104: Request processed in 119us (262us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825952.771269:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9092 00010000:00010000:0.0:1787825952.771297:0:33350:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787825952.771303:0:33350:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481200/0xbef42c0221d6e1f7 lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x398:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce5ef8 expref: -99 pid: 154270 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825952.771313:0:33350:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3481200 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787825952.771319:0:33350:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481200/0xbef42c0221d6e1f7 lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x398:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00ce5ef8 expref: -99 pid: 154270 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825952.771328:0:33350:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481200/0xbef42c0221d6e1f7 lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x398:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00ce5ef8 expref: -99 pid: 154270 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825952.771338:0:33350:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481200/0xbef42c0221d6e1f7 lrc: 3/0,0 mode: PR/PR res: [0x24000178a:0x398:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x408400000000 nid: local remote: 0x878a5eea00ce5ef8 expref: -99 pid: 154270 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825952.771410:0:33350:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481200/0xbef42c0221d6e1f7 lrc: 2/0,0 mode: --/PR res: [0x24000178a:0x398:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ce5ef8 expref: -99 pid: 154270 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825952.771417:0:33350:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787825952.771442:0:33350:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481200/0xbef42c0221d6e1f7 lrc: 1/0,0 mode: --/PR res: [0x24000178a:0x398:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ce5ef8 expref: -99 pid: 154270 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825952.771449:0:33350:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481200/0xbef42c0221d6e1f7 lrc: 0/0,0 mode: --/PR res: [0x24000178a:0x398:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ce5ef8 expref: -99 pid: 154270 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825952.771471:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787825952.771473:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787825952.771474:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787825952.771492:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042806656:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1787825952.782346:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042806656:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1787825952.897769:0:154288:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c4687b80 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154288:1874663042807552:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:0.0:1787825952.897807:0:154288:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6e21a lrc: 4/1,0 mode: --/CR res: [0x200025d90:0x399:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5fae expref: -99 pid: 154288 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825952.897823:0:154288:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6e21a lrc: 4/1,0 mode: --/PR res: [0x200025d90:0x399:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5fae expref: -99 pid: 154288 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825952.897839:0:154288:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6e21a lrc: 4/1,0 mode: PR/PR res: [0x200025d90:0x399:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5fae expref: -99 pid: 154288 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825952.897852:0:154288:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6e21a lrc: 4/1,0 mode: PR/PR res: [0x200025d90:0x399:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5fae expref: -99 pid: 154288 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825952.897866:0:154288:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6e21a lrc: 5/2,0 mode: PR/PR res: [0x200025d90:0x399:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5fae expref: -99 pid: 154288 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825952.897874:0:154288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6e21a lrc: 5/2,0 mode: PR/PR res: [0x200025d90:0x399:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5fae expref: -99 pid: 154288 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825952.897883:0:154288:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6e21a lrc: 4/1,0 mode: PR/PR res: [0x200025d90:0x399:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5fae expref: -99 pid: 154288 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825952.897898:0:154288:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c4687b80 x1874663042807552/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787825968 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787825952.897921:0:154288:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6e21a lrc: 3/1,0 mode: PR/PR res: [0x200025d90:0x399:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5fae expref: -99 pid: 154288 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825952.897937:0:154288:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590089616/921 (0 0) 00010000:00010000:0.0:1787825952.897945:0:154288:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6e21a lrc: 3/1,0 mode: PR/PR res: [0x200025d90:0x399:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce5fae expref: -99 pid: 154288 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825952.897955:0:154288:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6e21a lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x399:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce5fae expref: -99 pid: 154288 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000002:00010000:0.0:1787825952.897995:0:154288:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x24000178a:0x399:0x0]) in obj [0x24000178a:0x399:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825952.898058:0:154288:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e221 lrc: 3/1,0 mode: --/CR res: [0x24000178a:0x399:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 154288 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825952.898067:0:154288:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e221 lrc: 3/1,0 mode: --/CR res: [0x24000178a:0x399:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154288 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825952.898076:0:154288:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e221 lrc: 3/1,0 mode: --/CR res: [0x24000178a:0x399:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154288 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825952.898102:0:154288:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154288:1874663042807680:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787825952.898218:0:154288:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8780 going to sleep for 11 seconds 00000100:00100000:1.0:1787825952.929598:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787825952.929650:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048689664 00000100:00100000:1.0:1787825952.929664:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048689664 00000100:00100000:1.0:1787825952.929774:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9093 00000100:00100000:1.0:1787825952.929787:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b1500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6492:x1874663048689664:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787825952.929813:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6e21a lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x399:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce5fae expref: -99 pid: 154288 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825952.929905:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787825952.929912:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b1500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6492:x1874663048689664:12345-192.168.203.147@tcp:104: Request processed in 128us (341us total) trans 0 rc 0/0 00000100:00100000:1.0:1787825952.929920:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9093 00000100:00100000:1.0:1787825952.935347:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787825952.935399:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048689920 00000100:00100000:1.0:1787825952.935413:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048689920 00000100:00100000:1.0:1787825952.935471:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9094 00000100:00100000:1.0:1787825952.935482:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b3100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663048689920:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787825952.935518:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e221 lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x399:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce5fbc expref: -99 pid: 154288 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825952.935609:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787825952.935616:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b3100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663048689920:12345-192.168.203.147@tcp:104: Request processed in 135us (285us total) trans 0 rc 0/0 00000100:00100000:1.0:1787825952.935624:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9094 00010000:00010000:1.0:1787825952.935649:0:15161:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787825952.935656:0:15161:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e221 lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x399:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce5fbc expref: -99 pid: 154288 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825952.935665:0:15161:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0512400 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:1.0:1787825952.935670:0:15161:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e221 lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x399:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00ce5fbc expref: -99 pid: 154288 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825952.935686:0:15161:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e221 lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x399:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00ce5fbc expref: -99 pid: 154288 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825952.935706:0:15161:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e221 lrc: 3/0,0 mode: PR/PR res: [0x24000178a:0x399:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x408400000000 nid: local remote: 0x878a5eea00ce5fbc expref: -99 pid: 154288 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825952.935800:0:15161:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e221 lrc: 2/0,0 mode: --/PR res: [0x24000178a:0x399:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ce5fbc expref: -99 pid: 154288 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825952.935808:0:15161:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787825952.935834:0:15161:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e221 lrc: 1/0,0 mode: --/PR res: [0x24000178a:0x399:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ce5fbc expref: -99 pid: 154288 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825952.935841:0:15161:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e221 lrc: 0/0,0 mode: --/PR res: [0x24000178a:0x399:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ce5fbc expref: -99 pid: 154288 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825952.935860:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787825952.935861:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787825952.935864:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787825952.935879:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042808192:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787825952.941516:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042808192:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1787825953.082635:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825953.082682:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048691328 00000100:00100000:0.0:1787825953.082694:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048691328 00000100:00100000:0.0:1787825953.082807:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9095 00000100:00100000:0.0:1787825953.082820:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:7728:x1874663048691328:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825953.082847:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6200/0xbef42c0221d6e244 lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x39a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce6072 expref: -99 pid: 154304 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825953.082946:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825953.082954:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:7728:x1874663048691328:12345-192.168.203.147@tcp:104: Request processed in 137us (335us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825953.082962:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9095 00000100:00100000:0.0:1787825953.096624:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787825953.097099:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048691584 00000100:00100000:1.0:1787825953.097114:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048691584 00000100:00100000:1.0:1787825953.097280:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9096 00000100:00100000:1.0:1787825953.097293:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a6680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663048691584:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787825953.097322:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481800/0xbef42c0221d6e24b lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x39a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce6080 expref: -99 pid: 154304 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825953.097476:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787825953.097486:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a6680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663048691584:12345-192.168.203.147@tcp:104: Request processed in 194us (878us total) trans 0 rc 0/0 00000100:00100000:1.0:1787825953.097494:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9096 00000080:00200000:0.0:1787825953.169498:0:154315:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825953.169690:0:154315:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825953.169717:0:154315:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825953.169738:0:154315:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825953.169794:0:154315:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825953.169804:0:154315:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825953.169808:0:154315:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825953.169818:0:154315:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97504b33b58 00010000:00010000:0.0:1787825953.169823:0:154315:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825953.169833:0:154315:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825953.169852:0:154315:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825953.169863:0:154315:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=29 00000080:00200000:0.0:1787825953.169871:0:154315:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825953.169883:0:154315:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=13 00000080:00200000:0.0:1787825953.169889:0:154315:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787825953.169903:0:154315:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787825953.169963:0:154315:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6e252 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.169971:0:154315:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6e252 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.169982:0:154315:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6e252 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154315 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825953.170016:0:154315:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6300 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154315:1874663042809856:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787825953.172860:0:154315:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8d80 going to sleep for 11 seconds 00000100:00100000:0.0:1787825953.173419:0:154315:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6300 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154315:1874663042809856:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787825953.173442:0:154315:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6e252 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.173457:0:154315:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6e252 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.173464:0:154315:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6e252 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.173471:0:154315:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6e252 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.173480:0:154315:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6e252 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.173486:0:154315:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0513c00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787825953.173491:0:154315:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6e252 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.173509:0:154315:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6e252 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.173517:0:154315:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6e252 lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 154315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.173526:0:154315:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787825953.173533:0:154315:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6e252 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.173541:0:154315:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6e252 lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154315 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825953.174372:0:154315:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c55a6300 x1874663042809856/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787825969 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787825953.174397:0:154315:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787825953.174433:0:154315:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787825953.174533:0:154315:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825953.174563:0:154315:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825953.174576:0:154315:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825953.174590:0:154315:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825953.174604:0:154315:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825953.174610:0:154315:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825953.174614:0:154315:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825953.174620:0:154315:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97504b33be8 00010000:00010000:0.0:1787825953.174624:0:154315:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825953.174633:0:154315:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825953.174651:0:154315:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825953.174660:0:154315:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=16408 00000080:00200000:0.0:1787825953.174667:0:154315:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825953.174682:0:154315:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=16392 00000080:00200000:0.0:1787825953.174687:0:154315:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787825953.178688:0:154315:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787825953.178799:0:154315:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e259 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.178811:0:154315:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e259 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.178821:0:154315:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e259 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154315 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825953.178852:0:154315:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6300 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154315:1874663042809984:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787825953.179117:0:154315:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8d80 going to sleep for 11 seconds 00000100:00100000:0.0:1787825953.193134:0:154315:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6300 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154315:1874663042809984:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787825953.193166:0:154315:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e259 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.193183:0:154315:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e259 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.193192:0:154315:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e259 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.193199:0:154315:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e259 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.193208:0:154315:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e259 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.193214:0:154315:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0512400 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787825953.193219:0:154315:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e259 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.193226:0:154315:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e259 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.193234:0:154315:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e259 lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 154315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.193241:0:154315:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787825953.193248:0:154315:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e259 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.193255:0:154315:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e259 lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154315 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825953.193275:0:154315:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c55a6300 x1874663042809984/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787825969 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787825953.193293:0:154315:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787825953.193328:0:154315:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787825953.193412:0:154315:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825953.193427:0:154315:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825953.193437:0:154315:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825953.193450:0:154315:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825953.193457:0:154315:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825953.193462:0:154315:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825953.193465:0:154315:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825953.193471:0:154315:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97504b33b58 00010000:00010000:0.0:1787825953.193475:0:154315:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825953.193484:0:154315:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825953.193511:0:154315:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825953.193520:0:154315:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=13 00000080:00200008:0.0:1787825953.193532:0:154315:0:(file.c:6296:ll_getattr_dentry()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1787825953.193537:0:154315:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08),name=d71a.replay-single 00000002:00010000:0.0:1787825953.193551:0:154315:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x20001a9e3:0x17e0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825953.193556:0:154315:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00010000:00010000:0.0:1787825953.193607:0:154315:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6e260 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 154315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.193615:0:154315:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6e260 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 154315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.193625:0:154315:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6e260 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 154315 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825953.193647:0:154315:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6300 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154315:1874663042810112:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787825953.276868:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825953.276946:0:15150:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:0.0:1787825953.276970:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048692992 00000100:00100000:0.0:1787825953.276982:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048692992 00000100:00100000:0.0:1787825953.277049:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9097 00000100:00100000:0.0:1787825953.277062:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c5cfec50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6491:x1874663048692992:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825953.277085:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481800/0xbef42c0221d6e26e lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x39b:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce6136 expref: -99 pid: 154324 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825953.277176:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825953.277183:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c5cfec50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6491:x1874663048692992:12345-192.168.203.147@tcp:104: Request processed in 124us (331us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825953.277191:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9097 00000100:00100000:0.0:1787825953.294241:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825953.294281:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048693248 00000100:00100000:0.0:1787825953.294291:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048693248 00000100:00100000:0.0:1787825953.294350:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9098 00000100:00100000:0.0:1787825953.294361:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663048693248:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825953.294385:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481200/0xbef42c0221d6e275 lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x39b:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce6144 expref: -99 pid: 154324 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825953.294482:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825953.294489:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663048693248:12345-192.168.203.147@tcp:104: Request processed in 130us (627us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825953.294497:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9098 00000080:00200000:1.0:1787825953.350909:0:154331:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787825953.350953:0:154331:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787825953.350979:0:154331:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825953.350998:0:154331:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825953.351007:0:154331:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787825953.351017:0:154331:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787825953.351021:0:154331:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787825953.351030:0:154331:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97504b33b58 00010000:00010000:1.0:1787825953.351035:0:154331:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825953.351045:0:154331:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825953.351064:0:154331:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787825953.351072:0:154331:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=29 00000080:00200000:1.0:1787825953.351081:0:154331:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:1.0:1787825953.351093:0:154331:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=13 00000080:00200000:1.0:1787825953.351097:0:154331:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:1.0:1787825953.351110:0:154331:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787825953.351168:0:154331:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5600/0xbef42c0221d6e27c lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154331 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825953.351177:0:154331:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5600/0xbef42c0221d6e27c lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154331 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825953.351188:0:154331:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5600/0xbef42c0221d6e27c lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154331 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787825953.351221:0:154331:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154331:1874663042811776:192.168.203.147@tcp:101:lfs.0 00000100:00100000:1.0:1787825953.351493:0:154331:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b59b40 going to sleep for 11 seconds 00000080:00200000:0.0:1787825953.395813:0:154336:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825953.395855:0:154336:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825953.395881:0:154336:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825953.396042:0:154336:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825953.396054:0:154336:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825953.396063:0:154336:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825953.396067:0:154336:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825953.396078:0:154336:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97504b33c20 00010000:00010000:0.0:1787825953.396083:0:154336:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825953.396094:0:154336:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825953.396115:0:154336:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825953.396198:0:154336:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=24 00000080:00200000:0.0:1787825953.396206:0:154336:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825953.396219:0:154336:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=0 00000080:00200000:0.0:1787825953.396224:0:154336:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787825953.396240:0:154336:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787825953.396305:0:154336:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e291 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154336 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.396314:0:154336:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e291 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154336 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.396325:0:154336:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e291 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154336 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825953.396355:0:154336:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4a80 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154336:1874663042812416:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787825953.396472:0:154336:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c83c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787825953.400267:0:154336:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4a80 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154336:1874663042812416:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:0.0:1787825953.400304:0:154336:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e291 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce61ec expref: -99 pid: 154336 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.400329:0:154336:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e291 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1b7b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce61ec expref: -99 pid: 154336 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.400338:0:154336:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e291 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1b7b:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce61ec expref: -99 pid: 154336 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.400351:0:154336:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e291 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1b7b:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce61ec expref: -99 pid: 154336 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.400369:0:154336:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e291 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1b7b:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce61ec expref: -99 pid: 154336 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.400385:0:154336:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e291 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1b7b:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce61ec expref: -99 pid: 154336 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.400393:0:154336:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e291 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1b7b:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce61ec expref: -99 pid: 154336 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.400402:0:154336:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e291 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1b7b:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce61ec expref: -99 pid: 154336 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825953.400415:0:154336:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c55a4a80 x1874663042812416/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/672 e 0 to 0 dl 1787825969 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787825953.400432:0:154336:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e291 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1b7b:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce61ec expref: -99 pid: 154336 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.400444:0:154336:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/7035 (0 0) 00000080:00200000:0.0:1787825953.400452:0:154336:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00000002:00010000:0.0:1787825953.400532:0:154336:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200025d90:0x39c:0x0]) in obj [0x200025d90:0x39c:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825953.400658:0:154336:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6e298 lrc: 3/1,0 mode: --/CR res: [0x200025d90:0x39c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 154336 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.400668:0:154336:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6e298 lrc: 3/1,0 mode: --/CR res: [0x200025d90:0x39c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154336 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.400678:0:154336:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6e298 lrc: 3/1,0 mode: --/CR res: [0x200025d90:0x39c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154336 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825953.400712:0:154336:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154336:1874663042812544:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787825953.401376:0:154336:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c83c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787825953.412042:0:154336:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154336:1874663042812544:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:0.0:1787825953.412078:0:154336:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6e298 lrc: 4/1,0 mode: --/CR res: [0x200025d90:0x39c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce61fa expref: -99 pid: 154336 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.412094:0:154336:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6e298 lrc: 4/1,0 mode: --/PR res: [0x200025d90:0x39c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce61fa expref: -99 pid: 154336 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.412109:0:154336:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6e298 lrc: 4/1,0 mode: PR/PR res: [0x200025d90:0x39c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce61fa expref: -99 pid: 154336 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.412243:0:154336:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6e298 lrc: 4/1,0 mode: PR/PR res: [0x200025d90:0x39c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce61fa expref: -99 pid: 154336 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.412261:0:154336:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6e298 lrc: 5/2,0 mode: PR/PR res: [0x200025d90:0x39c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce61fa expref: -99 pid: 154336 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.412270:0:154336:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6e298 lrc: 5/2,0 mode: PR/PR res: [0x200025d90:0x39c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce61fa expref: -99 pid: 154336 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.412278:0:154336:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6e298 lrc: 4/1,0 mode: PR/PR res: [0x200025d90:0x39c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce61fa expref: -99 pid: 154336 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825953.412292:0:154336:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c55a5c00 x1874663042812544/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787825969 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787825953.412314:0:154336:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6e298 lrc: 3/1,0 mode: PR/PR res: [0x200025d90:0x39c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce61fa expref: -99 pid: 154336 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.412329:0:154336:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590089616/924 (0 0) 00010000:00010000:0.0:1787825953.412337:0:154336:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6e298 lrc: 3/1,0 mode: PR/PR res: [0x200025d90:0x39c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce61fa expref: -99 pid: 154336 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.412347:0:154336:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6e298 lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x39c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce61fa expref: -99 pid: 154336 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000002:00010000:0.0:1787825953.412385:0:154336:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x24000178a:0x39c:0x0]) in obj [0x24000178a:0x39c:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825953.412443:0:154336:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6e29f lrc: 3/1,0 mode: --/CR res: [0x24000178a:0x39c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 154336 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.412450:0:154336:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6e29f lrc: 3/1,0 mode: --/CR res: [0x24000178a:0x39c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154336 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.412459:0:154336:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6e29f lrc: 3/1,0 mode: --/CR res: [0x24000178a:0x39c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154336 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825953.412483:0:154336:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154336:1874663042812672:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787825953.412589:0:154336:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c83c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787825953.453402:0:154336:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154336:1874663042812672:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:0.0:1787825953.453439:0:154336:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6e29f lrc: 4/1,0 mode: --/CR res: [0x24000178a:0x39c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6208 expref: -99 pid: 154336 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.453454:0:154336:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6e29f lrc: 4/1,0 mode: --/PR res: [0x24000178a:0x39c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6208 expref: -99 pid: 154336 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.453469:0:154336:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6e29f lrc: 4/1,0 mode: PR/PR res: [0x24000178a:0x39c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6208 expref: -99 pid: 154336 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.453483:0:154336:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6e29f lrc: 4/1,0 mode: PR/PR res: [0x24000178a:0x39c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6208 expref: -99 pid: 154336 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.453509:0:154336:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6e29f lrc: 5/2,0 mode: PR/PR res: [0x24000178a:0x39c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6208 expref: -99 pid: 154336 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.453518:0:154336:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6e29f lrc: 5/2,0 mode: PR/PR res: [0x24000178a:0x39c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6208 expref: -99 pid: 154336 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.453526:0:154336:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6e29f lrc: 4/1,0 mode: PR/PR res: [0x24000178a:0x39c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6208 expref: -99 pid: 154336 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825953.453539:0:154336:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c55a5c00 x1874663042812672/t0(0) o101->lustre-MDT0001-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787825969 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787825953.453630:0:154336:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6e29f lrc: 3/1,0 mode: PR/PR res: [0x24000178a:0x39c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6208 expref: -99 pid: 154336 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.453648:0:154336:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d0f54800 type 13 mode 4 res 9663682442/924 (0 0) 00010000:00010000:0.0:1787825953.453659:0:154336:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6e29f lrc: 3/1,0 mode: PR/PR res: [0x24000178a:0x39c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6208 expref: -99 pid: 154336 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.453670:0:154336:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6e29f lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x39c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6208 expref: -99 pid: 154336 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787825953.453722:0:154336:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1b7b:0x0], updating i_size 8192 00000080:00200000:0.0:1787825953.453728:0:154336:0:(namei.c:147:ll_iget()) got inode: [0x20001a9e3:0x1b7b:0x0](ffff9181c83b2a08): rc = 0 00000080:00010000:0.0:1787825953.453739:0:154336:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x1b7b:0x0](ffff9181c83b2a08) for lock 0xbef42c0221d6e291 00000080:00010000:0.0:1787825953.453847:0:154336:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x1b7b:0x0](ffff9181c83b2a08) 00000080:00010000:0.0:1787825953.453852:0:154336:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6e291 from it ffffb97504b33e08 00010000:00010000:0.0:1787825953.453856:0:154336:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e291 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1b7b:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce61ec expref: -99 pid: 154336 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.453865:0:154336:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e291 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1b7b:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce61ec expref: -99 pid: 154336 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787825953.453889:0:154336:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 3 00000080:00200000:0.0:1787825953.453983:0:154336:0:(namei.c:2328:ll_rmdir()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00010000:00010000:0.0:1787825953.454025:0:154336:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e291 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1b7b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ce61ec expref: -99 pid: 154336 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825953.454087:0:154336:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e291 lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x1b7b:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce61ec expref: -99 pid: 154336 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825953.454309:0:154336:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787825953.454312:0:154336:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e291 lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x1b7b:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce61ec expref: -99 pid: 154336 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:0.0:1787825953.454342:0:154336:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787825953.454351:0:154336:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c55a4a80 x1874663042812928/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787825953.454378:0:154336:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4a80 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154336:1874663042812928:192.168.203.147@tcp:36:rmdir.0 00000100:00100000:0.0:1787825953.454542:0:154336:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c83c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787825953.459289:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825953.459383:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048694912 00000100:00100000:0.0:1787825953.459397:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048694912 00000100:00100000:0.0:1787825953.459480:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9099 00000100:00100000:0.0:1787825953.459493:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:41299:x1874663048694912:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825953.459530:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6e298 lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x39c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce61fa expref: -99 pid: 154336 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825953.459631:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825953.459639:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:41299:x1874663048694912:12345-192.168.203.147@tcp:104: Request processed in 149us (366us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825953.459647:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9099 00000100:00100000:1.0:1787825953.482682:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825953.483168:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048695168 00000100:00100000:0.0:1787825953.483182:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048695168 00000100:00100000:0.0:1787825953.483247:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9100 00000100:00100000:0.0:1787825953.483259:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663048695168:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825953.483286:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6e29f lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x39c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce6208 expref: -99 pid: 154336 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825953.483379:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825953.483387:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663048695168:12345-192.168.203.147@tcp:104: Request processed in 130us (724us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825953.483395:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9100 00000080:00200000:0.0:1787825953.535180:0:154344:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825953.535222:0:154344:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825953.535248:0:154344:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825953.535269:0:154344:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825953.535278:0:154344:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825953.535289:0:154344:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825953.535293:0:154344:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825953.535302:0:154344:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97504b73b58 00010000:00010000:0.0:1787825953.535306:0:154344:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825953.535315:0:154344:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825953.535333:0:154344:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825953.535343:0:154344:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=29 00000080:00200000:0.0:1787825953.535351:0:154344:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825953.535365:0:154344:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=13 00000080:00200000:0.0:1787825953.535370:0:154344:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787825953.535386:0:154344:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787825953.535450:0:154344:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e2a6 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154344 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.535459:0:154344:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e2a6 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154344 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.535472:0:154344:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e2a6 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154344 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825953.535525:0:154344:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154344:1874663042813312:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787825953.535644:0:154344:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8a80 going to sleep for 11 seconds 00000100:00100000:0.0:1787825953.545043:0:154344:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154344:1874663042813312:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787825953.545077:0:154344:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e2a6 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154344 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.545095:0:154344:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e2a6 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154344 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.545104:0:154344:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e2a6 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154344 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.545112:0:154344:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e2a6 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154344 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.545122:0:154344:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e2a6 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154344 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.545129:0:154344:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0512600 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787825953.545134:0:154344:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e2a6 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154344 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.545141:0:154344:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e2a6 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154344 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.545149:0:154344:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e2a6 lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 154344 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.545157:0:154344:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787825953.545166:0:154344:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e2a6 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154344 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.545174:0:154344:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e2a6 lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154344 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825953.545196:0:154344:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c55a5c00 x1874663042813312/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787825969 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787825953.545214:0:154344:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787825953.545251:0:154344:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787825953.545336:0:154344:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825953.545356:0:154344:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825953.547040:0:154344:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825953.547061:0:154344:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825953.547071:0:154344:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825953.547082:0:154344:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825953.547086:0:154344:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825953.547096:0:154344:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97504b73be8 00010000:00010000:0.0:1787825953.547101:0:154344:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825953.547111:0:154344:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825953.547132:0:154344:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825953.547145:0:154344:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=16408 00000080:00200000:0.0:1787825953.547155:0:154344:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825953.547172:0:154344:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=16392 00000080:00200000:0.0:1787825953.547177:0:154344:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787825953.547198:0:154344:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787825953.547251:0:154344:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6e2ad lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154344 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.547259:0:154344:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6e2ad lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154344 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.547269:0:154344:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6e2ad lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154344 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825953.547299:0:154344:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154344:1874663042813440:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787825953.548530:0:154344:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8a80 going to sleep for 11 seconds 00000100:00100000:0.0:1787825953.558049:0:154344:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154344:1874663042813440:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787825953.558078:0:154344:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6e2ad lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154344 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.558094:0:154344:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6e2ad lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154344 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.558101:0:154344:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6e2ad lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154344 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.558108:0:154344:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6e2ad lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154344 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.558116:0:154344:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6e2ad lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154344 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.558125:0:154344:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0513e00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787825953.558130:0:154344:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6e2ad lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154344 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.558137:0:154344:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6e2ad lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154344 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.558145:0:154344:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6e2ad lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 154344 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.558151:0:154344:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787825953.558158:0:154344:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6e2ad lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154344 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.558165:0:154344:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6e2ad lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154344 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825953.558182:0:154344:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c55a5c00 x1874663042813440/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787825969 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787825953.558197:0:154344:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787825953.558229:0:154344:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787825953.558322:0:154344:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825953.558340:0:154344:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825953.558349:0:154344:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825953.558362:0:154344:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825953.558376:0:154344:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825953.558381:0:154344:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825953.558385:0:154344:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825953.558390:0:154344:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97504b73b58 00010000:00010000:0.0:1787825953.558394:0:154344:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825953.558404:0:154344:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825953.558435:0:154344:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825953.558446:0:154344:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=13 00000080:00200008:0.0:1787825953.558457:0:154344:0:(file.c:6296:ll_getattr_dentry()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1787825953.558463:0:154344:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08),name=d71a.replay-single 00000002:00010000:0.0:1787825953.558478:0:154344:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x20001a9e3:0x17e0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825953.558484:0:154344:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00010000:00010000:0.0:1787825953.558545:0:154344:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e2b4 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 154344 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.558562:0:154344:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e2b4 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 154344 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.558573:0:154344:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e2b4 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 154344 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825953.558599:0:154344:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154344:1874663042813568:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787825953.558701:0:154344:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8a80 going to sleep for 11 seconds 00000100:00100000:0.0:1787825953.566111:0:154344:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154344:1874663042813568:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787825953.566145:0:154344:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e2b4 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce626a expref: -99 pid: 154344 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.566168:0:154344:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e2b4 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce626a expref: -99 pid: 154344 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.566177:0:154344:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e2b4 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce626a expref: -99 pid: 154344 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.566190:0:154344:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e2b4 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce626a expref: -99 pid: 154344 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.566202:0:154344:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e2b4 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce626a expref: -99 pid: 154344 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.566217:0:154344:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e2b4 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce626a expref: -99 pid: 154344 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.566225:0:154344:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e2b4 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce626a expref: -99 pid: 154344 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.566233:0:154344:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e2b4 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce626a expref: -99 pid: 154344 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825953.566246:0:154344:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c55a5c00 x1874663042813568/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787825969 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787825953.566263:0:154344:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e2b4 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce626a expref: -99 pid: 154344 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.566277:0:154344:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787825953.566286:0:154344:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1787825953.566325:0:154344:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x17e0:0x0], updating i_size 4096 00000080:00010000:0.0:1787825953.566332:0:154344:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) for lock 0xbef42c0221d6e2b4 00000080:00010000:0.0:1787825953.566342:0:154344:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00000080:00010000:0.0:1787825953.566346:0:154344:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6e2b4 from it ffffb97504b73cb8 00010000:00010000:0.0:1787825953.566349:0:154344:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e2b4 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce626a expref: -99 pid: 154344 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.566359:0:154344:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e2b4 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce626a expref: -99 pid: 154344 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:0.0:1787825953.571219:0:154344:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1787825953.571234:0:154344:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825953.571268:0:154344:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825953.571292:0:154344:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825953.571311:0:154344:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825953.571320:0:154344:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825953.571331:0:154344:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825953.571335:0:154344:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825953.571344:0:154344:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97504b73cb8 00010000:00010000:0.0:1787825953.571348:0:154344:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825953.571359:0:154344:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825953.571586:0:154344:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825953.571597:0:154344:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825953.571604:0:154344:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825953.571614:0:154344:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825953.571620:0:154344:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825953.571625:0:154344:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825953.571628:0:154344:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825953.571632:0:154344:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97504b73bf8 00010000:00010000:0.0:1787825953.571634:0:154344:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825953.571643:0:154344:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825953.571652:0:154344:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825953.571661:0:154344:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=267 00000080:00200000:0.0:1787825953.571670:0:154344:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 44 00000080:00200008:0.0:1787825953.571679:0:154344:0:(file.c:1090:ll_file_open()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00000080:00200080:0.0:1787825953.571712:0:154344:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) cmd=c03066f0 arg=7ffc9e7fc640 00000080:00200000:0.0:1787825953.571730:0:154344:0:(dir.c:451:ll_dir_setdirstripe()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) name=test stripe_offset=0 stripe_count=2, hash_type=0 00010000:00010000:0.0:1787825953.571793:0:154344:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e2b4 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ce626a expref: -99 pid: 154344 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825953.571813:0:154344:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6800/0xbef42c0221d64b3f lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cb9ffd expref: -99 pid: 152202 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825953.571872:0:154344:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e2b4 lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce626a expref: -99 pid: 154344 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825953.571881:0:154344:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787825953.571885:0:154344:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e2b4 lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce626a expref: -99 pid: 154344 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:0.0:1787825953.571906:0:154344:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787825953.571923:0:154344:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c55a5c00 x1874663042813824/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 552/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787825953.571952:0:154344:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154344:1874663042813824:192.168.203.147@tcp:36:lfs.0 00000100:00100000:0.0:1787825953.572054:0:154344:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8a80 going to sleep for 11 seconds 00000100:00100000:0.0:1787825953.596911:0:154344:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154344:1874663042813824:192.168.203.147@tcp:36:lfs.0 00000080:00200000:0.0:1787825953.597161:0:154344:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x20001a9e3:0x1b7c:0x0](ffff9181c83a1148) 00000080:00200000:0.0:1787825953.597170:0:154344:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1b7c:0x0], updating i_size 4096 00000080:00200008:0.0:1787825953.597212:0:154344:0:(file.c:479:ll_file_release()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00010000:00010000:0.0:1787825953.597247:0:154344:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513000/0xbef42c0221d64bee lrc: 2/0,0 mode: CR/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cba2b9 expref: -99 pid: 152210 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825953.620102:0:154349:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825953.620143:0:154349:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825953.620167:0:154349:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825953.620187:0:154349:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825953.620196:0:154349:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825953.620206:0:154349:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825953.620210:0:154349:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825953.620220:0:154349:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97504b53c20 00010000:00010000:0.0:1787825953.620224:0:154349:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825953.620233:0:154349:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825953.620251:0:154349:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825953.620260:0:154349:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=24 00000080:00200000:0.0:1787825953.620267:0:154349:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825953.620281:0:154349:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=0 00000080:00200000:0.0:1787825953.620286:0:154349:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787825953.620301:0:154349:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787825953.620358:0:154349:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e2bb lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154349 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.620366:0:154349:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e2bb lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154349 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.620382:0:154349:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e2bb lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154349 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825953.620414:0:154349:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154349:1874663042813952:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787825953.620537:0:154349:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8a80 going to sleep for 11 seconds 00010000:00010000:0.0:1787825953.634594:0:33329:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787825953.662971:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825953.663016:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048696576 00000100:00100000:0.0:1787825953.663026:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048696576 00000100:00100000:0.0:1787825953.663154:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9101 00000100:00100000:0.0:1787825953.663167:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:14309:x1874663048696576:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825953.663186:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481400/0xbef42c0221d6e2c2 lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x39d:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce62be expref: -99 pid: 154349 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825953.663264:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825953.663272:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:14309:x1874663048696576:12345-192.168.203.147@tcp:104: Request processed in 109us (311us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825953.663278:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9101 00000100:00100000:1.0:1787825953.680303:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787825953.680357:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048696832 00000100:00100000:1.0:1787825953.680369:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048696832 00000100:00100000:1.0:1787825953.680422:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9102 00000100:00100000:1.0:1787825953.680432:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b0e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8414:x1874663048696832:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787825953.680451:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6e2c9 lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x39d:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce62cc expref: -99 pid: 154349 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825953.680532:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787825953.680539:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b0e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8414:x1874663048696832:12345-192.168.203.147@tcp:104: Request processed in 108us (246us total) trans 0 rc 0/0 00000100:00100000:1.0:1787825953.680546:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9102 00000080:00200000:0.0:1787825953.769575:0:154353:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825953.769603:0:154353:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825953.769621:0:154353:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825953.769637:0:154353:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825953.769645:0:154353:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825953.769653:0:154353:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825953.769657:0:154353:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825953.769664:0:154353:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97500c97c20 00010000:00010000:0.0:1787825953.769668:0:154353:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825953.769677:0:154353:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825953.769691:0:154353:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825953.769697:0:154353:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=24 00000080:00200000:0.0:1787825953.769703:0:154353:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825953.769713:0:154353:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=0 00000080:00200000:0.0:1787825953.769717:0:154353:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787825953.769727:0:154353:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787825953.769784:0:154353:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6e2e5 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154353 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.769793:0:154353:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6e2e5 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154353 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.769802:0:154353:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6e2e5 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154353 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825953.769824:0:154353:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154353:1874663042815488:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787825953.770022:0:154353:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8a80 going to sleep for 11 seconds 00000002:00010000:1.0:1787825953.784067:0:154353:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x24000178a:0x39e:0x0]) in obj [0x24000178a:0x39e:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787825953.784135:0:154353:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6e2f3 lrc: 3/1,0 mode: --/CR res: [0x24000178a:0x39e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 154353 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825953.784144:0:154353:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6e2f3 lrc: 3/1,0 mode: --/CR res: [0x24000178a:0x39e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154353 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825953.784152:0:154353:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6e2f3 lrc: 3/1,0 mode: --/CR res: [0x24000178a:0x39e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154353 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787825953.784174:0:154353:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154353:1874663042815744:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:1.0:1787825953.784253:0:154353:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b59000 going to sleep for 11 seconds 00000100:00100000:1.0:1787825953.813381:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825953.813468:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048698240 00000100:00100000:0.0:1787825953.813477:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048698240 00000100:00100000:0.0:1787825953.813540:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9103 00000100:00100000:0.0:1787825953.813559:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6491:x1874663048698240:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825953.813576:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481400/0xbef42c0221d6e2ec lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x39e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce6382 expref: -99 pid: 154353 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825953.813652:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825953.813659:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6491:x1874663048698240:12345-192.168.203.147@tcp:104: Request processed in 110us (287us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825953.813666:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9103 00000100:00100000:1.0:1787825953.822281:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787825953.822328:0:33333:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:1.0:1787825953.822345:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048698496 00000100:00100000:1.0:1787825953.822352:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048698496 00000100:00100000:1.0:1787825953.822403:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9104 00000100:00100000:1.0:1787825953.822413:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c5cff450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663048698496:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787825953.822430:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6e2f3 lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x39e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce6390 expref: -99 pid: 154353 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825953.822506:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787825953.822513:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c5cff450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663048698496:12345-192.168.203.147@tcp:104: Request processed in 102us (243us total) trans 0 rc 0/0 00000100:00100000:1.0:1787825953.822521:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9104 00000100:00100000:1.0:1787825953.907932:0:154356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d805c380 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154356:1874663042816384:192.168.203.147@tcp:101:lfs.0 00010000:00010000:1.0:1787825953.907954:0:154356:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481400/0xbef42c0221d6e2fa lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154356 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825953.907968:0:154356:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481400/0xbef42c0221d6e2fa lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154356 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825953.907976:0:154356:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481400/0xbef42c0221d6e2fa lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154356 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825953.907982:0:154356:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481400/0xbef42c0221d6e2fa lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154356 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825953.907990:0:154356:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481400/0xbef42c0221d6e2fa lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154356 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825953.907996:0:154356:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3481400 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:1.0:1787825953.908001:0:154356:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481400/0xbef42c0221d6e2fa lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154356 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825953.908008:0:154356:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481400/0xbef42c0221d6e2fa lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154356 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825953.908015:0:154356:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481400/0xbef42c0221d6e2fa lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 154356 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825953.908023:0:154356:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787825953.908029:0:154356:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481400/0xbef42c0221d6e2fa lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154356 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825953.908035:0:154356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481400/0xbef42c0221d6e2fa lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154356 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787825953.908051:0:154356:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181d805c380 x1874663042816384/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787825969 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787825953.908066:0:154356:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:1.0:1787825953.908296:0:154356:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:1.0:1787825953.908356:0:154356:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787825953.908371:0:154356:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787825953.908380:0:154356:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825953.908390:0:154356:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825953.908397:0:154356:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787825953.908401:0:154356:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787825953.908404:0:154356:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787825953.908409:0:154356:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97500d53be8 00010000:00010000:1.0:1787825953.908413:0:154356:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825953.908420:0:154356:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825953.908433:0:154356:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787825953.908440:0:154356:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=16408 00000080:00200000:1.0:1787825953.908444:0:154356:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:1.0:1787825953.908462:0:154356:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=16392 00000080:00200000:1.0:1787825953.908465:0:154356:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:1.0:1787825953.908476:0:154356:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787825953.908523:0:154356:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5600/0xbef42c0221d6e301 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154356 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825953.908530:0:154356:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5600/0xbef42c0221d6e301 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154356 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825953.908539:0:154356:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5600/0xbef42c0221d6e301 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154356 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787825953.908568:0:154356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154356:1874663042816512:192.168.203.147@tcp:101:lfs.0 00000100:00100000:1.0:1787825953.908686:0:154356:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b59f00 going to sleep for 11 seconds 00000100:00100000:0.0:1787825953.932974:0:154356:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d805d180 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154356:1874663042816640:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787825953.933000:0:154356:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6e308 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce63f2 expref: -99 pid: 154356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.933016:0:154356:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6e308 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce63f2 expref: -99 pid: 154356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.933024:0:154356:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6e308 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce63f2 expref: -99 pid: 154356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.933034:0:154356:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6e308 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce63f2 expref: -99 pid: 154356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.933045:0:154356:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6e308 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce63f2 expref: -99 pid: 154356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.933056:0:154356:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6e308 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce63f2 expref: -99 pid: 154356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.933069:0:154356:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6e308 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce63f2 expref: -99 pid: 154356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.933212:0:154356:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6e308 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce63f2 expref: -99 pid: 154356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825953.933227:0:154356:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181d805d180 x1874663042816640/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787825969 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787825953.933241:0:154356:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6e308 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce63f2 expref: -99 pid: 154356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.933252:0:154356:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787825953.933259:0:154356:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1787825953.933285:0:154356:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x17e0:0x0], updating i_size 4096 00000080:00010000:0.0:1787825953.933293:0:154356:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) for lock 0xbef42c0221d6e308 00000080:00010000:0.0:1787825953.933300:0:154356:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00000080:00010000:0.0:1787825953.933304:0:154356:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6e308 from it ffffb97500d53cb8 00010000:00010000:0.0:1787825953.933307:0:154356:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6e308 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce63f2 expref: -99 pid: 154356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.933316:0:154356:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6e308 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce63f2 expref: -99 pid: 154356 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:0.0:1787825953.934323:0:154356:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1787825953.934331:0:154356:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825953.934349:0:154356:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825953.934362:0:154356:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825953.934374:0:154356:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825953.934380:0:154356:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825953.934385:0:154356:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825953.934388:0:154356:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825953.934392:0:154356:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97500d53cb8 00010000:00010000:0.0:1787825953.934395:0:154356:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825953.934403:0:154356:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825953.934526:0:154356:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825953.934534:0:154356:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825953.934541:0:154356:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825953.934559:0:154356:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825953.934565:0:154356:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825953.934569:0:154356:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825953.934571:0:154356:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825953.934575:0:154356:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97500d53bf8 00010000:00010000:0.0:1787825953.934578:0:154356:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825953.934584:0:154356:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825953.934592:0:154356:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825953.934599:0:154356:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=267 00000080:00200000:0.0:1787825953.934604:0:154356:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 44 00000080:00200008:0.0:1787825953.934610:0:154356:0:(file.c:1090:ll_file_open()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00000080:00200080:0.0:1787825953.934629:0:154356:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) cmd=c03066f0 arg=7ffdbe088260 00000080:00200000:0.0:1787825953.934641:0:154356:0:(dir.c:451:ll_dir_setdirstripe()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) name=test stripe_offset=0 stripe_count=2, hash_type=0 00010000:00010000:0.0:1787825953.934663:0:154356:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6e308 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ce63f2 expref: -99 pid: 154356 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825953.934678:0:154356:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6800/0xbef42c0221d64b3f lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cb9ffd expref: -99 pid: 152202 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825953.934720:0:154356:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6e308 lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce63f2 expref: -99 pid: 154356 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825953.934727:0:154356:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787825953.934730:0:154356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6e308 lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce63f2 expref: -99 pid: 154356 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:0.0:1787825953.934782:0:154356:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787825953.934792:0:154356:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c55a6d80 x1874663042816896/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 552/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787825953.934818:0:154356:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154356:1874663042816896:192.168.203.147@tcp:36:lfs.0 00000100:00100000:0.0:1787825953.935011:0:154356:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1d3da80 going to sleep for 11 seconds 00000080:00200000:0.0:1787825953.964878:0:154360:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825953.964906:0:154360:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825953.964924:0:154360:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825953.964940:0:154360:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825953.964948:0:154360:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825953.964956:0:154360:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825953.964959:0:154360:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825953.964967:0:154360:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97500d4bc20 00010000:00010000:0.0:1787825953.964971:0:154360:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825953.964980:0:154360:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825953.964994:0:154360:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825953.965001:0:154360:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=24 00000080:00200000:0.0:1787825953.965007:0:154360:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825953.965023:0:154360:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=0 00000080:00200000:0.0:1787825953.965027:0:154360:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787825953.965039:0:154360:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787825953.965082:0:154360:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e30f lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154360 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.965090:0:154360:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e30f lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154360 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.965100:0:154360:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e30f lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154360 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825953.965202:0:154360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154360:1874663042817024:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787825953.965719:0:154360:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1d3d300 going to sleep for 11 seconds 00000100:00100000:0.0:1787825953.976070:0:154360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154360:1874663042817024:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:0.0:1787825953.976096:0:154360:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e30f lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6438 expref: -99 pid: 154360 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.976117:0:154360:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e30f lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1b7e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6438 expref: -99 pid: 154360 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.976124:0:154360:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e30f lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1b7e:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6438 expref: -99 pid: 154360 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.976135:0:154360:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e30f lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1b7e:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6438 expref: -99 pid: 154360 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.976919:0:154360:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e30f lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1b7e:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6438 expref: -99 pid: 154360 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.976937:0:154360:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e30f lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1b7e:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6438 expref: -99 pid: 154360 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.976950:0:154360:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e30f lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1b7e:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6438 expref: -99 pid: 154360 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.976958:0:154360:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e30f lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1b7e:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6438 expref: -99 pid: 154360 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825953.977111:0:154360:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c55a5f80 x1874663042817024/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/672 e 0 to 0 dl 1787825969 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787825953.977126:0:154360:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e30f lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1b7e:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6438 expref: -99 pid: 154360 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.977138:0:154360:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/7038 (0 0) 00000080:00200000:0.0:1787825953.977144:0:154360:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00000002:00010000:0.0:1787825953.977780:0:154360:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200025d90:0x39f:0x0]) in obj [0x200025d90:0x39f:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825953.977904:0:154360:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512a00/0xbef42c0221d6e316 lrc: 3/1,0 mode: --/CR res: [0x200025d90:0x39f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 154360 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.977914:0:154360:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512a00/0xbef42c0221d6e316 lrc: 3/1,0 mode: --/CR res: [0x200025d90:0x39f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154360 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.977925:0:154360:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512a00/0xbef42c0221d6e316 lrc: 3/1,0 mode: --/CR res: [0x200025d90:0x39f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154360 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825953.977945:0:154360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154360:1874663042817152:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787825953.978559:0:154360:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1d3d300 going to sleep for 11 seconds 00000100:00100000:0.0:1787825953.991803:0:154360:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f1500 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154360:1874663042817280:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:0.0:1787825953.991832:0:154360:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6e31d lrc: 4/1,0 mode: --/CR res: [0x24000178a:0x39f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6454 expref: -99 pid: 154360 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.991844:0:154360:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6e31d lrc: 4/1,0 mode: --/PR res: [0x24000178a:0x39f:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6454 expref: -99 pid: 154360 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.991855:0:154360:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6e31d lrc: 4/1,0 mode: PR/PR res: [0x24000178a:0x39f:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6454 expref: -99 pid: 154360 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.991865:0:154360:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6e31d lrc: 4/1,0 mode: PR/PR res: [0x24000178a:0x39f:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6454 expref: -99 pid: 154360 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.991879:0:154360:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6e31d lrc: 5/2,0 mode: PR/PR res: [0x24000178a:0x39f:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6454 expref: -99 pid: 154360 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.991887:0:154360:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6e31d lrc: 5/2,0 mode: PR/PR res: [0x24000178a:0x39f:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6454 expref: -99 pid: 154360 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.991894:0:154360:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6e31d lrc: 4/1,0 mode: PR/PR res: [0x24000178a:0x39f:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6454 expref: -99 pid: 154360 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825953.991905:0:154360:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c71f1500 x1874663042817280/t0(0) o101->lustre-MDT0001-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787825969 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787825953.991919:0:154360:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6e31d lrc: 3/1,0 mode: PR/PR res: [0x24000178a:0x39f:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6454 expref: -99 pid: 154360 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.991932:0:154360:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d0f54800 type 13 mode 4 res 9663682442/927 (0 0) 00010000:00010000:0.0:1787825953.991940:0:154360:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6e31d lrc: 3/1,0 mode: PR/PR res: [0x24000178a:0x39f:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6454 expref: -99 pid: 154360 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.991948:0:154360:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6e31d lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x39f:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6454 expref: -99 pid: 154360 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787825953.991983:0:154360:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1b7e:0x0], updating i_size 8192 00000080:00200000:0.0:1787825953.991987:0:154360:0:(namei.c:147:ll_iget()) got inode: [0x20001a9e3:0x1b7e:0x0](ffff9181c83b9988): rc = 0 00000080:00010000:0.0:1787825953.991997:0:154360:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x1b7e:0x0](ffff9181c83b9988) for lock 0xbef42c0221d6e30f 00000080:00010000:0.0:1787825953.992013:0:154360:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x1b7e:0x0](ffff9181c83b9988) 00000080:00010000:0.0:1787825953.992017:0:154360:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6e30f from it ffffb97500d4be08 00010000:00010000:0.0:1787825953.992021:0:154360:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e30f lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1b7e:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6438 expref: -99 pid: 154360 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825953.992029:0:154360:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e30f lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1b7e:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6438 expref: -99 pid: 154360 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787825953.992048:0:154360:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 3 00000080:00200000:0.0:1787825953.992055:0:154360:0:(namei.c:2328:ll_rmdir()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00010000:00010000:0.0:1787825953.992081:0:154360:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e30f lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1b7e:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ce6438 expref: -99 pid: 154360 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825953.992123:0:154360:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e30f lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x1b7e:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce6438 expref: -99 pid: 154360 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825953.992129:0:154360:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787825953.992132:0:154360:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e30f lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x1b7e:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce6438 expref: -99 pid: 154360 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:0.0:1787825953.992160:0:154360:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787825953.992167:0:154360:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c55a5f80 x1874663042817536/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787825953.992199:0:154360:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154360:1874663042817536:192.168.203.147@tcp:36:rmdir.0 00000100:00100000:0.0:1787825953.992271:0:154360:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1d3d6c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787825953.996030:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825953.996296:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048699904 00000100:00100000:0.0:1787825953.996307:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048699904 00000100:00100000:0.0:1787825953.996365:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9105 00000100:00100000:0.0:1787825953.996375:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:41299:x1874663048699904:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825953.996392:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512a00/0xbef42c0221d6e316 lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x39f:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce6446 expref: -99 pid: 154360 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825953.996468:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825953.996474:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:41299:x1874663048699904:12345-192.168.203.147@tcp:104: Request processed in 102us (454us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825953.996481:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9105 00000100:00100000:0.0:1787825954.013898:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825954.015850:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048700160 00000100:00100000:0.0:1787825954.015865:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048700160 00000100:00100000:0.0:1787825954.015933:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9106 00000100:00100000:0.0:1787825954.015943:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663048700160:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825954.015959:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6e31d lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x39f:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce6454 expref: -99 pid: 154360 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825954.016044:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825954.016051:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663048700160:12345-192.168.203.147@tcp:104: Request processed in 110us (2160us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825954.016058:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9106 00000100:00100000:1.0:1787825954.090739:0:154367:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d805df80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154367:1874663042817920:192.168.203.147@tcp:101:lfs.0 00010000:00010000:1.0:1787825954.090810:0:154367:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480200/0xbef42c0221d6e324 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154367 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.090828:0:154367:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480200/0xbef42c0221d6e324 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154367 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.090837:0:154367:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480200/0xbef42c0221d6e324 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154367 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.090846:0:154367:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480200/0xbef42c0221d6e324 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154367 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.090858:0:154367:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480200/0xbef42c0221d6e324 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154367 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.090865:0:154367:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3480200 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:1.0:1787825954.090871:0:154367:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480200/0xbef42c0221d6e324 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154367 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.090885:0:154367:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480200/0xbef42c0221d6e324 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154367 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.090893:0:154367:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480200/0xbef42c0221d6e324 lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 154367 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.090990:0:154367:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787825954.090999:0:154367:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480200/0xbef42c0221d6e324 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154367 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.091007:0:154367:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480200/0xbef42c0221d6e324 lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154367 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787825954.091029:0:154367:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181d805df80 x1874663042817920/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787825970 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787825954.091048:0:154367:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:1.0:1787825954.091085:0:154367:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:1.0:1787825954.091286:0:154367:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787825954.091307:0:154367:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787825954.091318:0:154367:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825954.091333:0:154367:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825954.091340:0:154367:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787825954.091346:0:154367:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787825954.091350:0:154367:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787825954.091356:0:154367:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97500d9bbe8 00010000:00010000:1.0:1787825954.091360:0:154367:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825954.091369:0:154367:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825954.091388:0:154367:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787825954.091397:0:154367:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=16408 00000080:00200000:1.0:1787825954.091404:0:154367:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:1.0:1787825954.091427:0:154367:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=16392 00000080:00200000:1.0:1787825954.091433:0:154367:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:1.0:1787825954.091448:0:154367:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787825954.091509:0:154367:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5800/0xbef42c0221d6e32b lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154367 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.091518:0:154367:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5800/0xbef42c0221d6e32b lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154367 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.091529:0:154367:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5800/0xbef42c0221d6e32b lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154367 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787825954.093100:0:154367:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154367:1874663042818048:192.168.203.147@tcp:101:lfs.0 00000100:00100000:1.0:1787825954.093734:0:154367:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b593c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787825954.101829:0:154367:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154367:1874663042818048:192.168.203.147@tcp:101:lfs.0 00010000:00010000:1.0:1787825954.101860:0:154367:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5800/0xbef42c0221d6e32b lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154367 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.101877:0:154367:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5800/0xbef42c0221d6e32b lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154367 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.101887:0:154367:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5800/0xbef42c0221d6e32b lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154367 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.101895:0:154367:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5800/0xbef42c0221d6e32b lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154367 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.101919:0:154367:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5800/0xbef42c0221d6e32b lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154367 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.101926:0:154367:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b5800 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:1.0:1787825954.101931:0:154367:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5800/0xbef42c0221d6e32b lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154367 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.101938:0:154367:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5800/0xbef42c0221d6e32b lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154367 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.101946:0:154367:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5800/0xbef42c0221d6e32b lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 154367 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.101954:0:154367:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787825954.101961:0:154367:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5800/0xbef42c0221d6e32b lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154367 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.101968:0:154367:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5800/0xbef42c0221d6e32b lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154367 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787825954.101986:0:154367:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c35b0e00 x1874663042818048/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787825970 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787825954.102002:0:154367:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:1.0:1787825954.102035:0:154367:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:1.0:1787825954.102112:0:154367:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787825954.102130:0:154367:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787825954.102140:0:154367:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825954.102153:0:154367:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825954.102161:0:154367:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787825954.102167:0:154367:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787825954.102171:0:154367:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787825954.102177:0:154367:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97500d9bb58 00010000:00010000:1.0:1787825954.102181:0:154367:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825954.102190:0:154367:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825954.102207:0:154367:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787825954.102216:0:154367:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=13 00000080:00200008:1.0:1787825954.102227:0:154367:0:(file.c:6296:ll_getattr_dentry()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), request_mask 0, flags 0, foreign 0 00000080:00200000:1.0:1787825954.102234:0:154367:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08),name=d71a.replay-single 00000002:00010000:1.0:1787825954.102248:0:154367:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x20001a9e3:0x17e0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787825954.102254:0:154367:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00010000:00010000:1.0:1787825954.102300:0:154367:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6e332 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 154367 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.102313:0:154367:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6e332 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 154367 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.102324:0:154367:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6e332 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 154367 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787825954.102349:0:154367:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154367:1874663042818176:192.168.203.147@tcp:101:lfs.0 00000100:00100000:1.0:1787825954.102448:0:154367:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b593c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787825954.200729:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825954.200815:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048701568 00000100:00100000:0.0:1787825954.200841:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048701568 00000100:00100000:0.0:1787825954.200905:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9107 00000100:00100000:0.0:1787825954.200918:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a4000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6490:x1874663048701568:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825954.200942:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6e340 lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x3a0:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce650a expref: -99 pid: 154370 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825954.201033:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825954.201041:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a4000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6490:x1874663048701568:12345-192.168.203.147@tcp:104: Request processed in 125us (326us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825954.201049:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9107 00000100:00100000:1.0:1787825954.205680:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787825954.205730:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048701824 00000100:00100000:1.0:1787825954.205739:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048701824 00000100:00100000:1.0:1787825954.205828:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9108 00000100:00100000:1.0:1787825954.205839:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663048701824:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787825954.205861:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481000/0xbef42c0221d6e347 lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x3a0:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce6518 expref: -99 pid: 154370 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825954.205956:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787825954.205964:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663048701824:12345-192.168.203.147@tcp:104: Request processed in 126us (303us total) trans 0 rc 0/0 00000100:00100000:1.0:1787825954.205973:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9108 00000080:00200000:0.0:1787825954.265610:0:154380:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825954.265653:0:154380:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825954.265677:0:154380:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825954.265696:0:154380:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825954.265705:0:154380:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825954.265715:0:154380:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825954.265719:0:154380:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825954.265729:0:154380:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb9750150bb58 00010000:00010000:0.0:1787825954.265733:0:154380:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825954.265772:0:154380:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825954.265937:0:154380:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825954.265949:0:154380:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=29 00000080:00200000:0.0:1787825954.265957:0:154380:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825954.265971:0:154380:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=13 00000080:00200000:0.0:1787825954.265976:0:154380:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787825954.265992:0:154380:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787825954.266057:0:154380:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e34e lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825954.266067:0:154380:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e34e lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825954.266079:0:154380:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e34e lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154380 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825954.266113:0:154380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154380:1874663042819456:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787825954.266220:0:154380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c4aec240 going to sleep for 11 seconds 00000100:00100000:1.0:1787825954.271227:0:154380:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154380:1874663042819456:192.168.203.147@tcp:101:lfs.0 00010000:00010000:1.0:1787825954.271260:0:154380:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e34e lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.271279:0:154380:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e34e lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.271293:0:154380:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e34e lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.271301:0:154380:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e34e lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.271311:0:154380:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e34e lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.271318:0:154380:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0512c00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:1.0:1787825954.271323:0:154380:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e34e lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.271331:0:154380:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e34e lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.271340:0:154380:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e34e lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 154380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.271349:0:154380:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787825954.271358:0:154380:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e34e lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.271366:0:154380:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e34e lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154380 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787825954.271386:0:154380:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c55a4700 x1874663042819456/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787825970 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787825954.271404:0:154380:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:1.0:1787825954.271975:0:154380:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:1.0:1787825954.272066:0:154380:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787825954.272086:0:154380:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787825954.272099:0:154380:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825954.272114:0:154380:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825954.272122:0:154380:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787825954.272130:0:154380:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787825954.272134:0:154380:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787825954.272142:0:154380:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb9750150bbe8 00010000:00010000:1.0:1787825954.272146:0:154380:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825954.272157:0:154380:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825954.272177:0:154380:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787825954.272187:0:154380:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=16408 00000080:00200000:1.0:1787825954.272195:0:154380:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:1.0:1787825954.272214:0:154380:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=16392 00000080:00200000:1.0:1787825954.272220:0:154380:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:1.0:1787825954.272235:0:154380:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787825954.272292:0:154380:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6e355 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.272300:0:154380:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6e355 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.272312:0:154380:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6e355 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154380 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787825954.272340:0:154380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154380:1874663042819584:192.168.203.147@tcp:101:lfs.0 00000080:00200000:0.0:1787825954.331184:0:154389:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825954.331228:0:154389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825954.331254:0:154389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825954.343799:0:154389:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825954.343816:0:154389:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787825954.344067:0:154389:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787825954.344073:0:154389:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787825954.344086:0:154389:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97500c97c20 00010000:00010000:1.0:1787825954.344091:0:154389:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825954.344103:0:154389:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825954.344126:0:154389:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787825954.344141:0:154389:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=24 00000080:00200000:1.0:1787825954.344152:0:154389:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:1.0:1787825954.344171:0:154389:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=0 00000080:00200000:1.0:1787825954.344176:0:154389:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:1.0:1787825954.344203:0:154389:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787825954.344268:0:154389:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5800/0xbef42c0221d6e363 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.344277:0:154389:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5800/0xbef42c0221d6e363 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154389 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.344289:0:154389:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5800/0xbef42c0221d6e363 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154389 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787825954.344321:0:154389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154389:1874663042820096:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:1.0:1787825954.344431:0:154389:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b59840 going to sleep for 11 seconds 00000100:00100000:1.0:1787825954.419967:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825954.420291:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048703232 00000100:00100000:0.0:1787825954.420305:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048703232 00000100:00100000:0.0:1787825954.420370:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9109 00000100:00100000:0.0:1787825954.420382:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b1c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6491:x1874663048703232:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825954.420404:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6e36a lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x3a1:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce65ce expref: -99 pid: 154389 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825954.420502:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825954.420509:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b1c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6491:x1874663048703232:12345-192.168.203.147@tcp:104: Request processed in 129us (557us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825954.420517:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9109 00000100:00100000:1.0:1787825954.445982:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825954.446233:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048703488 00000100:00100000:0.0:1787825954.446247:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048703488 00000100:00100000:0.0:1787825954.446329:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9110 00000100:00100000:0.0:1787825954.446342:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b3b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663048703488:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825954.446364:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481000/0xbef42c0221d6e371 lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x3a1:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce65dc expref: -99 pid: 154389 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825954.446463:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825954.446470:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b3b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663048703488:12345-192.168.203.147@tcp:104: Request processed in 132us (504us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825954.446478:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9110 00000080:00200000:0.0:1787825954.485860:0:154392:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825954.485902:0:154392:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825954.485929:0:154392:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825954.485949:0:154392:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825954.485959:0:154392:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825954.485968:0:154392:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825954.485973:0:154392:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825954.485982:0:154392:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb975014fbb58 00010000:00010000:0.0:1787825954.485987:0:154392:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825954.485997:0:154392:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825954.486017:0:154392:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825954.486028:0:154392:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=29 00000080:00200000:0.0:1787825954.486035:0:154392:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825954.486050:0:154392:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=13 00000080:00200000:0.0:1787825954.486055:0:154392:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787825954.486069:0:154392:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787825954.486125:0:154392:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6e378 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154392 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825954.486134:0:154392:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6e378 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154392 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825954.486146:0:154392:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6e378 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154392 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825954.486180:0:154392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154392:1874663042820992:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787825954.486299:0:154392:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c4aec780 going to sleep for 11 seconds 00000100:00100000:1.0:1787825954.514469:0:154392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f2d80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154392:1874663042821248:192.168.203.147@tcp:101:lfs.0 00010000:00010000:1.0:1787825954.514520:0:154392:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7800/0xbef42c0221d6e386 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce663e expref: -99 pid: 154392 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.514553:0:154392:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7800/0xbef42c0221d6e386 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce663e expref: -99 pid: 154392 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.514564:0:154392:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7800/0xbef42c0221d6e386 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce663e expref: -99 pid: 154392 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.514577:0:154392:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7800/0xbef42c0221d6e386 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce663e expref: -99 pid: 154392 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.514590:0:154392:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7800/0xbef42c0221d6e386 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce663e expref: -99 pid: 154392 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.514606:0:154392:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7800/0xbef42c0221d6e386 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce663e expref: -99 pid: 154392 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.514621:0:154392:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7800/0xbef42c0221d6e386 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce663e expref: -99 pid: 154392 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.514629:0:154392:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7800/0xbef42c0221d6e386 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce663e expref: -99 pid: 154392 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787825954.514643:0:154392:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c71f2d80 x1874663042821248/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787825970 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787825954.514661:0:154392:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7800/0xbef42c0221d6e386 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce663e expref: -99 pid: 154392 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.514673:0:154392:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/6112 (0 0) 00000080:00200000:1.0:1787825954.514682:0:154392:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787825954.514721:0:154392:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x17e0:0x0], updating i_size 4096 00000080:00010000:1.0:1787825954.514728:0:154392:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) for lock 0xbef42c0221d6e386 00000080:00010000:1.0:1787825954.514737:0:154392:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00000080:00010000:1.0:1787825954.514740:0:154392:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6e386 from it ffffb975014fbcb8 00010000:00010000:1.0:1787825954.514766:0:154392:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7800/0xbef42c0221d6e386 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce663e expref: -99 pid: 154392 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.514777:0:154392:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7800/0xbef42c0221d6e386 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce663e expref: -99 pid: 154392 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:1.0:1787825954.515470:0:154392:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), request_mask 0, flags 0, foreign 0 00000080:00200000:1.0:1787825954.515479:0:154392:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787825954.515511:0:154392:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787825954.515526:0:154392:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825954.515540:0:154392:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825954.515547:0:154392:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787825954.515553:0:154392:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787825954.515556:0:154392:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787825954.515561:0:154392:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb975014fbcb8 00010000:00010000:1.0:1787825954.515564:0:154392:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825954.515573:0:154392:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825954.515713:0:154392:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787825954.515722:0:154392:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787825954.515731:0:154392:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825954.515739:0:154392:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825954.515772:0:154392:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787825954.515777:0:154392:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787825954.515779:0:154392:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787825954.515783:0:154392:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb975014fbbf8 00010000:00010000:1.0:1787825954.515787:0:154392:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825954.515795:0:154392:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825954.515804:0:154392:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787825954.515811:0:154392:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=267 00000080:00200000:1.0:1787825954.515817:0:154392:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 44 00000080:00200008:1.0:1787825954.515824:0:154392:0:(file.c:1090:ll_file_open()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00000080:00200080:1.0:1787825954.515850:0:154392:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) cmd=c03066f0 arg=7fff1443b6e0 00000080:00200000:1.0:1787825954.515866:0:154392:0:(dir.c:451:ll_dir_setdirstripe()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) name=test stripe_offset=0 stripe_count=2, hash_type=0 00010000:00010000:1.0:1787825954.515897:0:154392:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7800/0xbef42c0221d6e386 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ce663e expref: -99 pid: 154392 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825954.515917:0:154392:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6800/0xbef42c0221d64b3f lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cb9ffd expref: -99 pid: 152202 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825954.515970:0:154392:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7800/0xbef42c0221d6e386 lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce663e expref: -99 pid: 154392 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825954.515978:0:154392:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787825954.515982:0:154392:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7800/0xbef42c0221d6e386 lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce663e expref: -99 pid: 154392 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:1.0:1787825954.516004:0:154392:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787825954.516017:0:154392:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c35b1180 x1874663042821504/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 552/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787825954.516050:0:154392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154392:1874663042821504:192.168.203.147@tcp:36:lfs.0 00000100:00100000:1.0:1787825954.517537:0:154392:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b59c00 going to sleep for 11 seconds 00010000:00010000:1.0:1787825954.531237:0:154392:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513000/0xbef42c0221d64bee lrc: 2/0,0 mode: CR/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cba2b9 expref: -99 pid: 152210 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825954.552205:0:154398:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787825954.552243:0:154398:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787825954.552265:0:154398:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825954.552283:0:154398:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825954.552292:0:154398:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787825954.552302:0:154398:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787825954.552306:0:154398:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787825954.552314:0:154398:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb9750161bc20 00010000:00010000:1.0:1787825954.552318:0:154398:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825954.552328:0:154398:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825954.552346:0:154398:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787825954.552355:0:154398:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=24 00000080:00200000:1.0:1787825954.552361:0:154398:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:1.0:1787825954.552376:0:154398:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=0 00000080:00200000:1.0:1787825954.552382:0:154398:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:1.0:1787825954.552398:0:154398:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787825954.552454:0:154398:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5600/0xbef42c0221d6e38d lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154398 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.552463:0:154398:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5600/0xbef42c0221d6e38d lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154398 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.552473:0:154398:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5600/0xbef42c0221d6e38d lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154398 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787825954.552832:0:154398:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154398:1874663042821632:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:1.0:1787825954.552982:0:154398:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b59c00 going to sleep for 11 seconds 00000100:00100000:1.0:1787825954.590850:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787825954.590906:0:33333:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:1.0:1787825954.590928:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048704896 00000100:00100000:1.0:1787825954.590940:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048704896 00000100:00100000:1.0:1787825954.591003:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9111 00000100:00100000:1.0:1787825954.591015:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c4409050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6492:x1874663048704896:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787825954.591037:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7400/0xbef42c0221d6e394 lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x3a2:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce6692 expref: -99 pid: 154398 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825954.591129:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787825954.591136:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c4409050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6492:x1874663048704896:12345-192.168.203.147@tcp:104: Request processed in 123us (300us total) trans 0 rc 0/0 00000100:00100000:1.0:1787825954.591144:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9111 00000100:00100000:0.0:1787825954.601190:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825954.601245:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048705152 00000100:00100000:0.0:1787825954.601259:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048705152 00000100:00100000:0.0:1787825954.601341:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9112 00000100:00100000:0.0:1787825954.601355:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a5880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663048705152:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825954.601382:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f7e00/0xbef42c0221d6e39b lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x3a2:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce66a0 expref: -99 pid: 154398 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825954.601532:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825954.601540:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a5880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663048705152:12345-192.168.203.147@tcp:104: Request processed in 188us (365us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825954.601555:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9112 00000080:00200000:0.0:1787825954.676679:0:154403:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825954.676715:0:154403:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825954.676741:0:154403:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825954.676785:0:154403:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825954.676795:0:154403:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825954.676806:0:154403:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825954.676810:0:154403:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825954.676820:0:154403:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501813b58 00010000:00010000:0.0:1787825954.676825:0:154403:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825954.676835:0:154403:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825954.676854:0:154403:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825954.676862:0:154403:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=29 00000080:00200000:0.0:1787825954.676869:0:154403:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825954.676883:0:154403:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=13 00000080:00200000:0.0:1787825954.676889:0:154403:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787825954.676905:0:154403:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787825954.676966:0:154403:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6e3a2 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154403 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825954.676975:0:154403:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6e3a2 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154403 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825954.676987:0:154403:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6e3a2 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154403 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825954.677019:0:154403:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154403:1874663042822528:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787825954.677143:0:154403:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c4aeccc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787825954.684031:0:154403:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154403:1874663042822528:192.168.203.147@tcp:101:lfs.0 00010000:00010000:1.0:1787825954.684061:0:154403:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6e3a2 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154403 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.684078:0:154403:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6e3a2 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154403 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.684086:0:154403:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6e3a2 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154403 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.684095:0:154403:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6e3a2 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154403 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.684104:0:154403:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6e3a2 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154403 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.684110:0:154403:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0512e00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:1.0:1787825954.684115:0:154403:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6e3a2 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154403 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.684122:0:154403:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6e3a2 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154403 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.684129:0:154403:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6e3a2 lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 154403 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.684137:0:154403:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787825954.684144:0:154403:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6e3a2 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154403 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.684151:0:154403:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6e3a2 lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154403 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787825954.684171:0:154403:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c55a7b80 x1874663042822528/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787825970 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787825954.684188:0:154403:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:1.0:1787825954.684220:0:154403:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:1.0:1787825954.684303:0:154403:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787825954.684320:0:154403:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787825954.684330:0:154403:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825954.684352:0:154403:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825954.684360:0:154403:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787825954.684365:0:154403:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787825954.684369:0:154403:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787825954.684375:0:154403:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501813be8 00010000:00010000:1.0:1787825954.684378:0:154403:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825954.684388:0:154403:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825954.684404:0:154403:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787825954.684418:0:154403:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=16408 00000080:00200000:1.0:1787825954.684424:0:154403:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:1.0:1787825954.684451:0:154403:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=16392 00000080:00200000:1.0:1787825954.684456:0:154403:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:1.0:1787825954.684470:0:154403:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787825954.684531:0:154403:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e3a9 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154403 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.684550:0:154403:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e3a9 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154403 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.684561:0:154403:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e3a9 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154403 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787825954.684586:0:154403:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154403:1874663042822656:192.168.203.147@tcp:101:lfs.0 00000100:00100000:1.0:1787825954.684686:0:154403:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b596c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787825954.688909:0:154403:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154403:1874663042822656:192.168.203.147@tcp:101:lfs.0 00010000:00010000:1.0:1787825954.688951:0:154403:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e3a9 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154403 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.688968:0:154403:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e3a9 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154403 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.688975:0:154403:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e3a9 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154403 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.688984:0:154403:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e3a9 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154403 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.688994:0:154403:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e3a9 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154403 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.689001:0:154403:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b5c00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:1.0:1787825954.689007:0:154403:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e3a9 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154403 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.689014:0:154403:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e3a9 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154403 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.689022:0:154403:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e3a9 lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 154403 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.689030:0:154403:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787825954.689037:0:154403:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e3a9 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154403 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.689043:0:154403:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e3a9 lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154403 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787825954.689064:0:154403:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c35b2a00 x1874663042822656/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787825970 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787825954.689081:0:154403:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:1.0:1787825954.689246:0:154403:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:1.0:1787825954.689345:0:154403:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787825954.689363:0:154403:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787825954.689373:0:154403:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825954.689386:0:154403:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825954.689395:0:154403:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787825954.689401:0:154403:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787825954.689405:0:154403:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787825954.689411:0:154403:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501813b58 00010000:00010000:1.0:1787825954.689415:0:154403:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825954.689425:0:154403:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825954.689442:0:154403:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787825954.689451:0:154403:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=13 00000080:00200008:1.0:1787825954.689462:0:154403:0:(file.c:6296:ll_getattr_dentry()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), request_mask 0, flags 0, foreign 0 00000080:00200000:1.0:1787825954.689468:0:154403:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08),name=d71a.replay-single 00000002:00010000:1.0:1787825954.689481:0:154403:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x20001a9e3:0x17e0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787825954.689487:0:154403:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00010000:00010000:1.0:1787825954.689616:0:154403:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5400/0xbef42c0221d6e3b0 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 154403 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.689623:0:154403:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5400/0xbef42c0221d6e3b0 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 154403 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.689632:0:154403:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5400/0xbef42c0221d6e3b0 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 154403 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787825954.689657:0:154403:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154403:1874663042822784:192.168.203.147@tcp:101:lfs.0 00000100:00100000:1.0:1787825954.689796:0:154403:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b596c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787825954.700952:0:154403:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154403:1874663042822784:192.168.203.147@tcp:101:lfs.0 00010000:00010000:1.0:1787825954.700989:0:154403:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5400/0xbef42c0221d6e3b0 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6702 expref: -99 pid: 154403 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.701011:0:154403:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5400/0xbef42c0221d6e3b0 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6702 expref: -99 pid: 154403 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.701025:0:154403:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5400/0xbef42c0221d6e3b0 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6702 expref: -99 pid: 154403 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.701039:0:154403:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5400/0xbef42c0221d6e3b0 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6702 expref: -99 pid: 154403 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.701052:0:154403:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5400/0xbef42c0221d6e3b0 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6702 expref: -99 pid: 154403 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.701068:0:154403:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5400/0xbef42c0221d6e3b0 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6702 expref: -99 pid: 154403 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.701077:0:154403:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5400/0xbef42c0221d6e3b0 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6702 expref: -99 pid: 154403 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.701085:0:154403:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5400/0xbef42c0221d6e3b0 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6702 expref: -99 pid: 154403 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787825954.701099:0:154403:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c35b2a00 x1874663042822784/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787825970 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787825954.701117:0:154403:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5400/0xbef42c0221d6e3b0 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6702 expref: -99 pid: 154403 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.701128:0:154403:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/6112 (0 0) 00000080:00200000:1.0:1787825954.701136:0:154403:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787825954.701172:0:154403:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x17e0:0x0], updating i_size 4096 00000080:00010000:1.0:1787825954.701181:0:154403:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) for lock 0xbef42c0221d6e3b0 00000080:00010000:1.0:1787825954.701190:0:154403:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00000080:00010000:1.0:1787825954.701193:0:154403:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6e3b0 from it ffffb97501813cb8 00010000:00010000:1.0:1787825954.701197:0:154403:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5400/0xbef42c0221d6e3b0 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6702 expref: -99 pid: 154403 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.701206:0:154403:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5400/0xbef42c0221d6e3b0 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6702 expref: -99 pid: 154403 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:1.0:1787825954.710987:0:154403:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), request_mask 0, flags 0, foreign 0 00000080:00200000:1.0:1787825954.711002:0:154403:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787825954.711033:0:154403:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787825954.711058:0:154403:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825954.711078:0:154403:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825954.711088:0:154403:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787825954.711098:0:154403:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787825954.711102:0:154403:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787825954.711112:0:154403:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501813cb8 00010000:00010000:1.0:1787825954.711116:0:154403:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825954.711127:0:154403:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000100:00100000:0.0:1787825954.797700:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787825954.797905:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048706560 00000100:00100000:1.0:1787825954.797919:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048706560 00000100:00100000:1.0:1787825954.797987:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9113 00000100:00100000:1.0:1787825954.798002:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a7b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:14309:x1874663048706560:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787825954.798025:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6e3be lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x3a3:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce6756 expref: -99 pid: 154404 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825954.798136:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787825954.798143:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a7b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:14309:x1874663048706560:12345-192.168.203.147@tcp:104: Request processed in 145us (462us total) trans 0 rc 0/0 00000100:00100000:1.0:1787825954.798152:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9113 00000100:00100000:1.0:1787825954.810347:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825954.810781:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048706816 00000100:00100000:0.0:1787825954.810795:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048706816 00000100:00100000:0.0:1787825954.811141:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9114 00000100:00100000:0.0:1787825954.811155:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663048706816:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825954.811177:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480600/0xbef42c0221d6e3c5 lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x3a3:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce6764 expref: -99 pid: 154404 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825954.811276:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825954.811285:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663048706816:12345-192.168.203.147@tcp:104: Request processed in 133us (952us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825954.811293:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9114 00000080:00200000:1.0:1787825954.947101:0:154406:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787825954.947142:0:154406:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787825954.947995:0:154406:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825954.948022:0:154406:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825954.948033:0:154406:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787825954.948044:0:154406:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787825954.948048:0:154406:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787825954.948058:0:154406:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97504a6bc20 00010000:00010000:1.0:1787825954.948062:0:154406:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825954.948076:0:154406:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825954.948096:0:154406:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787825954.948107:0:154406:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=24 00000080:00200000:1.0:1787825954.948116:0:154406:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:1.0:1787825954.948135:0:154406:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=0 00000080:00200000:1.0:1787825954.948139:0:154406:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:1.0:1787825954.948314:0:154406:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787825954.948374:0:154406:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6e3e1 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154406 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.948382:0:154406:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6e3e1 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154406 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825954.948393:0:154406:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6e3e1 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154406 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787825954.948424:0:154406:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2300 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154406:1874663042824704:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:1.0:1787825954.953100:0:154406:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b59840 going to sleep for 11 seconds 00000100:00100000:0.0:1787825954.990090:0:154406:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d805c700 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154406:1874663042824832:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:0.0:1787825954.990129:0:154406:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6e3e8 lrc: 4/1,0 mode: --/CR res: [0x200025d90:0x3a4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce681a expref: -99 pid: 154406 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825954.990145:0:154406:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6e3e8 lrc: 4/1,0 mode: --/PR res: [0x200025d90:0x3a4:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce681a expref: -99 pid: 154406 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825954.990174:0:154406:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6e3e8 lrc: 4/1,0 mode: PR/PR res: [0x200025d90:0x3a4:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce681a expref: -99 pid: 154406 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825954.990188:0:154406:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6e3e8 lrc: 4/1,0 mode: PR/PR res: [0x200025d90:0x3a4:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce681a expref: -99 pid: 154406 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825954.990203:0:154406:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6e3e8 lrc: 5/2,0 mode: PR/PR res: [0x200025d90:0x3a4:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce681a expref: -99 pid: 154406 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825954.990212:0:154406:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6e3e8 lrc: 5/2,0 mode: PR/PR res: [0x200025d90:0x3a4:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce681a expref: -99 pid: 154406 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825954.990219:0:154406:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6e3e8 lrc: 4/1,0 mode: PR/PR res: [0x200025d90:0x3a4:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce681a expref: -99 pid: 154406 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825954.990237:0:154406:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181d805c700 x1874663042824832/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787825970 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787825954.990253:0:154406:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6e3e8 lrc: 3/1,0 mode: PR/PR res: [0x200025d90:0x3a4:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce681a expref: -99 pid: 154406 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825954.990269:0:154406:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590089616/932 (0 0) 00010000:00010000:0.0:1787825954.990277:0:154406:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6e3e8 lrc: 3/1,0 mode: PR/PR res: [0x200025d90:0x3a4:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce681a expref: -99 pid: 154406 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825954.990287:0:154406:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6e3e8 lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x3a4:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce681a expref: -99 pid: 154406 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000002:00010000:0.0:1787825954.990321:0:154406:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x24000178a:0x3a4:0x0]) in obj [0x24000178a:0x3a4:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825954.990388:0:154406:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6e3ef lrc: 3/1,0 mode: --/CR res: [0x24000178a:0x3a4:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 154406 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825954.990396:0:154406:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6e3ef lrc: 3/1,0 mode: --/CR res: [0x24000178a:0x3a4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154406 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825954.990405:0:154406:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6e3ef lrc: 3/1,0 mode: --/CR res: [0x24000178a:0x3a4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154406 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825954.990430:0:154406:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154406:1874663042824960:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787825954.990687:0:154406:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c4aec240 going to sleep for 11 seconds 00000100:00100000:1.0:1787825955.031957:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825955.032001:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048708224 00000100:00100000:0.0:1787825955.032016:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048708224 00000100:00100000:0.0:1787825955.032088:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9115 00000100:00100000:0.0:1787825955.032101:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b1c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6491:x1874663048708224:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825955.034502:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6e3e8 lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x3a4:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce681a expref: -99 pid: 154406 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825955.034897:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825955.034914:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b1c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6491:x1874663048708224:12345-192.168.203.147@tcp:104: Request processed in 2808us (2963us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825955.034927:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9115 00010000:00010000:0.0:1787825955.034967:0:33329:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787825955.034975:0:33329:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6e3e8 lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x3a4:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce681a expref: -99 pid: 154406 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825955.034984:0:33329:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3481a00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787825955.034990:0:33329:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6e3e8 lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x3a4:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00ce681a expref: -99 pid: 154406 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825955.034999:0:33329:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6e3e8 lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x3a4:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00ce681a expref: -99 pid: 154406 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825955.035008:0:33329:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6e3e8 lrc: 3/0,0 mode: PR/PR res: [0x200025d90:0x3a4:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x408400000000 nid: local remote: 0x878a5eea00ce681a expref: -99 pid: 154406 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825955.035088:0:33329:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6e3e8 lrc: 2/0,0 mode: --/PR res: [0x200025d90:0x3a4:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ce681a expref: -99 pid: 154406 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825955.035096:0:33329:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787825955.036513:0:33329:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6e3e8 lrc: 1/0,0 mode: --/PR res: [0x200025d90:0x3a4:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ce681a expref: -99 pid: 154406 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825955.037365:0:33329:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6e3e8 lrc: 0/0,0 mode: --/PR res: [0x200025d90:0x3a4:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ce681a expref: -99 pid: 154406 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825955.037478:0:33329:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787825955.037480:0:33329:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787825955.037483:0:33329:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787825955.037529:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042825344:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787825955.042368:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825955.044572:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048708480 00000100:00100000:0.0:1787825955.044596:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048708480 00000100:00100000:0.0:1787825955.044676:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9116 00000100:00100000:0.0:1787825955.044707:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b3b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663048708480:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825955.044731:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6e3ef lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x3a4:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce6828 expref: -99 pid: 154406 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825955.044864:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825955.044871:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b3b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663048708480:12345-192.168.203.147@tcp:104: Request processed in 175us (2521us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825955.044880:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9116 00000100:00100000:0.0:1787825955.045093:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042825344:192.168.203.147@tcp:103:ldlm_bl.0 00000080:00200000:1.0:1787825955.085605:0:154419:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787825955.085648:0:154419:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787825955.085674:0:154419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825955.085701:0:154419:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825955.085710:0:154419:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787825955.085721:0:154419:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787825955.085726:0:154419:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787825955.085736:0:154419:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb975019c3b58 00010000:00010000:1.0:1787825955.085740:0:154419:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825955.085769:0:154419:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825955.085792:0:154419:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787825955.085803:0:154419:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=29 00000080:00200000:1.0:1787825955.085811:0:154419:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:1.0:1787825955.085823:0:154419:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=13 00000080:00200000:1.0:1787825955.085828:0:154419:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:1.0:1787825955.085846:0:154419:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787825955.085906:0:154419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5000/0xbef42c0221d6e3f6 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825955.085914:0:154419:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5000/0xbef42c0221d6e3f6 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154419 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825955.085925:0:154419:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5000/0xbef42c0221d6e3f6 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154419 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787825955.085962:0:154419:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154419:1874663042825600:192.168.203.147@tcp:101:lfs.0 00000100:00100000:1.0:1787825955.086079:0:154419:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b59000 going to sleep for 11 seconds 00000100:00100000:0.0:1787825955.113672:0:154419:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5cb6600 going to sleep for 11 seconds 00000080:00010000:0.0:1787825955.130862:0:154419:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00000080:00010000:0.0:1787825955.130869:0:154419:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6e404 from it ffffb975019c3cb8 00010000:00010000:0.0:1787825955.130874:0:154419:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7400/0xbef42c0221d6e404 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce688a expref: -99 pid: 154419 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.130889:0:154419:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7400/0xbef42c0221d6e404 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce688a expref: -99 pid: 154419 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:0.0:1787825955.137081:0:154419:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1787825955.137094:0:154419:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825955.137127:0:154419:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825955.137152:0:154419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825955.137170:0:154419:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825955.137178:0:154419:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825955.137188:0:154419:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825955.137192:0:154419:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825955.137202:0:154419:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb975019c3cb8 00010000:00010000:0.0:1787825955.137206:0:154419:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825955.137216:0:154419:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825955.137424:0:154419:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825955.137434:0:154419:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825955.137442:0:154419:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825955.137456:0:154419:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825955.137463:0:154419:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825955.137467:0:154419:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825955.137470:0:154419:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825955.137474:0:154419:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb975019c3bf8 00010000:00010000:0.0:1787825955.137477:0:154419:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825955.137484:0:154419:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825955.137504:0:154419:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825955.137512:0:154419:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=267 00000080:00200000:0.0:1787825955.137521:0:154419:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 44 00000080:00200008:0.0:1787825955.137529:0:154419:0:(file.c:1090:ll_file_open()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00000080:00200080:0.0:1787825955.137558:0:154419:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) cmd=c03066f0 arg=7ffe354633b0 00000080:00200000:0.0:1787825955.137576:0:154419:0:(dir.c:451:ll_dir_setdirstripe()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) name=test stripe_offset=0 stripe_count=2, hash_type=0 00010000:00010000:0.0:1787825955.139827:0:154419:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7400/0xbef42c0221d6e404 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ce688a expref: -99 pid: 154419 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825955.139869:0:154419:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6800/0xbef42c0221d64b3f lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cb9ffd expref: -99 pid: 152202 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825955.139937:0:154419:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7400/0xbef42c0221d6e404 lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce688a expref: -99 pid: 154419 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825955.139947:0:154419:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787825955.139950:0:154419:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7400/0xbef42c0221d6e404 lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce688a expref: -99 pid: 154419 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:0.0:1787825955.139977:0:154419:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787825955.139993:0:154419:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c55a4000 x1874663042826112/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 552/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787825955.140024:0:154419:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4000 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154419:1874663042826112:192.168.203.147@tcp:36:lfs.0 00000100:00100000:0.0:1787825955.140144:0:154419:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c4aec900 going to sleep for 11 seconds 00000100:00100000:0.0:1787825955.198946:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825955.198989:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048709888 00000100:00100000:0.0:1787825955.199001:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048709888 00000100:00100000:0.0:1787825955.199058:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9117 00000100:00100000:0.0:1787825955.199069:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a4e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:41299:x1874663048709888:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825955.199091:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7c00/0xbef42c0221d6e412 lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x3a5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce68de expref: -99 pid: 154428 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825955.199217:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825955.199224:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a4e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:41299:x1874663048709888:12345-192.168.203.147@tcp:104: Request processed in 157us (294us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825955.199231:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9117 00000100:00100000:1.0:1787825955.216713:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787825955.216774:0:15150:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:1.0:1787825955.216792:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048710144 00000100:00100000:1.0:1787825955.216801:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048710144 00000100:00100000:1.0:1787825955.217213:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9118 00000100:00100000:1.0:1787825955.217224:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c4408c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8414:x1874663048710144:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787825955.217243:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f7800/0xbef42c0221d6e419 lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x3a5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce68ec expref: -99 pid: 154428 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825955.217312:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787825955.217319:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c4408c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8414:x1874663048710144:12345-192.168.203.147@tcp:104: Request processed in 98us (617us total) trans 0 rc 0/0 00000100:00100000:1.0:1787825955.217328:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9118 00000080:00200000:0.0:1787825955.303326:0:154434:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825955.303358:0:154434:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825955.303377:0:154434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825955.303398:0:154434:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825955.303406:0:154434:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825955.303413:0:154434:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825955.303417:0:154434:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825955.303425:0:154434:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501c43c20 00010000:00010000:0.0:1787825955.303429:0:154434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825955.303437:0:154434:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825955.303452:0:154434:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825955.303460:0:154434:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=24 00000080:00200000:0.0:1787825955.303466:0:154434:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825955.303480:0:154434:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=0 00000080:00200000:0.0:1787825955.303494:0:154434:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787825955.303507:0:154434:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787825955.303551:0:154434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6e435 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154434 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.303559:0:154434:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6e435 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154434 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.303569:0:154434:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6e435 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154434 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825955.303595:0:154434:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154434:1874663042827776:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787825955.303671:0:154434:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c4aec780 going to sleep for 11 seconds 00000100:00100000:0.0:1787825955.309733:0:154434:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154434:1874663042827776:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:0.0:1787825955.309777:0:154434:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6e435 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6994 expref: -99 pid: 154434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825955.309932:0:154434:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6e435 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1b85:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6994 expref: -99 pid: 154434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825955.309949:0:154434:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6e435 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1b85:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6994 expref: -99 pid: 154434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825955.309963:0:154434:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6e435 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1b85:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6994 expref: -99 pid: 154434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825955.309976:0:154434:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6e435 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1b85:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6994 expref: -99 pid: 154434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825955.310002:0:154434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6e435 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1b85:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6994 expref: -99 pid: 154434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825955.310011:0:154434:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6e435 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1b85:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6994 expref: -99 pid: 154434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825955.310018:0:154434:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6e435 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1b85:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6994 expref: -99 pid: 154434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787825955.310034:0:154434:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c55a4700 x1874663042827776/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/672 e 0 to 0 dl 1787825971 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787825955.310050:0:154434:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6e435 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1b85:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6994 expref: -99 pid: 154434 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825955.310062:0:154434:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/7045 (0 0) 00000080:00200000:1.0:1787825955.310069:0:154434:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00000002:00010000:1.0:1787825955.310158:0:154434:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200025d90:0x3a6:0x0]) in obj [0x200025d90:0x3a6:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787825955.310223:0:154434:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6e43c lrc: 3/1,0 mode: --/CR res: [0x200025d90:0x3a6:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 154434 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825955.310231:0:154434:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6e43c lrc: 3/1,0 mode: --/CR res: [0x200025d90:0x3a6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154434 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825955.310240:0:154434:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6e43c lrc: 3/1,0 mode: --/CR res: [0x200025d90:0x3a6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154434 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787825955.310268:0:154434:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154434:1874663042827904:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:1.0:1787825955.310377:0:154434:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b59600 going to sleep for 11 seconds 00000100:00100000:1.0:1787825955.328640:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825955.328987:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048711552 00000100:00100000:0.0:1787825955.329000:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048711552 00000100:00100000:0.0:1787825955.329073:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9119 00000100:00100000:0.0:1787825955.329084:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b1c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6490:x1874663048711552:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825955.329118:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6e43c lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x3a6:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce69a2 expref: -99 pid: 154434 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825955.329265:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825955.329273:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b1c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6490:x1874663048711552:12345-192.168.203.147@tcp:104: Request processed in 191us (650us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825955.329282:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9119 00000100:00100000:1.0:1787825955.340353:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787825955.340392:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048711808 00000100:00100000:1.0:1787825955.340400:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048711808 00000100:00100000:1.0:1787825955.340459:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9120 00000100:00100000:1.0:1787825955.340470:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b3b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663048711808:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787825955.340501:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6e443 lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x3a6:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce69b0 expref: -99 pid: 154434 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825955.340597:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787825955.340605:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b3b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663048711808:12345-192.168.203.147@tcp:104: Request processed in 136us (253us total) trans 0 rc 0/0 00000100:00100000:1.0:1787825955.340613:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9120 00000080:00200000:1.0:1787825955.431653:0:154436:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787825955.431696:0:154436:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787825955.431721:0:154436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825955.431741:0:154436:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825955.431967:0:154436:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787825955.431977:0:154436:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787825955.431980:0:154436:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787825955.431991:0:154436:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501603c20 00010000:00010000:1.0:1787825955.431995:0:154436:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825955.432005:0:154436:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825955.432025:0:154436:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787825955.432035:0:154436:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=24 00000080:00200000:1.0:1787825955.432042:0:154436:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:1.0:1787825955.432056:0:154436:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=0 00000080:00200000:1.0:1787825955.432060:0:154436:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:1.0:1787825955.432076:0:154436:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787825955.432280:0:154436:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5400/0xbef42c0221d6e45f lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154436 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825955.432289:0:154436:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5400/0xbef42c0221d6e45f lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154436 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825955.432301:0:154436:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5400/0xbef42c0221d6e45f lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154436 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787825955.432337:0:154436:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154436:1874663042829312:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:1.0:1787825955.432452:0:154436:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b59f00 going to sleep for 11 seconds 00000100:00100000:0.0:1787825955.473368:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825955.473419:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048713216 00000100:00100000:0.0:1787825955.473433:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048713216 00000100:00100000:0.0:1787825955.473512:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9121 00000100:00100000:0.0:1787825955.473525:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6491:x1874663048713216:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825955.473549:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6e466 lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x3a7:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce6a66 expref: -99 pid: 154436 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825955.473648:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825955.473655:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6491:x1874663048713216:12345-192.168.203.147@tcp:104: Request processed in 133us (303us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825955.473663:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9121 00000100:00100000:0.0:1787825955.484377:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825955.484416:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048713472 00000100:00100000:0.0:1787825955.484425:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048713472 00000100:00100000:0.0:1787825955.484473:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9122 00000100:00100000:0.0:1787825955.484492:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663048713472:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825955.484509:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480600/0xbef42c0221d6e46d lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x3a7:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce6a74 expref: -99 pid: 154436 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825955.484637:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825955.484645:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663048713472:12345-192.168.203.147@tcp:104: Request processed in 163us (278us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825955.484653:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9122 00010000:00010000:0.0:1787825955.551186:0:154437:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787825955.551282:0:154437:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825955.551299:0:154437:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825955.551321:0:154437:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825955.551336:0:154437:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825955.551344:0:154437:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825955.551351:0:154437:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825955.551355:0:154437:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825955.551361:0:154437:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501c53be8 00010000:00010000:0.0:1787825955.551365:0:154437:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825955.551373:0:154437:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825955.551387:0:154437:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825955.551394:0:154437:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=16408 00000080:00200000:0.0:1787825955.551400:0:154437:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825955.551413:0:154437:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=16392 00000080:00200000:0.0:1787825955.551417:0:154437:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787825955.551429:0:154437:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787825955.551474:0:154437:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6e47b lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154437 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.551481:0:154437:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6e47b lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154437 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.551504:0:154437:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6e47b lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154437 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825955.551528:0:154437:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154437:1874663042830336:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787825955.551637:0:154437:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c4aec780 going to sleep for 11 seconds 00000100:00100000:0.0:1787825955.558142:0:154437:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154437:1874663042830336:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787825955.559116:0:154437:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6e47b lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154437 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.559140:0:154437:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6e47b lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154437 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.559149:0:154437:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6e47b lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154437 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.559158:0:154437:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6e47b lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154437 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.559282:0:154437:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6e47b lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154437 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.559290:0:154437:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0513400 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787825955.559295:0:154437:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6e47b lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154437 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.559303:0:154437:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6e47b lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154437 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.559312:0:154437:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6e47b lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 154437 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.559321:0:154437:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787825955.561012:0:154437:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6e47b lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154437 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.561042:0:154437:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6e47b lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154437 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825955.561087:0:154437:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c55a4380 x1874663042830336/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787825971 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787825955.561107:0:154437:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00000002:00010000:0.0:1787825955.658635:0:154438:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x24000178a:0x3a8:0x0]) in obj [0x24000178a:0x3a8:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825955.658776:0:154438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6e497 lrc: 3/1,0 mode: --/CR res: [0x24000178a:0x3a8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 154438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.658790:0:154438:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6e497 lrc: 3/1,0 mode: --/CR res: [0x24000178a:0x3a8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154438 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.658801:0:154438:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6e497 lrc: 3/1,0 mode: --/CR res: [0x24000178a:0x3a8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154438 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825955.658832:0:154438:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154438:1874663042831104:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787825955.660723:0:154438:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c4aec780 going to sleep for 11 seconds 00000100:00100000:0.0:1787825955.679140:0:154438:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154438:1874663042831104:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:0.0:1787825955.679176:0:154438:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6e497 lrc: 4/1,0 mode: --/CR res: [0x24000178a:0x3a8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6b38 expref: -99 pid: 154438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.679191:0:154438:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6e497 lrc: 4/1,0 mode: --/PR res: [0x24000178a:0x3a8:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6b38 expref: -99 pid: 154438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.679206:0:154438:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6e497 lrc: 4/1,0 mode: PR/PR res: [0x24000178a:0x3a8:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6b38 expref: -99 pid: 154438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.679220:0:154438:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6e497 lrc: 4/1,0 mode: PR/PR res: [0x24000178a:0x3a8:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6b38 expref: -99 pid: 154438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.679236:0:154438:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6e497 lrc: 5/2,0 mode: PR/PR res: [0x24000178a:0x3a8:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6b38 expref: -99 pid: 154438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.679247:0:154438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6e497 lrc: 5/2,0 mode: PR/PR res: [0x24000178a:0x3a8:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6b38 expref: -99 pid: 154438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.679256:0:154438:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6e497 lrc: 4/1,0 mode: PR/PR res: [0x24000178a:0x3a8:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6b38 expref: -99 pid: 154438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825955.679393:0:154438:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c55a5500 x1874663042831104/t0(0) o101->lustre-MDT0001-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787825971 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787825955.679414:0:154438:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6e497 lrc: 3/1,0 mode: PR/PR res: [0x24000178a:0x3a8:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6b38 expref: -99 pid: 154438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.679431:0:154438:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d0f54800 type 13 mode 4 res 9663682442/936 (0 0) 00010000:00010000:0.0:1787825955.679441:0:154438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6e497 lrc: 3/1,0 mode: PR/PR res: [0x24000178a:0x3a8:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6b38 expref: -99 pid: 154438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.679451:0:154438:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6e497 lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x3a8:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6b38 expref: -99 pid: 154438 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787825955.679504:0:154438:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1b87:0x0], updating i_size 8192 00000080:00200000:0.0:1787825955.679509:0:154438:0:(namei.c:147:ll_iget()) got inode: [0x20001a9e3:0x1b87:0x0](ffff9181c83b00c8): rc = 0 00000080:00010000:0.0:1787825955.679522:0:154438:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x1b87:0x0](ffff9181c83b00c8) for lock 0xbef42c0221d6e489 00000080:00010000:0.0:1787825955.679546:0:154438:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x1b87:0x0](ffff9181c83b00c8) 00000080:00010000:0.0:1787825955.679550:0:154438:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6e489 from it ffffb97501c63e08 00010000:00010000:0.0:1787825955.679555:0:154438:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480600/0xbef42c0221d6e489 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1b87:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6b1c expref: -99 pid: 154438 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.679570:0:154438:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480600/0xbef42c0221d6e489 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1b87:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6b1c expref: -99 pid: 154438 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787825955.679596:0:154438:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 3 00000080:00200000:0.0:1787825955.679607:0:154438:0:(namei.c:2328:ll_rmdir()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00010000:00010000:0.0:1787825955.679646:0:154438:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480600/0xbef42c0221d6e489 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1b87:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ce6b1c expref: -99 pid: 154438 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825955.679703:0:154438:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480600/0xbef42c0221d6e489 lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x1b87:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce6b1c expref: -99 pid: 154438 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825955.679711:0:154438:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787825955.679713:0:154438:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480600/0xbef42c0221d6e489 lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x1b87:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce6b1c expref: -99 pid: 154438 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:0.0:1787825955.679868:0:154438:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787825955.679878:0:154438:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c55a5500 x1874663042831360/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787825955.679915:0:154438:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154438:1874663042831360:192.168.203.147@tcp:36:rmdir.0 00000100:00100000:0.0:1787825955.680664:0:154438:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c4aec780 going to sleep for 11 seconds 00000100:00100000:0.0:1787825955.687211:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825955.687279:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048714880 00000100:00100000:0.0:1787825955.687294:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048714880 00000100:00100000:0.0:1787825955.687364:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9123 00000100:00100000:0.0:1787825955.687376:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a6680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:41299:x1874663048714880:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825955.687403:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6e490 lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x3a8:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce6b2a expref: -99 pid: 154438 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825955.687514:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825955.687521:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a6680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:41299:x1874663048714880:12345-192.168.203.147@tcp:104: Request processed in 148us (1259us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825955.687529:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9123 00010000:00010000:1.0:1787825955.688229:0:33329:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787825955.720931:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825955.721069:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048715136 00000100:00100000:0.0:1787825955.721086:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048715136 00000100:00100000:0.0:1787825955.721173:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9124 00000100:00100000:0.0:1787825955.721187:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8414:x1874663048715136:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825955.721212:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6e497 lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x3a8:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce6b38 expref: -99 pid: 154438 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825955.721316:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825955.721325:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8414:x1874663048715136:12345-192.168.203.147@tcp:104: Request processed in 141us (411us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825955.721334:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9124 00000100:00100000:0.0:1787825955.748031:0:154438:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154438:1874663042831360:192.168.203.147@tcp:36:rmdir.0 00000080:00200000:0.0:1787825955.748094:0:154438:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1b87:0x0](ffff9181c83b00c8) 00000080:00200000:0.0:1787825955.775149:0:154439:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825955.779496:0:154439:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825955.779524:0:154439:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825955.779545:0:154439:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825955.779555:0:154439:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825955.779569:0:154439:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825955.779574:0:154439:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825955.779583:0:154439:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501603b58 00010000:00010000:0.0:1787825955.779659:0:154439:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825955.779670:0:154439:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825955.779693:0:154439:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825955.779706:0:154439:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=29 00000080:00200000:0.0:1787825955.779716:0:154439:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825955.779736:0:154439:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=13 00000080:00200000:0.0:1787825955.779760:0:154439:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787825955.779778:0:154439:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787825955.779840:0:154439:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e49e lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154439 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.779850:0:154439:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e49e lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154439 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.779861:0:154439:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e49e lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154439 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825955.779897:0:154439:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6300 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154439:1874663042831744:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787825955.780015:0:154439:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c4aec780 going to sleep for 11 seconds 00000100:00100000:0.0:1787825955.790331:0:154439:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6300 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154439:1874663042831744:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787825955.790363:0:154439:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e49e lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154439 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.790380:0:154439:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e49e lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154439 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.790390:0:154439:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e49e lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154439 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.790398:0:154439:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e49e lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154439 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.790408:0:154439:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e49e lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154439 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.790415:0:154439:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0512600 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787825955.790420:0:154439:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e49e lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154439 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.790428:0:154439:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e49e lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154439 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.790435:0:154439:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e49e lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 154439 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.790443:0:154439:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787825955.790451:0:154439:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e49e lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154439 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.790459:0:154439:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e49e lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154439 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825955.790480:0:154439:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c55a6300 x1874663042831744/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787825971 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787825955.790511:0:154439:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787825955.790551:0:154439:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787825955.790657:0:154439:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825955.790676:0:154439:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825955.790687:0:154439:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825955.790701:0:154439:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825955.790710:0:154439:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825955.790716:0:154439:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825955.790720:0:154439:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825955.790726:0:154439:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501603be8 00010000:00010000:0.0:1787825955.790730:0:154439:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825955.790740:0:154439:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825955.790821:0:154439:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825955.790831:0:154439:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=16408 00000080:00200000:0.0:1787825955.790838:0:154439:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825955.790854:0:154439:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=16392 00000080:00200000:0.0:1787825955.790859:0:154439:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787825955.790875:0:154439:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787825955.790925:0:154439:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e4a5 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154439 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.790934:0:154439:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e4a5 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154439 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.790944:0:154439:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e4a5 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154439 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825955.790970:0:154439:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6300 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154439:1874663042831872:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787825955.791063:0:154439:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c4aec780 going to sleep for 11 seconds 00000100:00100000:0.0:1787825955.799847:0:154439:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6300 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154439:1874663042831872:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787825955.799883:0:154439:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e4a5 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154439 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.799902:0:154439:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e4a5 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154439 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.799911:0:154439:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e4a5 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154439 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.799920:0:154439:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e4a5 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154439 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.799930:0:154439:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e4a5 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154439 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.799937:0:154439:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0512c00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787825955.799943:0:154439:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e4a5 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154439 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.799952:0:154439:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e4a5 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154439 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.799960:0:154439:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e4a5 lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 154439 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.799969:0:154439:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787825955.799977:0:154439:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e4a5 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154439 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.799984:0:154439:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e4a5 lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154439 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825955.800007:0:154439:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c55a6300 x1874663042831872/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787825971 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787825955.800025:0:154439:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787825955.800064:0:154439:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787825955.800184:0:154439:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825955.800204:0:154439:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825955.800216:0:154439:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825955.800230:0:154439:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825955.800237:0:154439:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825955.800244:0:154439:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825955.800250:0:154439:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825955.800257:0:154439:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501603b58 00010000:00010000:0.0:1787825955.800261:0:154439:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825955.800270:0:154439:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825955.800288:0:154439:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825955.800298:0:154439:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=13 00000080:00200008:0.0:1787825955.800311:0:154439:0:(file.c:6296:ll_getattr_dentry()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1787825955.800317:0:154439:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08),name=d71a.replay-single 00000002:00010000:0.0:1787825955.800331:0:154439:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x20001a9e3:0x17e0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825955.800337:0:154439:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00010000:00010000:0.0:1787825955.800388:0:154439:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e4ac lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 154439 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.800397:0:154439:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e4ac lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 154439 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.800408:0:154439:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e4ac lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 154439 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825955.800436:0:154439:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6300 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154439:1874663042832000:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787825955.800558:0:154439:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c4aec780 going to sleep for 11 seconds 00000100:00100000:0.0:1787825955.804138:0:154439:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6300 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154439:1874663042832000:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787825955.804186:0:154439:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e4ac lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6b9a expref: -99 pid: 154439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.804212:0:154439:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e4ac lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6b9a expref: -99 pid: 154439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.804221:0:154439:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e4ac lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6b9a expref: -99 pid: 154439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.804235:0:154439:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e4ac lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6b9a expref: -99 pid: 154439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.804247:0:154439:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e4ac lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6b9a expref: -99 pid: 154439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.804263:0:154439:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e4ac lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6b9a expref: -99 pid: 154439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.804272:0:154439:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e4ac lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6b9a expref: -99 pid: 154439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.804280:0:154439:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e4ac lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6b9a expref: -99 pid: 154439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825955.804294:0:154439:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c55a6300 x1874663042832000/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787825971 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787825955.804312:0:154439:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e4ac lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6b9a expref: -99 pid: 154439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.804331:0:154439:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787825955.804340:0:154439:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1787825955.804377:0:154439:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x17e0:0x0], updating i_size 4096 00000080:00010000:0.0:1787825955.804387:0:154439:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) for lock 0xbef42c0221d6e4ac 00000080:00010000:0.0:1787825955.804397:0:154439:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00000080:00010000:0.0:1787825955.804401:0:154439:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6e4ac from it ffffb97501603cb8 00010000:00010000:0.0:1787825955.804405:0:154439:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e4ac lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6b9a expref: -99 pid: 154439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.804414:0:154439:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e4ac lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6b9a expref: -99 pid: 154439 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:0.0:1787825955.805206:0:154439:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1787825955.805216:0:154439:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825955.805239:0:154439:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825955.805253:0:154439:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825955.805266:0:154439:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825955.805274:0:154439:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825955.805279:0:154439:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825955.805282:0:154439:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825955.805288:0:154439:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501603cb8 00010000:00010000:0.0:1787825955.805291:0:154439:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825955.805300:0:154439:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825955.805455:0:154439:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825955.805464:0:154439:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825955.805472:0:154439:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825955.805495:0:154439:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825955.805502:0:154439:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825955.805506:0:154439:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825955.805508:0:154439:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825955.805512:0:154439:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501603bf8 00010000:00010000:0.0:1787825955.805515:0:154439:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825955.805522:0:154439:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825955.805531:0:154439:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825955.805538:0:154439:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=267 00000080:00200000:0.0:1787825955.805544:0:154439:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 44 00000080:00200008:0.0:1787825955.805550:0:154439:0:(file.c:1090:ll_file_open()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00000080:00200080:0.0:1787825955.805576:0:154439:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) cmd=c03066f0 arg=7fff95bd7680 00000080:00200000:0.0:1787825955.805591:0:154439:0:(dir.c:451:ll_dir_setdirstripe()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) name=test stripe_offset=0 stripe_count=2, hash_type=0 00010000:00010000:0.0:1787825955.805628:0:154439:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e4ac lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ce6b9a expref: -99 pid: 154439 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825955.805646:0:154439:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6800/0xbef42c0221d64b3f lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cb9ffd expref: -99 pid: 152202 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825955.805699:0:154439:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e4ac lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce6b9a expref: -99 pid: 154439 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825955.805706:0:154439:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787825955.805709:0:154439:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e4ac lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce6b9a expref: -99 pid: 154439 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:0.0:1787825955.805729:0:154439:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787825955.805740:0:154439:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c55a6300 x1874663042832256/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 552/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787825955.805830:0:154439:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6300 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154439:1874663042832256:192.168.203.147@tcp:36:lfs.0 00000100:00100000:0.0:1787825955.805929:0:154439:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c4aec780 going to sleep for 11 seconds 00000100:00100000:0.0:1787825955.843785:0:154439:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6300 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154439:1874663042832256:192.168.203.147@tcp:36:lfs.0 00000080:00200000:0.0:1787825955.843855:0:154439:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x20001a9e3:0x1b88:0x0](ffff9181c83a21c8) 00000080:00200000:0.0:1787825955.843864:0:154439:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1b88:0x0], updating i_size 4096 00000080:00200008:0.0:1787825955.843906:0:154439:0:(file.c:479:ll_file_release()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00010000:00010000:0.0:1787825955.843944:0:154439:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513000/0xbef42c0221d64bee lrc: 2/0,0 mode: CR/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cba2b9 expref: -99 pid: 152210 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825955.852902:0:154440:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825955.852944:0:154440:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825955.852969:0:154440:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825955.852988:0:154440:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825955.852998:0:154440:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825955.853007:0:154440:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825955.853011:0:154440:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825955.853020:0:154440:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501c53c20 00010000:00010000:0.0:1787825955.853025:0:154440:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825955.853035:0:154440:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825955.853053:0:154440:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825955.853063:0:154440:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=24 00000080:00200000:0.0:1787825955.853069:0:154440:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825955.853083:0:154440:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=0 00000080:00200000:0.0:1787825955.853089:0:154440:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787825955.853104:0:154440:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787825955.853164:0:154440:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e4b3 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154440 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.853172:0:154440:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e4b3 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154440 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825955.853187:0:154440:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e4b3 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154440 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825955.853218:0:154440:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154440:1874663042832384:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787825955.853320:0:154440:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c4aec780 going to sleep for 11 seconds 00000100:00100000:0.0:1787825955.862819:0:154440:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154440:1874663042832384:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:0.0:1787825955.862866:0:154440:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e4b3 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6be0 expref: -99 pid: 154440 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787825955.909288:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825955.909414:0:33333:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:0.0:1787825955.909439:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048716544 00000100:00100000:0.0:1787825955.909450:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048716544 00000100:00100000:0.0:1787825955.909546:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9125 00000100:00100000:0.0:1787825955.909558:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c4409050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6490:x1874663048716544:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825955.909582:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6e4ba lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x3a9:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce6bee expref: -99 pid: 154440 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825955.909684:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825955.909691:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c4409050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6490:x1874663048716544:12345-192.168.203.147@tcp:104: Request processed in 137us (422us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825955.909698:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9125 00000100:00100000:0.0:1787825955.918088:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787825955.918199:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048716800 00000100:00100000:1.0:1787825955.918210:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048716800 00000100:00100000:1.0:1787825955.918284:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9126 00000100:00100000:1.0:1787825955.918296:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663048716800:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787825955.918319:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6600/0xbef42c0221d6e4c1 lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x3a9:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce6bfc expref: -99 pid: 154440 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825955.919706:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787825955.919729:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663048716800:12345-192.168.203.147@tcp:104: Request processed in 1428us (1649us total) trans 0 rc 0/0 00000100:00100000:1.0:1787825955.919755:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9126 00000080:00200000:1.0:1787825955.963409:0:154441:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787825955.963447:0:154441:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787825955.963472:0:154441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825955.963504:0:154441:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825955.963514:0:154441:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787825955.963523:0:154441:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787825955.963527:0:154441:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787825955.963658:0:154441:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501c63b58 00010000:00010000:1.0:1787825955.963665:0:154441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825955.963675:0:154441:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825955.964024:0:154441:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787825955.964031:0:154441:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=29 00000080:00200000:1.0:1787825955.964038:0:154441:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:1.0:1787825955.964049:0:154441:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=13 00000080:00200000:1.0:1787825955.964053:0:154441:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:1.0:1787825955.964067:0:154441:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787825955.964115:0:154441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4e00/0xbef42c0221d6e4c8 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154441 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825955.964121:0:154441:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4e00/0xbef42c0221d6e4c8 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154441 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825955.964130:0:154441:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4e00/0xbef42c0221d6e4c8 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154441 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787825955.964155:0:154441:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154441:1874663042833280:192.168.203.147@tcp:101:lfs.0 00000100:00100000:1.0:1787825955.964283:0:154441:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b593c0 going to sleep for 11 seconds 00000080:00200000:0.0:1787825956.028447:0:154450:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825956.028884:0:154450:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825956.028910:0:154450:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825956.028930:0:154450:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825956.028939:0:154450:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825956.028948:0:154450:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825956.028953:0:154450:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825956.028962:0:154450:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501c73c20 00010000:00010000:0.0:1787825956.028966:0:154450:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825956.028977:0:154450:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825956.028997:0:154450:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825956.029006:0:154450:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=24 00000080:00200000:0.0:1787825956.029014:0:154450:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825956.029028:0:154450:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=0 00000080:00200000:0.0:1787825956.029032:0:154450:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787825956.029048:0:154450:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787825956.029108:0:154450:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6e4dd lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154450 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825956.029117:0:154450:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6e4dd lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154450 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825956.029129:0:154450:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6e4dd lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154450 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825956.029165:0:154450:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6a00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154450:1874663042833920:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787825956.029326:0:154450:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c4aec240 going to sleep for 11 seconds 00000100:00100000:0.0:1787825956.071555:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787825956.072876:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048718336 00000100:00100000:1.0:1787825956.072904:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048718336 00000100:00100000:1.0:1787825956.073000:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9127 00000100:00100000:1.0:1787825956.073013:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a6680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6492:x1874663048718336:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787825956.073039:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481000/0xbef42c0221d6e4e4 lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x3aa:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce6cb2 expref: -99 pid: 154450 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825956.073161:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787825956.073169:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a6680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6492:x1874663048718336:12345-192.168.203.147@tcp:104: Request processed in 159us (1630us total) trans 0 rc 0/0 00000100:00100000:1.0:1787825956.073177:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9127 00000100:00100000:1.0:1787825956.086547:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825956.086666:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048718592 00000100:00100000:0.0:1787825956.086679:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048718592 00000100:00100000:0.0:1787825956.086806:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9128 00000100:00100000:0.0:1787825956.086818:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b3100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663048718592:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825956.086840:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6e4eb lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x3aa:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce6cc0 expref: -99 pid: 154450 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825956.087048:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825956.087058:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b3100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663048718592:12345-192.168.203.147@tcp:104: Request processed in 241us (524us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825956.087066:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9128 00010000:00010000:0.0:1787825956.089980:0:15160:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6e4eb lrc: 2/0,0 mode: --/PR res: [0x24000178a:0x3aa:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ce6cc0 expref: -99 pid: 154450 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825956.089989:0:15160:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787825956.090027:0:15160:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6e4eb lrc: 1/0,0 mode: --/PR res: [0x24000178a:0x3aa:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ce6cc0 expref: -99 pid: 154450 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825956.090036:0:15160:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6e4eb lrc: 0/0,0 mode: --/PR res: [0x24000178a:0x3aa:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ce6cc0 expref: -99 pid: 154450 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825956.090061:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787825956.090063:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787825956.090066:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787825956.090075:0:2390:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1787825956.090100:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042834688:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1787825956.105452:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042834688:192.168.203.147@tcp:103:ldlm_bl.0 00000080:00200000:1.0:1787825956.247308:0:154452:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787825956.247354:0:154452:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787825956.247379:0:154452:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825956.247399:0:154452:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825956.247408:0:154452:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787825956.247420:0:154452:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787825956.247424:0:154452:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787825956.247434:0:154452:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501c83c20 00010000:00010000:1.0:1787825956.247439:0:154452:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825956.247450:0:154452:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825956.247469:0:154452:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787825956.247492:0:154452:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=24 00000080:00200000:1.0:1787825956.247502:0:154452:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:1.0:1787825956.247522:0:154452:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=0 00000080:00200000:1.0:1787825956.247527:0:154452:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:1.0:1787825956.247600:0:154452:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787825956.247666:0:154452:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6e507 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154452 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.247674:0:154452:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6e507 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154452 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.247686:0:154452:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6e507 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154452 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787825956.247721:0:154452:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154452:1874663042835456:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:1.0:1787825956.247972:0:154452:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b59180 going to sleep for 11 seconds 00000100:00100000:1.0:1787825956.252063:0:154452:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154452:1874663042835456:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:1.0:1787825956.252101:0:154452:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6e507 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6d68 expref: -99 pid: 154452 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.252406:0:154452:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6e507 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1b8a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6d68 expref: -99 pid: 154452 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.252416:0:154452:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6e507 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1b8a:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6d68 expref: -99 pid: 154452 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.252430:0:154452:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6e507 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1b8a:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6d68 expref: -99 pid: 154452 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.252445:0:154452:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6e507 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1b8a:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6d68 expref: -99 pid: 154452 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.252462:0:154452:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6e507 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1b8a:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6d68 expref: -99 pid: 154452 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.252472:0:154452:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6e507 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1b8a:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6d68 expref: -99 pid: 154452 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.252491:0:154452:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6e507 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1b8a:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6d68 expref: -99 pid: 154452 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787825956.252513:0:154452:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c35b2680 x1874663042835456/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/672 e 0 to 0 dl 1787825972 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787825956.252531:0:154452:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6e507 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1b8a:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6d68 expref: -99 pid: 154452 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.252552:0:154452:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/7050 (0 0) 00000080:00200000:1.0:1787825956.252561:0:154452:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00000002:00010000:1.0:1787825956.252645:0:154452:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200025d90:0x3ab:0x0]) in obj [0x200025d90:0x3ab:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787825956.252707:0:154452:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5600/0xbef42c0221d6e50e lrc: 3/1,0 mode: --/CR res: [0x200025d90:0x3ab:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 154452 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.252716:0:154452:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5600/0xbef42c0221d6e50e lrc: 3/1,0 mode: --/CR res: [0x200025d90:0x3ab:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154452 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.252725:0:154452:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5600/0xbef42c0221d6e50e lrc: 3/1,0 mode: --/CR res: [0x200025d90:0x3ab:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154452 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787825956.253014:0:154452:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154452:1874663042835584:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:1.0:1787825956.253288:0:154452:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b59180 going to sleep for 11 seconds 00000100:00100000:1.0:1787825956.266888:0:154452:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154452:1874663042835584:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:1.0:1787825956.266925:0:154452:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5600/0xbef42c0221d6e50e lrc: 4/1,0 mode: --/CR res: [0x200025d90:0x3ab:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6d76 expref: -99 pid: 154452 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.266941:0:154452:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5600/0xbef42c0221d6e50e lrc: 4/1,0 mode: --/PR res: [0x200025d90:0x3ab:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6d76 expref: -99 pid: 154452 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.266960:0:154452:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5600/0xbef42c0221d6e50e lrc: 4/1,0 mode: PR/PR res: [0x200025d90:0x3ab:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6d76 expref: -99 pid: 154452 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.266975:0:154452:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5600/0xbef42c0221d6e50e lrc: 4/1,0 mode: PR/PR res: [0x200025d90:0x3ab:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6d76 expref: -99 pid: 154452 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.266990:0:154452:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5600/0xbef42c0221d6e50e lrc: 5/2,0 mode: PR/PR res: [0x200025d90:0x3ab:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6d76 expref: -99 pid: 154452 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.266999:0:154452:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5600/0xbef42c0221d6e50e lrc: 5/2,0 mode: PR/PR res: [0x200025d90:0x3ab:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6d76 expref: -99 pid: 154452 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.267006:0:154452:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5600/0xbef42c0221d6e50e lrc: 4/1,0 mode: PR/PR res: [0x200025d90:0x3ab:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6d76 expref: -99 pid: 154452 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787825956.267020:0:154452:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c35b1f80 x1874663042835584/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787825972 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787825956.267038:0:154452:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5600/0xbef42c0221d6e50e lrc: 3/1,0 mode: PR/PR res: [0x200025d90:0x3ab:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6d76 expref: -99 pid: 154452 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.267056:0:154452:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590089616/939 (0 0) 00010000:00010000:1.0:1787825956.267065:0:154452:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5600/0xbef42c0221d6e50e lrc: 3/1,0 mode: PR/PR res: [0x200025d90:0x3ab:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6d76 expref: -99 pid: 154452 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.267075:0:154452:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5600/0xbef42c0221d6e50e lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x3ab:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6d76 expref: -99 pid: 154452 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000002:00010000:1.0:1787825956.267940:0:154452:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x24000178a:0x3ab:0x0]) in obj [0x24000178a:0x3ab:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787825956.268019:0:154452:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e515 lrc: 3/1,0 mode: --/CR res: [0x24000178a:0x3ab:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 154452 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.268030:0:154452:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e515 lrc: 3/1,0 mode: --/CR res: [0x24000178a:0x3ab:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154452 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.268040:0:154452:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e515 lrc: 3/1,0 mode: --/CR res: [0x24000178a:0x3ab:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154452 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787825956.268068:0:154452:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154452:1874663042835712:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:1.0:1787825956.269243:0:154452:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b59180 going to sleep for 11 seconds 00000100:00100000:1.0:1787825956.280386:0:154452:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154452:1874663042835712:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:1.0:1787825956.280424:0:154452:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e515 lrc: 4/1,0 mode: --/CR res: [0x24000178a:0x3ab:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6d84 expref: -99 pid: 154452 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.280438:0:154452:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e515 lrc: 4/1,0 mode: --/PR res: [0x24000178a:0x3ab:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6d84 expref: -99 pid: 154452 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.280450:0:154452:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e515 lrc: 4/1,0 mode: PR/PR res: [0x24000178a:0x3ab:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6d84 expref: -99 pid: 154452 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.280463:0:154452:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e515 lrc: 4/1,0 mode: PR/PR res: [0x24000178a:0x3ab:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6d84 expref: -99 pid: 154452 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.280488:0:154452:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e515 lrc: 5/2,0 mode: PR/PR res: [0x24000178a:0x3ab:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6d84 expref: -99 pid: 154452 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.280503:0:154452:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e515 lrc: 5/2,0 mode: PR/PR res: [0x24000178a:0x3ab:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6d84 expref: -99 pid: 154452 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.280511:0:154452:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e515 lrc: 4/1,0 mode: PR/PR res: [0x24000178a:0x3ab:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6d84 expref: -99 pid: 154452 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787825956.280525:0:154452:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c35b1f80 x1874663042835712/t0(0) o101->lustre-MDT0001-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787825972 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787825956.281281:0:154452:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e515 lrc: 3/1,0 mode: PR/PR res: [0x24000178a:0x3ab:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6d84 expref: -99 pid: 154452 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.281300:0:154452:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d0f54800 type 13 mode 4 res 9663682442/939 (0 0) 00010000:00010000:1.0:1787825956.281310:0:154452:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e515 lrc: 3/1,0 mode: PR/PR res: [0x24000178a:0x3ab:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6d84 expref: -99 pid: 154452 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.281321:0:154452:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e515 lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x3ab:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6d84 expref: -99 pid: 154452 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787825956.281372:0:154452:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1b8a:0x0], updating i_size 8192 00000080:00200000:1.0:1787825956.281376:0:154452:0:(namei.c:147:ll_iget()) got inode: [0x20001a9e3:0x1b8a:0x0](ffff9181c83b42c8): rc = 0 00000080:00010000:1.0:1787825956.281388:0:154452:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x1b8a:0x0](ffff9181c83b42c8) for lock 0xbef42c0221d6e507 00000080:00010000:1.0:1787825956.281416:0:154452:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x1b8a:0x0](ffff9181c83b42c8) 00000080:00010000:1.0:1787825956.281421:0:154452:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6e507 from it ffffb97501c83e08 00010000:00010000:1.0:1787825956.281425:0:154452:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6e507 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1b8a:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6d68 expref: -99 pid: 154452 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.281439:0:154452:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6e507 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1b8a:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6d68 expref: -99 pid: 154452 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787825956.281464:0:154452:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 3 00000080:00200000:1.0:1787825956.281474:0:154452:0:(namei.c:2328:ll_rmdir()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00010000:00010000:1.0:1787825956.281521:0:154452:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6e507 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1b8a:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ce6d68 expref: -99 pid: 154452 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825956.281960:0:154452:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6e507 lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x1b8a:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce6d68 expref: -99 pid: 154452 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825956.281969:0:154452:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787825956.281972:0:154452:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6e507 lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x1b8a:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce6d68 expref: -99 pid: 154452 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:1.0:1787825956.282001:0:154452:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787825956.282012:0:154452:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c35b2680 x1874663042835968/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787825956.282040:0:154452:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154452:1874663042835968:192.168.203.147@tcp:36:rmdir.0 00000100:00100000:1.0:1787825956.282671:0:154452:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b59180 going to sleep for 11 seconds 00000100:00100000:1.0:1787825956.292306:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825956.292828:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048720000 00000100:00100000:0.0:1787825956.292842:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048720000 00000100:00100000:0.0:1787825956.292910:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9129 00000100:00100000:0.0:1787825956.292921:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:41299:x1874663048720000:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825956.292946:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5600/0xbef42c0221d6e50e lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x3ab:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce6d76 expref: -99 pid: 154452 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825956.293036:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825956.293043:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:41299:x1874663048720000:12345-192.168.203.147@tcp:104: Request processed in 124us (750us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825956.293050:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9129 00000100:00100000:0.0:1787825956.312617:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787825956.314821:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048720256 00000100:00100000:1.0:1787825956.314838:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048720256 00000100:00100000:1.0:1787825956.314913:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9130 00000100:00100000:1.0:1787825956.314925:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8414:x1874663048720256:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787825956.314950:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e515 lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x3ab:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce6d84 expref: -99 pid: 154452 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825956.315059:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787825956.315066:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8414:x1874663048720256:12345-192.168.203.147@tcp:104: Request processed in 144us (2467us total) trans 0 rc 0/0 00000100:00100000:1.0:1787825956.315073:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9130 00000100:00100000:1.0:1787825956.336725:0:154452:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154452:1874663042835968:192.168.203.147@tcp:36:rmdir.0 00000080:00200000:1.0:1787825956.336826:0:154452:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1b8a:0x0](ffff9181c83b42c8) 00000100:00100000:0.0:1787825956.445187:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787825956.445840:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048721664 00000100:00100000:1.0:1787825956.445855:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048721664 00000100:00100000:1.0:1787825956.445927:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9131 00000100:00100000:1.0:1787825956.445939:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a7480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:14309:x1874663048721664:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787825956.445963:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480200/0xbef42c0221d6e538 lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x3ac:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce6e3a expref: -99 pid: 154454 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825956.446065:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787825956.446073:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a7480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:14309:x1874663048721664:12345-192.168.203.147@tcp:104: Request processed in 137us (904us total) trans 0 rc 0/0 00000100:00100000:1.0:1787825956.446082:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9131 00000100:00100000:0.0:1787825956.451914:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787825956.452039:0:33333:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:1.0:1787825956.452064:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048721920 00000100:00100000:1.0:1787825956.452074:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048721920 00000100:00100000:1.0:1787825956.452150:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9132 00000100:00100000:1.0:1787825956.452162:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c442e850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663048721920:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787825956.452184:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481000/0xbef42c0221d6e53f lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x3ac:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce6e48 expref: -99 pid: 154454 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825956.452292:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787825956.452300:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c442e850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663048721920:12345-192.168.203.147@tcp:104: Request processed in 140us (388us total) trans 0 rc 0/0 00000100:00100000:1.0:1787825956.452309:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9132 00000080:00200000:0.0:1787825956.505524:0:154455:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825956.505571:0:154455:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825956.505597:0:154455:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825956.505618:0:154455:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825956.505627:0:154455:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825956.505638:0:154455:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825956.505643:0:154455:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825956.505653:0:154455:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501c8bb58 00010000:00010000:0.0:1787825956.505657:0:154455:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825956.505667:0:154455:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825956.505688:0:154455:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825956.505698:0:154455:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=29 00000080:00200000:0.0:1787825956.505706:0:154455:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825956.505720:0:154455:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=13 00000080:00200000:0.0:1787825956.505724:0:154455:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787825956.505740:0:154455:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787825956.505839:0:154455:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e546 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154455 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825956.505847:0:154455:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e546 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154455 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825956.505859:0:154455:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e546 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154455 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825956.505892:0:154455:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154455:1874663042837888:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787825956.505998:0:154455:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c4aec780 going to sleep for 11 seconds 00000100:00100000:1.0:1787825956.585463:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787825956.585523:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048723328 00000100:00100000:1.0:1787825956.585547:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048723328 00000100:00100000:1.0:1787825956.585614:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9133 00000100:00100000:1.0:1787825956.585625:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b3100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6492:x1874663048723328:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787825956.585649:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6e562 lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x3ad:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce6efe expref: -99 pid: 154456 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825956.585781:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787825956.585789:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b3100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:6492:x1874663048723328:12345-192.168.203.147@tcp:104: Request processed in 166us (344us total) trans 0 rc 0/0 00000100:00100000:1.0:1787825956.585798:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9133 00000100:00100000:1.0:1787825956.589853:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787825956.589946:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048723584 00000100:00100000:0.0:1787825956.589959:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048723584 00000100:00100000:0.0:1787825956.590018:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9134 00000100:00100000:0.0:1787825956.590031:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8414:x1874663048723584:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787825956.590053:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f7600/0xbef42c0221d6e569 lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x3ad:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce6f0c expref: -99 pid: 154456 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787825956.590187:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787825956.590204:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8414:x1874663048723584:12345-192.168.203.147@tcp:104: Request processed in 167us (360us total) trans 0 rc 0/0 00000100:00100000:0.0:1787825956.590212:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9134 00000080:00200000:1.0:1787825956.652474:0:154457:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787825956.652621:0:154457:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787825956.652651:0:154457:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825956.652671:0:154457:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825956.652680:0:154457:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787825956.652691:0:154457:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787825956.652711:0:154457:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787825956.652721:0:154457:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501cabb58 00010000:00010000:1.0:1787825956.652726:0:154457:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825956.652737:0:154457:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825956.652784:0:154457:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787825956.652794:0:154457:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=29 00000080:00200000:1.0:1787825956.652801:0:154457:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:1.0:1787825956.652814:0:154457:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=13 00000080:00200000:1.0:1787825956.652820:0:154457:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:1.0:1787825956.652835:0:154457:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787825956.652897:0:154457:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6e570 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154457 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.652906:0:154457:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6e570 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154457 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.652918:0:154457:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6e570 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154457 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787825956.652952:0:154457:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154457:1874663042839424:192.168.203.147@tcp:101:lfs.0 00000100:00100000:1.0:1787825956.653071:0:154457:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b59c00 going to sleep for 11 seconds 00000100:00100000:1.0:1787825956.658052:0:154457:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154457:1874663042839424:192.168.203.147@tcp:101:lfs.0 00010000:00010000:1.0:1787825956.658085:0:154457:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6e570 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154457 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.658239:0:154457:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6e570 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154457 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.658249:0:154457:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6e570 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154457 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.658257:0:154457:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6e570 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154457 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.658267:0:154457:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6e570 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154457 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.658274:0:154457:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4000 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:1.0:1787825956.658280:0:154457:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6e570 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154457 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.658287:0:154457:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6e570 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154457 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.658296:0:154457:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6e570 lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 154457 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.658305:0:154457:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787825956.658312:0:154457:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6e570 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154457 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.658319:0:154457:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6e570 lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154457 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787825956.658340:0:154457:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c35b0700 x1874663042839424/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787825972 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787825956.658360:0:154457:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:1.0:1787825956.658400:0:154457:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:1.0:1787825956.659445:0:154457:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787825956.659489:0:154457:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787825956.659505:0:154457:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825956.659522:0:154457:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825956.659541:0:154457:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787825956.659551:0:154457:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787825956.659556:0:154457:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787825956.659564:0:154457:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501cabbe8 00010000:00010000:1.0:1787825956.659569:0:154457:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787825956.659580:0:154457:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787825956.659598:0:154457:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787825956.659608:0:154457:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=16408 00000080:00200000:1.0:1787825956.659616:0:154457:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:1.0:1787825956.659634:0:154457:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=16392 00000080:00200000:1.0:1787825956.659639:0:154457:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:1.0:1787825956.659653:0:154457:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787825956.660048:0:154457:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5800/0xbef42c0221d6e577 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154457 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.660060:0:154457:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5800/0xbef42c0221d6e577 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154457 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.660072:0:154457:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5800/0xbef42c0221d6e577 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154457 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787825956.661222:0:154457:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154457:1874663042839552:192.168.203.147@tcp:101:lfs.0 00000100:00100000:1.0:1787825956.661354:0:154457:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b59c00 going to sleep for 11 seconds 00000080:00200000:0.0:1787825956.710500:0:154458:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825956.711673:0:154458:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825956.711706:0:154458:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825956.711728:0:154458:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825956.711738:0:154458:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825956.712336:0:154458:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825956.712344:0:154458:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825956.712355:0:154458:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb975019ebc20 00010000:00010000:0.0:1787825956.712360:0:154458:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825956.712373:0:154458:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825956.712395:0:154458:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825956.712408:0:154458:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=24 00000080:00200000:0.0:1787825956.712417:0:154458:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825956.712437:0:154458:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=0 00000080:00200000:0.0:1787825956.712442:0:154458:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787825956.712461:0:154458:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787825956.712550:0:154458:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6e585 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154458 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825956.712559:0:154458:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6e585 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154458 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825956.712572:0:154458:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6e585 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154458 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825956.712604:0:154458:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154458:1874663042840064:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787825956.712783:0:154458:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c4aeccc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787825956.723570:0:154458:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154458:1874663042840064:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:1.0:1787825956.723609:0:154458:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6e585 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6fb4 expref: -99 pid: 154458 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.723636:0:154458:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6e585 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1b8d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6fb4 expref: -99 pid: 154458 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.723644:0:154458:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6e585 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1b8d:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6fb4 expref: -99 pid: 154458 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.723657:0:154458:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6e585 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1b8d:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6fb4 expref: -99 pid: 154458 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.723669:0:154458:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6e585 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1b8d:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6fb4 expref: -99 pid: 154458 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.723684:0:154458:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6e585 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1b8d:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6fb4 expref: -99 pid: 154458 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.723698:0:154458:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6e585 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1b8d:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6fb4 expref: -99 pid: 154458 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.723706:0:154458:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6e585 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1b8d:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6fb4 expref: -99 pid: 154458 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787825956.723719:0:154458:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c55a4700 x1874663042840064/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/672 e 0 to 0 dl 1787825972 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787825956.723736:0:154458:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6e585 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1b8d:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6fb4 expref: -99 pid: 154458 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.723780:0:154458:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/7053 (0 0) 00000080:00200000:1.0:1787825956.723789:0:154458:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00000002:00010000:1.0:1787825956.723872:0:154458:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200025d90:0x3ae:0x0]) in obj [0x200025d90:0x3ae:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787825956.723935:0:154458:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6e58c lrc: 3/1,0 mode: --/CR res: [0x200025d90:0x3ae:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 154458 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.723943:0:154458:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6e58c lrc: 3/1,0 mode: --/CR res: [0x200025d90:0x3ae:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154458 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.723953:0:154458:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6e58c lrc: 3/1,0 mode: --/CR res: [0x200025d90:0x3ae:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154458 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787825956.723978:0:154458:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1880 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154458:1874663042840192:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:1.0:1787825956.724091:0:154458:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b59480 going to sleep for 11 seconds 00000100:00100000:1.0:1787825956.733419:0:154458:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1880 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154458:1874663042840192:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:1.0:1787825956.733458:0:154458:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6e58c lrc: 4/1,0 mode: --/CR res: [0x200025d90:0x3ae:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6fc2 expref: -99 pid: 154458 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.733485:0:154458:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6e58c lrc: 4/1,0 mode: --/PR res: [0x200025d90:0x3ae:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6fc2 expref: -99 pid: 154458 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.733500:0:154458:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6e58c lrc: 4/1,0 mode: PR/PR res: [0x200025d90:0x3ae:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6fc2 expref: -99 pid: 154458 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.733513:0:154458:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6e58c lrc: 4/1,0 mode: PR/PR res: [0x200025d90:0x3ae:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6fc2 expref: -99 pid: 154458 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.733587:0:154458:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6e58c lrc: 5/2,0 mode: PR/PR res: [0x200025d90:0x3ae:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6fc2 expref: -99 pid: 154458 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.733600:0:154458:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6e58c lrc: 5/2,0 mode: PR/PR res: [0x200025d90:0x3ae:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6fc2 expref: -99 pid: 154458 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.733609:0:154458:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6e58c lrc: 4/1,0 mode: PR/PR res: [0x200025d90:0x3ae:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6fc2 expref: -99 pid: 154458 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787825956.733624:0:154458:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c35b1880 x1874663042840192/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787825972 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787825956.733641:0:154458:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6e58c lrc: 3/1,0 mode: PR/PR res: [0x200025d90:0x3ae:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6fc2 expref: -99 pid: 154458 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.733660:0:154458:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590089616/942 (0 0) 00010000:00010000:1.0:1787825956.733671:0:154458:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6e58c lrc: 3/1,0 mode: PR/PR res: [0x200025d90:0x3ae:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6fc2 expref: -99 pid: 154458 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.733682:0:154458:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6e58c lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x3ae:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6fc2 expref: -99 pid: 154458 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000002:00010000:1.0:1787825956.733717:0:154458:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x24000178a:0x3ae:0x0]) in obj [0x24000178a:0x3ae:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787825956.734260:0:154458:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4e00/0xbef42c0221d6e593 lrc: 3/1,0 mode: --/CR res: [0x24000178a:0x3ae:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 154458 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.734271:0:154458:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4e00/0xbef42c0221d6e593 lrc: 3/1,0 mode: --/CR res: [0x24000178a:0x3ae:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154458 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.734282:0:154458:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4e00/0xbef42c0221d6e593 lrc: 3/1,0 mode: --/CR res: [0x24000178a:0x3ae:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 154458 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787825956.734307:0:154458:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1880 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154458:1874663042840320:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:1.0:1787825956.734450:0:154458:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b59480 going to sleep for 11 seconds 00000100:00100000:1.0:1787825956.767065:0:154458:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1880 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154458:1874663042840320:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:1.0:1787825956.767243:0:154458:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4e00/0xbef42c0221d6e593 lrc: 4/1,0 mode: --/CR res: [0x24000178a:0x3ae:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6fd0 expref: -99 pid: 154458 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.767259:0:154458:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4e00/0xbef42c0221d6e593 lrc: 4/1,0 mode: --/PR res: [0x24000178a:0x3ae:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6fd0 expref: -99 pid: 154458 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.767273:0:154458:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4e00/0xbef42c0221d6e593 lrc: 4/1,0 mode: PR/PR res: [0x24000178a:0x3ae:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6fd0 expref: -99 pid: 154458 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.767288:0:154458:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4e00/0xbef42c0221d6e593 lrc: 4/1,0 mode: PR/PR res: [0x24000178a:0x3ae:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6fd0 expref: -99 pid: 154458 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.767306:0:154458:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4e00/0xbef42c0221d6e593 lrc: 5/2,0 mode: PR/PR res: [0x24000178a:0x3ae:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6fd0 expref: -99 pid: 154458 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.767316:0:154458:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4e00/0xbef42c0221d6e593 lrc: 5/2,0 mode: PR/PR res: [0x24000178a:0x3ae:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6fd0 expref: -99 pid: 154458 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.767324:0:154458:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4e00/0xbef42c0221d6e593 lrc: 4/1,0 mode: PR/PR res: [0x24000178a:0x3ae:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6fd0 expref: -99 pid: 154458 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787825956.767340:0:154458:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c35b1880 x1874663042840320/t0(0) o101->lustre-MDT0001-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787825972 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787825956.767357:0:154458:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4e00/0xbef42c0221d6e593 lrc: 3/1,0 mode: PR/PR res: [0x24000178a:0x3ae:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6fd0 expref: -99 pid: 154458 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.767373:0:154458:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d0f54800 type 13 mode 4 res 9663682442/942 (0 0) 00010000:00010000:1.0:1787825956.767382:0:154458:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4e00/0xbef42c0221d6e593 lrc: 3/1,0 mode: PR/PR res: [0x24000178a:0x3ae:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6fd0 expref: -99 pid: 154458 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.767392:0:154458:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4e00/0xbef42c0221d6e593 lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x3ae:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6fd0 expref: -99 pid: 154458 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787825956.767437:0:154458:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1b8d:0x0], updating i_size 8192 00000080:00200000:1.0:1787825956.767442:0:154458:0:(namei.c:147:ll_iget()) got inode: [0x20001a9e3:0x1b8d:0x0](ffff9181c83b42c8): rc = 0 00000080:00010000:1.0:1787825956.767454:0:154458:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x1b8d:0x0](ffff9181c83b42c8) for lock 0xbef42c0221d6e585 00000080:00010000:1.0:1787825956.767493:0:154458:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x1b8d:0x0](ffff9181c83b42c8) 00000080:00010000:1.0:1787825956.767499:0:154458:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6e585 from it ffffb975019ebe08 00010000:00010000:1.0:1787825956.767503:0:154458:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6e585 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1b8d:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce6fb4 expref: -99 pid: 154458 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787825956.767513:0:154458:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6e585 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1b8d:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce6fb4 expref: -99 pid: 154458 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787825956.768337:0:154458:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 3 00000080:00200000:1.0:1787825956.768354:0:154458:0:(namei.c:2328:ll_rmdir()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00010000:00010000:1.0:1787825956.768401:0:154458:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6e585 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1b8d:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ce6fb4 expref: -99 pid: 154458 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825956.768469:0:154458:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6e585 lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x1b8d:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce6fb4 expref: -99 pid: 154458 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825956.768487:0:154458:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787825956.768491:0:154458:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6e585 lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x1b8d:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce6fb4 expref: -99 pid: 154458 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:1.0:1787825956.768521:0:154458:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787825956.768920:0:154458:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c35b1880 x1874663042840576/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787825956.768952:0:154458:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1880 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154458:1874663042840576:192.168.203.147@tcp:36:rmdir.0 00000100:00100000:1.0:1787825956.769065:0:154458:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b59480 going to sleep for 11 seconds 00000100:00100000:0.0:1787825956.784794:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787825956.785138:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048724992 00000100:00100000:1.0:1787825956.785151:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048724992 00000100:00100000:1.0:1787825956.785222:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9135 00000100:00100000:1.0:1787825956.785236:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a5880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:7728:x1874663048724992:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787825956.785264:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6e58c lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x3ae:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce6fc2 expref: -99 pid: 154458 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825956.785362:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787825956.785370:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a5880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:7728:x1874663048724992:12345-192.168.203.147@tcp:104: Request processed in 137us (639us total) trans 0 rc 0/0 00000100:00100000:1.0:1787825956.785378:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9135 00000100:00100000:0.0:1787825956.803271:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787825956.803646:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048725248 00000100:00100000:1.0:1787825956.803661:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048725248 00000100:00100000:1.0:1787825956.803734:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9136 00000100:00100000:1.0:1787825956.803776:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a7b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663048725248:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787825956.803799:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4e00/0xbef42c0221d6e593 lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x3ae:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce6fd0 expref: -99 pid: 154458 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787825956.803951:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787825956.803960:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a7b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663048725248:12345-192.168.203.147@tcp:104: Request processed in 215us (704us total) trans 0 rc 0/0 00000100:00100000:1.0:1787825956.803970:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9136 00000100:00100000:1.0:1787825956.842355:0:154458:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1880 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154458:1874663042840576:192.168.203.147@tcp:36:rmdir.0 00000080:00200000:1.0:1787825956.842417:0:154458:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1b8d:0x0](ffff9181c83b42c8) 00000080:00200000:0.0:1787825956.880656:0:154459:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787825956.880699:0:154459:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787825956.880723:0:154459:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825956.880776:0:154459:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825956.880785:0:154459:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787825956.880794:0:154459:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787825956.880798:0:154459:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787825956.880809:0:154459:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501c7bb58 00010000:00010000:0.0:1787825956.880813:0:154459:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787825956.880825:0:154459:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce1292 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787825956.880846:0:154459:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825956.880855:0:154459:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=29 00000080:00200000:0.0:1787825956.880988:0:154459:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787825956.881003:0:154459:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=13 00000080:00200000:0.0:1787825956.881007:0:154459:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787825956.881025:0:154459:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787825956.881085:0:154459:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512a00/0xbef42c0221d6e59a lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 154459 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825956.881093:0:154459:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512a00/0xbef42c0221d6e59a lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154459 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825956.881496:0:154459:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512a00/0xbef42c0221d6e59a lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154459 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787825956.881532:0:154459:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154459:1874663042840960:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787825956.881706:0:154459:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c4aecb40 going to sleep for 11 seconds 00000100:00100000:0.0:1787825956.887684:0:154459:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154459:1874663042840960:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787825956.887718:0:154459:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512a00/0xbef42c0221d6e59a lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 154459 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825956.887735:0:154459:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512a00/0xbef42c0221d6e59a lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154459 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825956.887774:0:154459:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512a00/0xbef42c0221d6e59a lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154459 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825956.887783:0:154459:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512a00/0xbef42c0221d6e59a lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 154459 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825956.887793:0:154459:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512a00/0xbef42c0221d6e59a lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 154459 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825956.887800:0:154459:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0512a00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787825956.887805:0:154459:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512a00/0xbef42c0221d6e59a lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154459 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825956.887814:0:154459:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512a00/0xbef42c0221d6e59a lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 154459 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825956.887821:0:154459:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512a00/0xbef42c0221d6e59a lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 154459 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825956.887829:0:154459:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787825956.887836:0:154459:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512a00/0xbef42c0221d6e59a lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154459 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825956.887842:0:154459:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512a00/0xbef42c0221d6e59a lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 154459 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787825956.888781:0:154459:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c55a4e00 x1874663042840960/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787825972 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787825956.888806:0:154459:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00000100:00100000:1.0:1787825956.992462:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042841728:192.168.203.147@tcp:38:rmdir.0 00000100:00080000:1.0:1787825958.433155:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) a9ce6693-0634-46c5-bf21-5ca7d026d44e->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787825958.433164:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787825958.433168:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c39f6800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787825958.433194:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c4d2b000: connect to NID 192.168.203.147@tcp last attempt 8028 00000100:00080000:1.0:1787825958.433212:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c4d2b000: tried all connections, increasing latency to 5s 00000100:00080000:1.0:1787825958.433221:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c4d2b000: import ffff9181c39f6800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787825958.433235:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787825958.433261:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0700 x1874663042841984/t0(0) o38->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787825958.433297:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c2af3000 lustre-OST0000_UUID: changing import state from FULL to CONNECTING 00000100:00080000:1.0:1787825958.433301:0:111835:0:(import.c:2101:ptlrpc_disconnect_and_idle_import()) lustre-OST0000-osc-ffff9181c4d2b000: disconnect after 23s idle 00000100:00080000:1.0:1787825958.433309:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181e0e59800 lustre-OST0001_UUID: changing import state from FULL to CONNECTING 00000100:00080000:1.0:1787825958.433311:0:111835:0:(import.c:2101:ptlrpc_disconnect_and_idle_import()) lustre-OST0001-osc-ffff9181c4d2b000: disconnect after 23s idle 00000100:00100000:1.0:1787825958.433337:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042842112:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787825958.433460:0:2389:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1787825958.433478:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7528148b-9bee-4d31-b597-8aae3b0b4982:2389:1874663042841856:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787825958.433528:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042841984:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787825958.433556:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042842240:192.168.203.147@tcp:9:kworker.0 00000100:00100000:1.0:1787825958.433580:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042842368:192.168.203.147@tcp:9:kworker.0 00000100:00100000:1.0:1787825958.442525:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042842112:192.168.203.147@tcp:400:kworker.0 00000100:00080000:0.0:1787825958.464553:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c39f6800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787825958.464576:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-19) 00000100:00100000:0.0:1787825958.464583:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042841984:192.168.203.147@tcp:38:kworker.0 00000100:00080000:1.0:1787825958.466583:0:2388:0:(import.c:2011:ptlrpc_disconnect_idle_interpret()) @@@ inflight=1, refcount=3: rc = 0 req@ffff9181c35b1c00 x1874663042842240/t0(0) o9->lustre-OST0000-osc-ffff9181c4d2b000@192.168.203.147@tcp:28/4 lens 224/224 e 0 to 0 dl 1787825968 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787825958.466601:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c2af3000 lustre-OST0000_UUID: changing import state from CONNECTING to IDLE 00000100:00100000:1.0:1787825958.466607:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042842240:192.168.203.147@tcp:9:kworker.0 00000100:00080000:1.0:1787825958.466627:0:2388:0:(import.c:2011:ptlrpc_disconnect_idle_interpret()) @@@ inflight=1, refcount=3: rc = 0 req@ffff9181c35b3b80 x1874663042842368/t0(0) o9->lustre-OST0001-osc-ffff9181c4d2b000@192.168.203.147@tcp:28/4 lens 224/224 e 0 to 0 dl 1787825968 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787825958.466635:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181e0e59800 lustre-OST0001_UUID: changing import state from CONNECTING to IDLE 00000100:00100000:1.0:1787825958.466638:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042842368:192.168.203.147@tcp:9:kworker.0 00000100:00100000:1.0:1787825963.559426:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c49e6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042842624:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787825968.293766:0:154460:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6eb3000 going to sleep for 1 seconds 00000100:00080000:0.0:1787825968.677892:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) a9ce6693-0634-46c5-bf21-5ca7d026d44e->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787825968.677899:0:111835:0:(pinger.c:247:ptlrpc_pinger_process_import()) a9ce6693-0634-46c5-bf21-5ca7d026d44e->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1787825968.677999:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7528148b-9bee-4d31-b597-8aae3b0b4982:2390:1874663042842880:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787825968.678103:0:2390:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1787825968.678110:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042843008:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787825969.312959:0:154460:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6eb3000 going to sleep for 1 seconds 00000100:00100000:0.0:1787825970.337158:0:154460:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6eb3000 going to sleep for 1 seconds 00000100:00100000:0.0:1787825971.360936:0:154460:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6eb3000 going to sleep for 1 seconds 00000100:00100000:0.0:1787825972.386020:0:154460:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6eb3000 going to sleep for 1 seconds 00000100:00100000:0.0:1787825973.409010:0:154460:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6eb3000 going to sleep for 1 seconds 00000100:00080000:0.0:1787825973.793298:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) a9ce6693-0634-46c5-bf21-5ca7d026d44e->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787825973.793306:0:111835:0:(pinger.c:247:ptlrpc_pinger_process_import()) a9ce6693-0634-46c5-bf21-5ca7d026d44e->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1787825973.793353:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7528148b-9bee-4d31-b597-8aae3b0b4982:2390:1874663042843136:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787825973.793460:0:2390:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1787825973.793468:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042843264:192.168.203.147@tcp:400:kworker.0 00000100:00000400:0.0:1787825974.627289:0:2389:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787825958/real 1787825958] req@ffff9181c35b0380 x1874663042841856/t0(0) o400->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 224/224 e 0 to 1 dl 1787825974 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787825974.627319:0:2389:0:(client.c:2547:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff9181c35b0380 x1874663042841856/t0(0) o400->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 224/224 e 0 to 1 dl 1787825974 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787825974.627329:0:2389:0:(import.c:65:import_set_state_nolock()) ffff9181c5a6c800 MGS: changing import state from FULL to DISCONN 00000100:02020000:0.0:1787825974.627333:0:2389:0:(import.c:186:ptlrpc_set_import_discon()) MGC192.168.203.147@tcp: Connection to MGS (at 192.168.203.147@tcp) was lost; in progress operations using this service will fail 00000100:00080000:0.0:1787825974.627336:0:2389:0:(import.c:192:ptlrpc_set_import_discon()) import MGS@192.168.203.147@tcp for MGC192.168.203.147@tcp not replayable, auto-deactivating 00000100:00080000:0.0:1787825974.627339:0:2389:0:(import.c:138:ptlrpc_deactivate_import_nolock()) setting import MGS INVALID 00000100:00100000:0.0:1787825974.627344:0:2389:0:(client.c:3589:ptlrpc_abort_inflight()) @@@ inflight req@ffff9181c35b0380 x1874663042841856/t0(0) o400->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 224/224 e 0 to 1 dl 1787825974 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787825974.627365:0:2389:0:(client.c:3589:ptlrpc_abort_inflight()) @@@ inflight req@ffff9181c71f0700 x1874663042842496/t0(0) o400->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 224/224 e 0 to 0 dl 1787825979 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787825974.627388:0:2389:0:(client.c:3589:ptlrpc_abort_inflight()) @@@ inflight req@ffff9181c55a7100 x1874663042842880/t0(0) o400->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 224/224 e 0 to 0 dl 1787825984 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787825974.627403:0:2389:0:(client.c:3589:ptlrpc_abort_inflight()) @@@ inflight req@ffff9181c55a4700 x1874663042843136/t0(0) o400->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 224/224 e 0 to 0 dl 1787825989 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 10000000:01000000:0.0:1787825974.627412:0:2389:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808001 00000100:00080000:0.0:1787825974.627414:0:2389:0:(pinger.c:193:ptlrpc_pinger_ir_down()) IR down 10000000:01000000:0.0:1787825974.627416:0:2389:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808002 00000100:00080000:0.0:1787825974.627417:0:2389:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00100000:0.0:1787825974.627443:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7528148b-9bee-4d31-b597-8aae3b0b4982:2389:1874663042841856:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787825974.627480:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7528148b-9bee-4d31-b597-8aae3b0b4982:2390:1874663042843136:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787825974.627488:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7528148b-9bee-4d31-b597-8aae3b0b4982:2390:1874663042842880:192.168.203.147@tcp:400:kworker.0 00000100:00080000:0.0:1787825974.627508:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7528148b-9bee-4d31-b597-8aae3b0b4982->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787825974.627513:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787825974.627516:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5a6c800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787825974.627521:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 7770 00000100:00080000:0.0:1787825974.627534:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c5a6c800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787825974.627561:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7100 x1874663042843392/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787825974.627580:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) a9ce6693-0634-46c5-bf21-5ca7d026d44e->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787825974.627584:0:99984:0:(pinger.c:247:ptlrpc_pinger_process_import()) a9ce6693-0634-46c5-bf21-5ca7d026d44e->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1787825974.627618:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042843520:192.168.203.147@tcp:400:kworker.0 00010000:00010000:0.0:1787825977.188873:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787825977.510913:0:154460:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6eb3000 going to sleep for 1 seconds 00000100:00100000:0.0:1787825979.745709:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c49e7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042843648:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1787825981.283256:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787825983.843429:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787825963/real 1787825963] req@ffff9181c49e6300 x1874663042842624/t0(0) o38->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787825983 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787825983.843472:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c49e6300 x1874663042842624/t0(0) o38->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787825983 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787825983.843509:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c39f6800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787825983.843528:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787825983.843533:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c49e6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042842624:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787825983.846213:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c494c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042843776:192.168.203.147@tcp:38:kworker.0 00000100:00000400:0.0:1787825984.032906:0:2390:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787825968/real 1787825968] req@ffff9181c55a4e00 x1874663042843008/t0(0) o400->lustre-MDT0001-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 224/224 e 0 to 1 dl 1787825984 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787825984.032960:0:2390:0:(client.c:2547:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff9181c55a4e00 x1874663042843008/t0(0) o400->lustre-MDT0001-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 224/224 e 0 to 1 dl 1787825984 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787825984.032970:0:2390:0:(import.c:214:ptlrpc_set_import_discon()) mdc: import ffff9181c39f4800 already not connected (conn 15, was 14): CONNECTING 00000100:00100000:0.0:1787825984.032991:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042843008:192.168.203.147@tcp:400:kworker.0 00010000:00010000:0.0:1787825985.378162:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000400:0.0:1787825989.153646:0:2390:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787825973/real 1787825973] req@ffff9181c55a4380 x1874663042843264/t0(0) o400->lustre-MDT0001-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 224/224 e 0 to 1 dl 1787825989 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787825989.153676:0:2390:0:(client.c:2547:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff9181c55a4380 x1874663042843264/t0(0) o400->lustre-MDT0001-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 224/224 e 0 to 1 dl 1787825989 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787825989.153686:0:2390:0:(import.c:214:ptlrpc_set_import_discon()) mdc: import ffff9181c39f4800 already not connected (conn 15, was 14): CONNECTING 00000100:00100000:0.0:1787825989.153701:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042843264:192.168.203.147@tcp:400:kworker.0 00010000:00010000:0.0:1787825989.473650:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787825989.985184:0:152015:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7528148b-9bee-4d31-b597-8aae3b0b4982->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787825989.985190:0:152015:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7528148b-9bee-4d31-b597-8aae3b0b4982->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1787825989.985193:0:152015:0:(pinger.c:229:ptlrpc_pinger_process_import()) a9ce6693-0634-46c5-bf21-5ca7d026d44e->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787825989.985197:0:152015:0:(pinger.c:247:ptlrpc_pinger_process_import()) a9ce6693-0634-46c5-bf21-5ca7d026d44e->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1787825989.985199:0:152015:0:(pinger.c:229:ptlrpc_pinger_process_import()) a9ce6693-0634-46c5-bf21-5ca7d026d44e->lustre-MDT0001_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787825989.985203:0:152015:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787825989.985207:0:152015:0:(import.c:65:import_set_state_nolock()) ffff9181c39f4800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787825989.985213:0:152015:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c4d2b000: connect to NID 192.168.203.147@tcp last attempt 8051 00000100:00080000:0.0:1787825989.985227:0:152015:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c4d2b000: tried all connections, increasing latency to 5s 00000100:00080000:0.0:1787825989.985237:0:152015:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c4d2b000: import ffff9181c39f4800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787825989.985262:0:152015:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787825989.985305:0:152015:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4380 x1874663042844032/t0(0) o38->lustre-MDT0001-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787825989.985435:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042844032:192.168.203.147@tcp:38:kworker.0 00010000:00080000:1.0:1787825990.001859:0:2388:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffff9181c55a4380 x1874663042844032/t0(0) o38->lustre-MDT0001-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 520/416 e 0 to 0 dl 1787826004 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787825990.001900:0:2388:0:(import.c:1132:ptlrpc_connect_interpret()) lustre-MDT0001-mdc-ffff9181c4d2b000: connect to target with instance 4 00000100:00080000:1.0:1787825990.001929:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c39f4800 lustre-MDT0001_UUID: changing import state from CONNECTING to REPLAY 00000100:00080000:1.0:1787825990.001937:0:2388:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0001_UUID 00000100:00100000:1.0:1787825990.001940:0:2388:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff9181c4d2b000: committing for last_committed 12884907729 gen 1 00000100:00080000:1.0:1787825990.001945:0:2388:0:(recover.c:53:ptlrpc_replay_next()) import ffff9181c39f4800 from lustre-MDT0001_UUID committed 12884907729 last 0 00000100:00080000:1.0:1787825990.001949:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c39f4800 lustre-MDT0001_UUID: changing import state from REPLAY to REPLAY_LOCKS 00010000:00080000:1.0:1787825990.002706:0:154951:0:(ldlm_request.c:2972:ldlm_lock_replay_thread()) lock replay thread lustre-MDT0001-mdc-ffff9181c4d2b000 to lustre-MDT0001_UUID@192.168.203.147@tcp 00010000:00010000:1.0:1787825990.002712:0:154951:0:(ldlm_request.c:2888:ldlm_cancel_unused_locks_for_replay()) Dropping as many unused locks as possible before replay for namespace lustre-MDT0001-mdc-ffff9181c4d2b000 (1) 00010000:00010000:1.0:1787825990.002851:0:154951:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6400/0xbef42c0221d64b07 lrc: 0/0,0 mode: --/PR res: [0x2400007ed:0xa85:0x0].0x0 bits 0x10/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000000 nid: local remote: 0x878a5eea00cb9f32 expref: -99 pid: 151586 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787825990.002940:0:154951:0:(ldlm_request.c:2901:ldlm_cancel_unused_locks_for_replay()) Canceled 0 unused locks from namespace lustre-MDT0001-mdc-ffff9181c4d2b000 00000100:00080000:1.0:1787825990.004617:0:154951:0:(import.c:65:import_set_state_nolock()) ffff9181c39f4800 lustre-MDT0001_UUID: changing import state from REPLAY_LOCKS to REPLAY_WAIT 00000100:00000400:0.0:1787825990.178637:0:2390:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787825974/real 1787825974] req@ffff9181c55a4700 x1874663042843520/t0(0) o400->lustre-MDT0001-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 224/224 e 0 to 1 dl 1787825990 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787825990.178683:0:2390:0:(client.c:2547:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff9181c55a4700 x1874663042843520/t0(0) o400->lustre-MDT0001-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 224/224 e 0 to 1 dl 1787825990 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787825990.178715:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042843520:192.168.203.147@tcp:400:kworker.0 00010000:00010000:0.0:1787825993.572903:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787825993.896393:0:154460:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6eb3000 going to sleep for 1 seconds 00000100:00100000:1.0:1787825994.916432:0:154460:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6eb3000 going to sleep for 1 seconds 00000100:00080000:0.0:1787825995.296899:0:138695:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7528148b-9bee-4d31-b597-8aae3b0b4982->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787825995.296910:0:138695:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7528148b-9bee-4d31-b597-8aae3b0b4982->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1787825995.296913:0:138695:0:(pinger.c:229:ptlrpc_pinger_process_import()) a9ce6693-0634-46c5-bf21-5ca7d026d44e->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787825995.296917:0:138695:0:(pinger.c:247:ptlrpc_pinger_process_import()) a9ce6693-0634-46c5-bf21-5ca7d026d44e->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1787825995.297010:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042844288:192.168.203.147@tcp:400:kworker.0 00010000:00080000:1.0:1787825995.310389:0:2389:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffff9181c55a4700 x1874663042844288/t0(0) o400->lustre-MDT0001-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 224/224 e 0 to 0 dl 1787826011 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787825995.310416:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042844288:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787825995.937071:0:154460:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6eb3000 going to sleep for 1 seconds 00000100:00100000:1.0:1787825996.960969:0:154460:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6eb3000 going to sleep for 1 seconds 00010000:00010000:0.0:1787825997.665003:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787825997.984897:0:154460:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6eb3000 going to sleep for 1 seconds 00000100:00100000:1.0:1787825999.008941:0:154460:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6eb3000 going to sleep for 1 seconds 00000100:00080000:0.0:1787825999.290556:0:155046:0:(import.c:1598:ptlrpc_invalidate_import_thread()) thread invalidate import MGC192.168.203.147@tcp to MGS@192.168.203.147@tcp 00000100:00100000:0.0:1787825999.290562:0:155046:0:(import.c:323:ptlrpc_invalidate_import()) Sleeping 20s for inflight to error out 10000000:01000000:0.0:1787825999.290567:0:155046:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808003 00010000:00010000:0.0:1787825999.290583:0:155046:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e06f6000/0xbef42c0221d63551 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x110e400000000 nid: local remote: 0x878a5eea00cb0635 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1787825999.290594:0:155046:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e06f6000/0xbef42c0221d63551 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x110f400000000 nid: local remote: 0x878a5eea00cb0635 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787825999.290600:0:155046:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:0.0:1787825999.290687:0:155046:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787825999.290697:0:155046:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e06f6000/0xbef42c0221d63551 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x590f400000000 nid: local remote: 0x878a5eea00cb0635 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825999.290714:0:155046:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181c44fb000/0xbef42c0221d6355f lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0x878a5eea00cb0643 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1787825999.290720:0:155046:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181c44fb000/0xbef42c0221d6355f lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0x878a5eea00cb0643 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787825999.290725:0:155046:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:0.0:1787825999.293529:0:155046:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787825999.293547:0:155046:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181c44fb000/0xbef42c0221d6355f lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0x878a5eea00cb0643 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825999.293576:0:155046:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e0138200/0xbef42c0221d645d5 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0x878a5eea00cb919b expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1787825999.293584:0:155046:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e0138200/0xbef42c0221d645d5 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0x878a5eea00cb919b expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787825999.293591:0:155046:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:0.0:1787825999.293597:0:155046:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787825999.293600:0:155046:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e0138200/0xbef42c0221d645d5 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0x878a5eea00cb919b expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787825999.293612:0:155046:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e06f7000/0xbef42c0221d63558 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0x878a5eea00cb063c expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1787825999.293618:0:155046:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e06f7000/0xbef42c0221d63558 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0x878a5eea00cb063c expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787825999.293623:0:155046:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:0.0:1787825999.293627:0:155046:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787825999.293630:0:155046:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e06f7000/0xbef42c0221d63558 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0x878a5eea00cb063c expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787825999.293648:0:155046:0:(import.c:65:import_set_state_nolock()) ffff9181c5a6c800 MGS: changing import state from EVICTED to RECOVER 00000100:00080000:0.0:1787825999.293653:0:155046:0:(import.c:65:import_set_state_nolock()) ffff9181c5a6c800 MGS: changing import state from RECOVER to FULL 10000000:01000000:0.0:1787825999.293656:0:155046:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808004 00000100:00080000:0.0:1787825999.293660:0:155046:0:(pinger.c:186:ptlrpc_pinger_ir_up()) IR up 00000100:02000000:0.0:1787825999.293666:0:155046:0:(import.c:1743:ptlrpc_import_recovery_state_machine()) MGC192.168.203.147@tcp: Connection restored to 192.168.203.147@tcp (at 192.168.203.147@tcp) 00000100:00100000:1.0:1787826000.032881:0:154460:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6eb3000 going to sleep for 1 seconds 00000020:01000000:1.0:1787826000.582099:0:15163:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-client gen 52-51 (rc=0) 10000000:01000000:1.0:1787826000.582118:0:15163:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-client' succeeded (0). 00010000:00010000:1.0:1787826000.582129:0:15163:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e06f7600/0xbef42c0221d6e5bd lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00ce7499 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826000.582145:0:15163:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.147@tcp lock: ffff9181e06f7600/0xbef42c0221d6e5bd lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00ce7499 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826000.582155:0:15163:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:1.0:1787826000.582158:0:15163:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff9181c4d2b000 from 7 10000000:01000000:1.0:1787826000.582160:0:15163:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00010000:00010000:1.0:1787826000.582214:0:15163:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6e5c4 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826000.582285:0:15163:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6e5c4 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826000.582296:0:15163:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6e5c4 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826000.582324:0:15163:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:7528148b-9bee-4d31-b597-8aae3b0b4982:15163:1874663042845440:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00100000:1.0:1787826000.582538:0:15163:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c80c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826000.594808:0:15163:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:7528148b-9bee-4d31-b597-8aae3b0b4982:15163:1874663042845440:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787826000.594840:0:15163:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6e5c4 lrc: 4/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00ce74a0 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826000.594853:0:15163:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6e5c4 lrc: 4/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x878a5eea00ce74a0 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826000.594940:0:15163:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:7528148b-9bee-4d31-b597-8aae3b0b4982:15163:1874663042845568:192.168.203.147@tcp:501:ll_cfg_requeue.0 00000100:00100000:1.0:1787826000.597965:0:15163:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c80c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826000.601234:0:15163:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:7528148b-9bee-4d31-b597-8aae3b0b4982:15163:1874663042845568:192.168.203.147@tcp:501:ll_cfg_requeue.0 00000100:00100000:1.0:1787826000.601304:0:15163:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:7528148b-9bee-4d31-b597-8aae3b0b4982:15163:1874663042845696:192.168.203.147@tcp:503:ll_cfg_requeue.0 00000100:00100000:1.0:1787826000.601730:0:15163:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c80c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826000.611735:0:15163:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:7528148b-9bee-4d31-b597-8aae3b0b4982:15163:1874663042845696:192.168.203.147@tcp:503:ll_cfg_requeue.0 00000040:00080000:0.0:1787826000.622142:0:155049:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x2:0x0] index 64768 count 7 00000020:01000000:1.0:1787826000.622784:0:15163:0:(obd_config.c:2145:class_config_parse_llog()) Processed log params gen 7-6 (rc=0) 10000000:01000000:1.0:1787826000.622816:0:15163:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'params' succeeded (0). 00010000:00010000:1.0:1787826000.622828:0:15163:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6e5c4 lrc: 3/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00ce74a0 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826000.622841:0:15163:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6e5c4 lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00ce74a0 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826000.622851:0:15163:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:1.0:1787826000.622854:0:15163:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff9181c3d3cd28 from 1 10000000:01000000:1.0:1787826000.622857:0:15163:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00010000:00010000:1.0:1787826000.622898:0:15163:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5a00/0xbef42c0221d6e5cb lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826000.622907:0:15163:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b5a00/0xbef42c0221d6e5cb lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826000.622919:0:15163:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b5a00/0xbef42c0221d6e5cb lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826000.622959:0:15163:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:7528148b-9bee-4d31-b597-8aae3b0b4982:15163:1874663042845824:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00100000:1.0:1787826000.623046:0:15163:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c80c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826000.634841:0:15163:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:7528148b-9bee-4d31-b597-8aae3b0b4982:15163:1874663042845824:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787826000.634872:0:15163:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.147@tcp lock: ffff9181e01b5a00/0xbef42c0221d6e5cb lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00ce74a7 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826000.634886:0:15163:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.147@tcp lock: ffff9181e01b5a00/0xbef42c0221d6e5cb lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x878a5eea00ce74a7 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826000.634981:0:15163:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:7528148b-9bee-4d31-b597-8aae3b0b4982:15163:1874663042845952:192.168.203.147@tcp:501:ll_cfg_requeue.0 00000100:00100000:1.0:1787826000.635066:0:15163:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c80c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826000.641381:0:15163:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:7528148b-9bee-4d31-b597-8aae3b0b4982:15163:1874663042845952:192.168.203.147@tcp:501:ll_cfg_requeue.0 10000000:01000000:1.0:1787826000.641433:0:15163:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-sptlrpc' failed (-2). 00010000:00010000:1.0:1787826000.641444:0:15163:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5a00/0xbef42c0221d6e5cb lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00ce74a7 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826000.641455:0:15163:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.147@tcp lock: ffff9181e01b5a00/0xbef42c0221d6e5cb lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00ce74a7 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826001.058243:0:154460:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6eb3000 going to sleep for 1 seconds 00010000:00010000:0.0:1787826001.762688:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826002.080960:0:154460:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6eb3000 going to sleep for 1 seconds 00000100:00100000:1.0:1787826003.104882:0:154460:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6eb3000 going to sleep for 1 seconds 00000100:00100000:1.0:1787826004.128877:0:154460:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6eb3000 going to sleep for 1 seconds 00000100:00100000:1.0:1787826005.154925:0:154460:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6eb3000 going to sleep for 1 seconds 00010000:00010000:0.0:1787826005.857115:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826006.176881:0:154460:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6eb3000 going to sleep for 1 seconds 00000100:00100000:1.0:1787826007.201020:0:154460:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6eb3000 going to sleep for 1 seconds 00000100:00100000:1.0:1787826008.226787:0:154460:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6eb3000 going to sleep for 1 seconds 00000100:00100000:1.0:1787826008.482301:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7528148b-9bee-4d31-b597-8aae3b0b4982:2390:1874663042846336:192.168.203.147@tcp:400:kworker.0 00000100:00080000:0.0:1787826008.482410:0:138695:0:(pinger.c:229:ptlrpc_pinger_process_import()) a9ce6693-0634-46c5-bf21-5ca7d026d44e->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826008.482414:0:138695:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787826008.482417:0:138695:0:(import.c:65:import_set_state_nolock()) ffff9181c39f6800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826008.482422:0:138695:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c4d2b000: connect to NID 192.168.203.147@tcp last attempt 0 00000100:00080000:0.0:1787826008.482432:0:138695:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c4d2b000: import ffff9181c39f6800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787826008.482443:0:138695:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826008.482459:0:138695:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7800 x1874663042846464/t0(0) o38->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826008.482598:0:2390:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1787826008.482604:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042846592:192.168.203.147@tcp:400:kworker.0 00010000:00080000:1.0:1787826008.488404:0:2390:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffff9181c55a6680 x1874663042846592/t0(0) o400->lustre-MDT0001-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 224/224 e 0 to 0 dl 1787826024 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787826008.488425:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042846592:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826008.488437:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7528148b-9bee-4d31-b597-8aae3b0b4982:2390:1874663042846336:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826009.249288:0:154460:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6eb3000 going to sleep for 1 seconds 00000100:00100000:0.0:1787826009.828884:0:155031:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5cf9240 going to sleep for 0 seconds 00000100:00100000:1.0:1787826010.273029:0:154460:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6eb3000 going to sleep for 1 seconds 00000100:00100000:0.0:1787826010.631499:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c33fd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042711680:192.168.203.147@tcp:36:lfs.0 00000100:00100000:0.0:1787826010.848826:0:155031:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5cf9240 going to sleep for 0 seconds 00000100:00100000:1.0:1787826011.297308:0:154460:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6eb3000 going to sleep for 1 seconds 00000100:00100000:0.0:1787826011.872893:0:155031:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5cf9240 going to sleep for 0 seconds 00000100:00100000:1.0:1787826012.320844:0:154460:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6eb3000 going to sleep for 1 seconds 00000100:00100000:0.0:1787826012.896923:0:155031:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5cf9240 going to sleep for 0 seconds 00000100:00100000:0.0:1787826013.105074:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042841472:192.168.203.147@tcp:36:lfs.0 00000100:00100000:0.0:1787826013.106227:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c49e6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042846720:192.168.203.147@tcp:101:ldlm_lock_repla.0 00000100:00100000:0.0:1787826013.106323:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c49e5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042846848:192.168.203.147@tcp:101:ldlm_lock_repla.0 00000100:00100000:0.0:1787826013.106348:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c49e7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042846976:192.168.203.147@tcp:101:ldlm_lock_repla.0 00010000:00010000:0.0:1787826013.110025:0:2388:0:(ldlm_request.c:2777:replay_lock_interpret()) ### replayed lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000100:00100000:0.0:1787826013.110059:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c49e6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042846720:192.168.203.147@tcp:101:ldlm_lock_repla.0 00010000:00010000:0.0:1787826013.110082:0:2388:0:(ldlm_request.c:2777:replay_lock_interpret()) ### replayed lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513000/0xbef42c0221d64bee lrc: 2/0,0 mode: CR/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce962d expref: -99 pid: 152210 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000100:00100000:0.0:1787826013.110103:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c49e5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042846848:192.168.203.147@tcp:101:ldlm_lock_repla.0 00010000:00010000:0.0:1787826013.111989:0:2388:0:(ldlm_request.c:2777:replay_lock_interpret()) ### replayed lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6800/0xbef42c0221d64b3f lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x11/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9634 expref: -99 pid: 152202 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000100:00080000:0.0:1787826013.112012:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c39f6800 lustre-MDT0000_UUID: changing import state from REPLAY_LOCKS to REPLAY_WAIT 00000100:00100000:0.0:1787826013.112057:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c49e7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042846976:192.168.203.147@tcp:101:ldlm_lock_repla.0 00000100:00100000:0.0:1787826013.112101:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042847104:192.168.203.147@tcp:400:ptlrpcd_rcv.0 00000100:00100000:0.0:1787826013.134723:0:2388:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9181c4d2b000: committing for last_committed 330712484066 gen 11 00000100:00080000:0.0:1787826013.138342:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c39f6800 lustre-MDT0000_UUID: changing import state from REPLAY_WAIT to RECOVER 00000100:00080000:0.0:1787826013.138370:0:2388:0:(client.c:3247:ptlrpc_resend_req()) @@@ going to resend req@ffff9181c55a6a00 x1874663042847104/t0(0) o400->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 224/224 e 0 to 0 dl 1787826029 ref 1 fl Interpret:RQU/2c0/0 rc 0/0 job:'ptlrpcd_rcv.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826013.138383:0:2388:0:(client.c:3256:ptlrpc_resend_req()) @@@ it has reply, so skip it req@ffff9181c55a6a00 x1874663042847104/t0(0) o400->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 224/224 e 0 to 0 dl 1787826029 ref 1 fl Interpret:RQU/2c0/0 rc 0/0 job:'ptlrpcd_rcv.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826013.138391:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c39f6800 lustre-MDT0000_UUID: changing import state from RECOVER to FULL 00000100:02000000:0.0:1787826013.139655:0:2388:0:(import.c:1743:ptlrpc_import_recovery_state_machine()) lustre-MDT0000-mdc-ffff9181c4d2b000: Connection restored to 192.168.203.147@tcp (at 192.168.203.147@tcp) 00000100:00080000:0.0:1787826013.139669:0:2388:0:(recover.c:191:ptlrpc_wake_delayed()) @@@ waking (set 000000008b2b6d8d): req@ffff9181c71f1880 x1874663042841600/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/224 e 0 to 0 dl 1787825972 ref 2 fl Rpc:RSQU/600/0 rc -107/-107 job:'rmdir.0' uid:0 gid:0 projid:0 00000100:00080000:0.0:1787826013.139697:0:2388:0:(recover.c:191:ptlrpc_wake_delayed()) @@@ waking (set 0000000015129327): req@ffff9181c49e7100 x1874663042844416/t0(0) o41->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 224/368 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826013.139729:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042847104:192.168.203.147@tcp:400:ptlrpcd_rcv.0 00000100:00080000:1.0:1787826013.142253:0:154460:0:(client.c:3740:ptlrpc_set_mbits()) resend with new mbits old x1874663042841600 new x1874663042847232 00000100:00100000:1.0:1787826013.142352:0:154460:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6eb3000 going to sleep for 11 seconds 00000100:00100000:1.0:1787826013.156284:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c49e6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042847488:192.168.203.147@tcp:8:lfs.0 00000100:00100000:1.0:1787826013.168670:0:2388:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff9181c4d2b000: committing for last_committed 34359738457 gen 39 00000100:00080000:1.0:1787826013.168700:0:2388:0:(import.c:1132:ptlrpc_connect_interpret()) lustre-OST0000-osc-ffff9181c4d2b000: connect to target with instance 8 00000100:00080000:1.0:1787826013.168723:0:2388:0:(import.c:956:ptlrpc_connect_set_flags()) lustre-OST0000-osc-ffff9181c4d2b000: Resetting ns_connect_flags to server flags: 0xa0427af2e3440038 00000100:00080000:1.0:1787826013.168740:0:2388:0:(import.c:1213:ptlrpc_connect_interpret()) connected to replayable target: lustre-OST0000_UUID 00000100:00080000:1.0:1787826013.168785:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c2af3000 lustre-OST0000_UUID: changing import state from CONNECTING to FULL 00000080:00000004:1.0:1787826013.168800:0:2388:0:(lcommon_misc.c:94:cl_ocd_update()) Changing connect_flags: 0xa0427af2e3440038 -> 0xa0427af2e3440038 00000080:00080000:1.0:1787826013.168807:0:2388:0:(lcommon_misc.c:56:cl_init_ea_size()) updating def/max_easize: 72/65536 00000100:00080000:1.0:1787826013.168826:0:2388:0:(recover.c:191:ptlrpc_wake_delayed()) @@@ waking (set 0000000015129327): req@ffff9181c49e7100 x1874663042847616/t0(0) o13->lustre-OST0000-osc-ffff9181c4d2b000@192.168.203.147@tcp:7/4 lens 224/368 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787826013.168854:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c49e6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042847488:192.168.203.147@tcp:8:lfs.0 00000100:00100000:0.0:1787826013.204371:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c49e7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042848000:192.168.203.147@tcp:8:lfs.0 00000100:00100000:1.0:1787826013.209678:0:2388:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-ffff9181c4d2b000: committing for last_committed 4294975530 gen 32 00000100:00080000:1.0:1787826013.209900:0:2388:0:(import.c:1132:ptlrpc_connect_interpret()) lustre-OST0001-osc-ffff9181c4d2b000: connect to target with instance 1 00000100:00080000:1.0:1787826013.209914:0:2388:0:(import.c:956:ptlrpc_connect_set_flags()) lustre-OST0001-osc-ffff9181c4d2b000: Resetting ns_connect_flags to server flags: 0xa0427af2e3440038 00000100:00080000:1.0:1787826013.209929:0:2388:0:(import.c:1213:ptlrpc_connect_interpret()) connected to replayable target: lustre-OST0001_UUID 00000100:00080000:1.0:1787826013.209935:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181e0e59800 lustre-OST0001_UUID: changing import state from CONNECTING to FULL 00000080:00000004:1.0:1787826013.209948:0:2388:0:(lcommon_misc.c:94:cl_ocd_update()) Changing connect_flags: 0xa0427af2e3440038 -> 0xa0427af2e3440038 00000080:00080000:1.0:1787826013.209954:0:2388:0:(lcommon_misc.c:56:cl_init_ea_size()) updating def/max_easize: 72/65536 00000100:00080000:1.0:1787826013.209974:0:2388:0:(recover.c:191:ptlrpc_wake_delayed()) @@@ waking (set 0000000015129327): req@ffff9181c49e7100 x1874663042848128/t0(0) o13->lustre-OST0001-osc-ffff9181c4d2b000@192.168.203.147@tcp:7/4 lens 224/368 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787826013.210002:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c49e7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042848000:192.168.203.147@tcp:8:lfs.0 00000100:00100000:0.0:1787826013.250001:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826013.250046:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048995584 00000100:00100000:0.0:1787826013.250058:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048995584 00000100:00100000:0.0:1787826013.250133:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9137 00000100:00100000:0.0:1787826013.250146:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:188682:x1874663048995584:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826013.250167:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6e5d2 lrc: 2/0,0 mode: PR/PR res: [0x200025d90:0x3af:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce9665 expref: -99 pid: 154460 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826013.250254:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826013.250261:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:188682:x1874663048995584:12345-192.168.203.147@tcp:104: Request processed in 118us (273us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826013.250270:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9137 00000100:00100000:0.0:1787826013.268961:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826013.269103:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048995840 00000100:00100000:0.0:1787826013.269116:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048995840 00000100:00100000:0.0:1787826013.269234:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9138 00000100:00100000:0.0:1787826013.269245:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a6300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663048995840:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826013.269268:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6e5d9 lrc: 2/0,0 mode: PR/PR res: [0x24000178a:0x3af:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce97e6 expref: -99 pid: 154460 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826013.269363:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826013.269371:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a6300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663048995840:12345-192.168.203.147@tcp:104: Request processed in 128us (420us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826013.269378:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9138 00000100:00100000:0.0:1787826013.296644:0:154460:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c494c700 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:154460:1874663042848384:192.168.203.147@tcp:36:rmdir.0 00000080:00200000:0.0:1787826013.296703:0:154460:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1b8e:0x0](ffff9181c83b42c8) 00000100:00100000:0.0:1787826013.339295:0:155064:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c49e7100 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155064:1874663042848896:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787826013.339328:0:155064:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6e5e7 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155064 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826013.339349:0:155064:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6e5e7 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155064 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826013.339357:0:155064:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6e5e7 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155064 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826013.339366:0:155064:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6e5e7 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155064 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826013.339376:0:155064:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6e5e7 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155064 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826013.339383:0:155064:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e06f6a00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787826013.339388:0:155064:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6e5e7 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155064 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826013.339396:0:155064:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6e5e7 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155064 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826013.339405:0:155064:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6e5e7 lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 155064 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826013.339413:0:155064:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787826013.339421:0:155064:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6e5e7 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155064 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826013.339429:0:155064:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6e5e7 lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155064 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826013.339449:0:155064:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c49e7100 x1874663042848896/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787826029 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787826013.339467:0:155064:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787826013.339499:0:155064:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787826013.339589:0:155064:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826013.339609:0:155064:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826013.339618:0:155064:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826013.339631:0:155064:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826013.339652:0:155064:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826013.339658:0:155064:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826013.339662:0:155064:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826013.339668:0:155064:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb975019ebb58 00010000:00010000:0.0:1787826013.339672:0:155064:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826013.339681:0:155064:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826013.339696:0:155064:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826013.339713:0:155064:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=13 00000080:00200008:0.0:1787826013.339725:0:155064:0:(file.c:6296:ll_getattr_dentry()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1787826013.339731:0:155064:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08),name=d71a.replay-single 00000002:00010000:0.0:1787826013.339785:0:155064:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x20001a9e3:0x17e0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826013.339792:0:155064:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00010000:00010000:0.0:1787826013.339838:0:155064:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e5ee lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 155064 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826013.339846:0:155064:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e5ee lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 155064 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826013.339858:0:155064:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e5ee lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 155064 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826013.339882:0:155064:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155064:1874663042849024:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787826013.339979:0:155064:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bb0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826013.342815:0:155064:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155064:1874663042849024:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787826013.342847:0:155064:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e5ee lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9848 expref: -99 pid: 155064 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826013.342866:0:155064:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e5ee lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9848 expref: -99 pid: 155064 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826013.342876:0:155064:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e5ee lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9848 expref: -99 pid: 155064 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826013.342889:0:155064:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e5ee lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9848 expref: -99 pid: 155064 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826013.342902:0:155064:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e5ee lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9848 expref: -99 pid: 155064 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826013.342916:0:155064:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e5ee lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9848 expref: -99 pid: 155064 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826013.342930:0:155064:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e5ee lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9848 expref: -99 pid: 155064 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826013.342938:0:155064:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e5ee lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9848 expref: -99 pid: 155064 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826013.342951:0:155064:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c55a5500 x1874663042849024/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826029 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787826013.342967:0:155064:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e5ee lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9848 expref: -99 pid: 155064 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826013.342978:0:155064:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787826013.342986:0:155064:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787826013.343077:0:155064:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x17e0:0x0], updating i_size 4096 00000080:00010000:1.0:1787826013.343106:0:155064:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) for lock 0xbef42c0221d6e5ee 00000080:00010000:1.0:1787826013.343116:0:155064:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00000080:00010000:1.0:1787826013.343119:0:155064:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6e5ee from it ffffb975019ebcb8 00010000:00010000:1.0:1787826013.343125:0:155064:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e5ee lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9848 expref: -99 pid: 155064 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826013.343138:0:155064:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e5ee lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9848 expref: -99 pid: 155064 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:1.0:1787826013.344099:0:155064:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), request_mask 0, flags 0, foreign 0 00000080:00200000:1.0:1787826013.344108:0:155064:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826013.344130:0:155064:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826013.344145:0:155064:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826013.344183:0:155064:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826013.344192:0:155064:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826013.344200:0:155064:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826013.344204:0:155064:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787826013.344210:0:155064:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb975019ebcb8 00010000:00010000:1.0:1787826013.344215:0:155064:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826013.344225:0:155064:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826013.345253:0:155064:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826013.345274:0:155064:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826013.345285:0:155064:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826013.345308:0:155064:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826013.345315:0:155064:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826013.345528:0:155064:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826013.345532:0:155064:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787826013.345538:0:155064:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb975019ebbf8 00010000:00010000:1.0:1787826013.345541:0:155064:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826013.345551:0:155064:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826013.345564:0:155064:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826013.345572:0:155064:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=267 00000080:00200000:1.0:1787826013.345579:0:155064:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 44 00000080:00200008:1.0:1787826013.345587:0:155064:0:(file.c:1090:ll_file_open()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00000080:00200080:1.0:1787826013.345614:0:155064:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) cmd=c03066f0 arg=7ffed059f8f0 00000080:00200000:1.0:1787826013.345627:0:155064:0:(dir.c:451:ll_dir_setdirstripe()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) name=test stripe_offset=0 stripe_count=2, hash_type=0 00010000:00010000:1.0:1787826013.345654:0:155064:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e5ee lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ce9848 expref: -99 pid: 155064 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826013.345669:0:155064:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6800/0xbef42c0221d64b3f lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9634 expref: -99 pid: 152202 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826013.345713:0:155064:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e5ee lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce9848 expref: -99 pid: 155064 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826013.345721:0:155064:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826013.345724:0:155064:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e5ee lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce9848 expref: -99 pid: 155064 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:1.0:1787826013.345771:0:155064:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787826013.345788:0:155064:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c35b1c00 x1874663042849280/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 552/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826013.345812:0:155064:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155064:1874663042849280:192.168.203.147@tcp:36:lfs.0 00000100:00100000:1.0:1787826013.345892:0:155064:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8180 going to sleep for 11 seconds 00000100:00100000:1.0:1787826013.387706:0:155069:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c49e5500 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155069:1874663042849408:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:1.0:1787826013.387765:0:155069:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6c00/0xbef42c0221d6e5f5 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce988e expref: -99 pid: 155069 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826013.387793:0:155069:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6c00/0xbef42c0221d6e5f5 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1b8f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce988e expref: -99 pid: 155069 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826013.387804:0:155069:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6c00/0xbef42c0221d6e5f5 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1b8f:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce988e expref: -99 pid: 155069 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826013.387818:0:155069:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6c00/0xbef42c0221d6e5f5 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1b8f:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce988e expref: -99 pid: 155069 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826013.387830:0:155069:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6c00/0xbef42c0221d6e5f5 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1b8f:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce988e expref: -99 pid: 155069 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826013.387846:0:155069:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6c00/0xbef42c0221d6e5f5 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1b8f:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce988e expref: -99 pid: 155069 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826013.387854:0:155069:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6c00/0xbef42c0221d6e5f5 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1b8f:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce988e expref: -99 pid: 155069 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826013.387862:0:155069:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6c00/0xbef42c0221d6e5f5 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1b8f:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce988e expref: -99 pid: 155069 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826013.387876:0:155069:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c49e5500 x1874663042849408/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/672 e 0 to 0 dl 1787826029 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826013.387893:0:155069:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6c00/0xbef42c0221d6e5f5 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1b8f:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce988e expref: -99 pid: 155069 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826013.387904:0:155069:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/7055 (0 0) 00000080:00200000:1.0:1787826013.387912:0:155069:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00000002:00010000:1.0:1787826013.388264:0:155069:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200026560:0x3:0x0]) in obj [0x200026560:0x3:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826013.388326:0:155069:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6e5fc lrc: 3/1,0 mode: --/CR res: [0x200026560:0x3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 155069 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826013.388336:0:155069:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6e5fc lrc: 3/1,0 mode: --/CR res: [0x200026560:0x3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155069 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826013.388346:0:155069:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6e5fc lrc: 3/1,0 mode: --/CR res: [0x200026560:0x3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155069 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826013.388374:0:155069:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155069:1874663042849536:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:1.0:1787826013.388607:0:155069:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c83c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826013.392245:0:155069:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155069:1874663042849536:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:0.0:1787826013.392280:0:155069:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6e5fc lrc: 4/1,0 mode: --/CR res: [0x200026560:0x3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce989c expref: -99 pid: 155069 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826013.392297:0:155069:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6e5fc lrc: 4/1,0 mode: --/PR res: [0x200026560:0x3:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce989c expref: -99 pid: 155069 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826013.392311:0:155069:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6e5fc lrc: 4/1,0 mode: PR/PR res: [0x200026560:0x3:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce989c expref: -99 pid: 155069 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826013.392324:0:155069:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6e5fc lrc: 4/1,0 mode: PR/PR res: [0x200026560:0x3:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce989c expref: -99 pid: 155069 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826013.392346:0:155069:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6e5fc lrc: 5/2,0 mode: PR/PR res: [0x200026560:0x3:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce989c expref: -99 pid: 155069 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826013.392354:0:155069:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6e5fc lrc: 5/2,0 mode: PR/PR res: [0x200026560:0x3:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce989c expref: -99 pid: 155069 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826013.392362:0:155069:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6e5fc lrc: 4/1,0 mode: PR/PR res: [0x200026560:0x3:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce989c expref: -99 pid: 155069 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826013.392374:0:155069:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c35b0700 x1874663042849536/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787826029 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787826013.392390:0:155069:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6e5fc lrc: 3/1,0 mode: PR/PR res: [0x200026560:0x3:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce989c expref: -99 pid: 155069 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826013.392405:0:155069:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590091616/3 (0 0) 00010000:00010000:0.0:1787826013.392413:0:155069:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6e5fc lrc: 3/1,0 mode: PR/PR res: [0x200026560:0x3:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce989c expref: -99 pid: 155069 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826013.392422:0:155069:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6e5fc lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x3:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce989c expref: -99 pid: 155069 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000002:00010000:0.0:1787826013.392457:0:155069:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001f58:0x3:0x0]) in obj [0x240001f58:0x3:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826013.392516:0:155069:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e603 lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 155069 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826013.392523:0:155069:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e603 lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155069 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826013.392534:0:155069:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e603 lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155069 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826013.392557:0:155069:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4000 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155069:1874663042849664:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787826013.393031:0:155069:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bb3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826013.409957:0:155069:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4000 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155069:1874663042849664:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:0.0:1787826013.409996:0:155069:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e603 lrc: 4/1,0 mode: --/CR res: [0x240001f58:0x3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce98aa expref: -99 pid: 155069 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826013.410011:0:155069:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e603 lrc: 4/1,0 mode: --/PR res: [0x240001f58:0x3:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce98aa expref: -99 pid: 155069 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826013.410025:0:155069:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e603 lrc: 4/1,0 mode: PR/PR res: [0x240001f58:0x3:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce98aa expref: -99 pid: 155069 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826013.410038:0:155069:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e603 lrc: 4/1,0 mode: PR/PR res: [0x240001f58:0x3:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce98aa expref: -99 pid: 155069 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826013.410054:0:155069:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e603 lrc: 5/2,0 mode: PR/PR res: [0x240001f58:0x3:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce98aa expref: -99 pid: 155069 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826013.410064:0:155069:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e603 lrc: 5/2,0 mode: PR/PR res: [0x240001f58:0x3:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce98aa expref: -99 pid: 155069 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826013.410073:0:155069:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e603 lrc: 4/1,0 mode: PR/PR res: [0x240001f58:0x3:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce98aa expref: -99 pid: 155069 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826013.410112:0:155069:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c55a4000 x1874663042849664/t0(0) o101->lustre-MDT0001-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787826029 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787826013.410130:0:155069:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e603 lrc: 3/1,0 mode: PR/PR res: [0x240001f58:0x3:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce98aa expref: -99 pid: 155069 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826013.410146:0:155069:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d0f54800 type 13 mode 4 res 9663684440/3 (0 0) 00010000:00010000:0.0:1787826013.410156:0:155069:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e603 lrc: 3/1,0 mode: PR/PR res: [0x240001f58:0x3:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce98aa expref: -99 pid: 155069 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826013.410166:0:155069:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e603 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x3:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce98aa expref: -99 pid: 155069 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826013.410213:0:155069:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1b8f:0x0], updating i_size 8192 00000080:00200000:0.0:1787826013.410218:0:155069:0:(namei.c:147:ll_iget()) got inode: [0x20001a9e3:0x1b8f:0x0](ffff9181c83b42c8): rc = 0 00000080:00010000:0.0:1787826013.410231:0:155069:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x1b8f:0x0](ffff9181c83b42c8) for lock 0xbef42c0221d6e5f5 00000080:00010000:0.0:1787826013.410250:0:155069:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x1b8f:0x0](ffff9181c83b42c8) 00000080:00010000:0.0:1787826013.410254:0:155069:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6e5f5 from it ffffb97501e8be08 00010000:00010000:0.0:1787826013.410259:0:155069:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6c00/0xbef42c0221d6e5f5 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1b8f:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce988e expref: -99 pid: 155069 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826013.410269:0:155069:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6c00/0xbef42c0221d6e5f5 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1b8f:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce988e expref: -99 pid: 155069 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826013.410299:0:155069:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 3 00000080:00200000:0.0:1787826013.410309:0:155069:0:(namei.c:2328:ll_rmdir()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00010000:00010000:0.0:1787826013.410348:0:155069:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6c00/0xbef42c0221d6e5f5 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1b8f:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ce988e expref: -99 pid: 155069 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826013.410413:0:155069:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6c00/0xbef42c0221d6e5f5 lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x1b8f:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce988e expref: -99 pid: 155069 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826013.410421:0:155069:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826013.410424:0:155069:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6c00/0xbef42c0221d6e5f5 lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x1b8f:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce988e expref: -99 pid: 155069 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:0.0:1787826013.410454:0:155069:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787826013.410465:0:155069:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c55a4000 x1874663042849920/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826013.410637:0:155069:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4000 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155069:1874663042849920:192.168.203.147@tcp:36:rmdir.0 00000100:00100000:0.0:1787826013.410993:0:155069:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bb3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826013.413778:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826013.413835:0:15151:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:1.0:1787826013.413857:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048998528 00000100:00100000:1.0:1787826013.413868:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048998528 00000100:00100000:1.0:1787826013.413927:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9139 00000100:00100000:1.0:1787826013.413938:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c440ac50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187905:x1874663048998528:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826013.413975:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6e5fc lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x3:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce989c expref: -99 pid: 155069 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826013.414060:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826013.414068:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c440ac50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187905:x1874663048998528:12345-192.168.203.147@tcp:104: Request processed in 132us (304us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826013.414077:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9139 00000100:00100000:1.0:1787826013.419969:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826013.420011:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663048998784 00000100:00100000:1.0:1787826013.420023:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663048998784 00000100:00100000:1.0:1787826013.420098:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9140 00000100:00100000:1.0:1787826013.420111:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b3100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8414:x1874663048998784:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826013.420139:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e603 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x3:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce98aa expref: -99 pid: 155069 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826013.420238:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826013.420247:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b3100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8414:x1874663048998784:12345-192.168.203.147@tcp:104: Request processed in 139us (288us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826013.420256:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9140 00000100:00100000:1.0:1787826013.448460:0:155069:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4000 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155069:1874663042849920:192.168.203.147@tcp:36:rmdir.0 00000080:00200000:1.0:1787826013.448581:0:155069:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1b8f:0x0](ffff9181c83b42c8) 00000100:00100000:1.0:1787826013.501291:0:155084:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c49e5880 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155084:1874663042850560:192.168.203.147@tcp:101:lfs.0 00010000:00010000:1.0:1787826013.501386:0:155084:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6600/0xbef42c0221d6e618 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce990c expref: -99 pid: 155084 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826013.501406:0:155084:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6600/0xbef42c0221d6e618 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce990c expref: -99 pid: 155084 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826013.501415:0:155084:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6600/0xbef42c0221d6e618 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce990c expref: -99 pid: 155084 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826013.501430:0:155084:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6600/0xbef42c0221d6e618 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce990c expref: -99 pid: 155084 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826013.501443:0:155084:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6600/0xbef42c0221d6e618 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce990c expref: -99 pid: 155084 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826013.501458:0:155084:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6600/0xbef42c0221d6e618 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce990c expref: -99 pid: 155084 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826013.501468:0:155084:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6600/0xbef42c0221d6e618 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce990c expref: -99 pid: 155084 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826013.501475:0:155084:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6600/0xbef42c0221d6e618 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce990c expref: -99 pid: 155084 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826013.501489:0:155084:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c49e5880 x1874663042850560/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826029 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826013.501506:0:155084:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6600/0xbef42c0221d6e618 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce990c expref: -99 pid: 155084 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826013.501518:0:155084:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/6112 (0 0) 00000080:00200000:1.0:1787826013.501526:0:155084:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787826013.501558:0:155084:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x17e0:0x0], updating i_size 4096 00000080:00010000:1.0:1787826013.501565:0:155084:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) for lock 0xbef42c0221d6e618 00000080:00010000:1.0:1787826013.501573:0:155084:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00000080:00010000:1.0:1787826013.501577:0:155084:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6e618 from it ffffb97502b13cb8 00010000:00010000:1.0:1787826013.501580:0:155084:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6600/0xbef42c0221d6e618 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce990c expref: -99 pid: 155084 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826013.501590:0:155084:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6600/0xbef42c0221d6e618 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce990c expref: -99 pid: 155084 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:1.0:1787826013.502497:0:155084:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), request_mask 0, flags 0, foreign 0 00000080:00200000:1.0:1787826013.502507:0:155084:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826013.502530:0:155084:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826013.502544:0:155084:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826013.502558:0:155084:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826013.502565:0:155084:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826013.502571:0:155084:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826013.502575:0:155084:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787826013.502579:0:155084:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97502b13cb8 00010000:00010000:1.0:1787826013.502583:0:155084:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826013.502592:0:155084:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826013.502740:0:155084:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826013.502866:0:155084:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826013.502874:0:155084:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826013.502884:0:155084:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826013.502890:0:155084:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826013.502895:0:155084:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826013.502898:0:155084:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787826013.502902:0:155084:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97502b13bf8 00010000:00010000:1.0:1787826013.502906:0:155084:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826013.502914:0:155084:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826013.502925:0:155084:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826013.502933:0:155084:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=267 00000080:00200000:1.0:1787826013.502940:0:155084:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 44 00000080:00200008:1.0:1787826013.502946:0:155084:0:(file.c:1090:ll_file_open()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00000080:00200080:1.0:1787826013.502973:0:155084:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) cmd=c03066f0 arg=7ffc989a8690 00000080:00200000:1.0:1787826013.503104:0:155084:0:(dir.c:451:ll_dir_setdirstripe()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) name=test stripe_offset=0 stripe_count=2, hash_type=0 00010000:00010000:1.0:1787826013.503139:0:155084:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6600/0xbef42c0221d6e618 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ce990c expref: -99 pid: 155084 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826013.503156:0:155084:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6800/0xbef42c0221d64b3f lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9634 expref: -99 pid: 152202 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826013.503207:0:155084:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6600/0xbef42c0221d6e618 lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce990c expref: -99 pid: 155084 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826013.503216:0:155084:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826013.503219:0:155084:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6600/0xbef42c0221d6e618 lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce990c expref: -99 pid: 155084 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:1.0:1787826013.503239:0:155084:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787826013.503249:0:155084:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c35b1f80 x1874663042850816/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 552/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826013.503280:0:155084:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155084:1874663042850816:192.168.203.147@tcp:36:lfs.0 00000100:00100000:1.0:1787826013.503373:0:155084:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8180 going to sleep for 11 seconds 00000100:00100000:1.0:1787826013.517694:0:155084:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155084:1874663042850816:192.168.203.147@tcp:36:lfs.0 00000080:00200000:1.0:1787826013.517795:0:155084:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x20001a9e3:0x1b90:0x0](ffff9181c839a1c8) 00000080:00200000:1.0:1787826013.517805:0:155084:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1b90:0x0], updating i_size 4096 00000080:00200008:1.0:1787826013.517843:0:155084:0:(file.c:479:ll_file_release()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00010000:00010000:1.0:1787826013.517881:0:155084:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513000/0xbef42c0221d64bee lrc: 2/0,0 mode: CR/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce962d expref: -99 pid: 152210 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000100:00100000:1.0:1787826013.581777:0:155092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c494c000 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155092:1874663042851200:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:1.0:1787826013.581811:0:155092:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6e62d lrc: 4/1,0 mode: --/CR res: [0x240001f58:0x4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce996e expref: -99 pid: 155092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826013.581825:0:155092:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6e62d lrc: 4/1,0 mode: --/PR res: [0x240001f58:0x4:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce996e expref: -99 pid: 155092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826013.581839:0:155092:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6e62d lrc: 4/1,0 mode: PR/PR res: [0x240001f58:0x4:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce996e expref: -99 pid: 155092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826013.581853:0:155092:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6e62d lrc: 4/1,0 mode: PR/PR res: [0x240001f58:0x4:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce996e expref: -99 pid: 155092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826013.581868:0:155092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6e62d lrc: 5/2,0 mode: PR/PR res: [0x240001f58:0x4:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce996e expref: -99 pid: 155092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826013.581877:0:155092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6e62d lrc: 5/2,0 mode: PR/PR res: [0x240001f58:0x4:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce996e expref: -99 pid: 155092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826013.581886:0:155092:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6e62d lrc: 4/1,0 mode: PR/PR res: [0x240001f58:0x4:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce996e expref: -99 pid: 155092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826013.581899:0:155092:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c494c000 x1874663042851200/t0(0) o101->lustre-MDT0001-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787826029 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826013.581915:0:155092:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6e62d lrc: 3/1,0 mode: PR/PR res: [0x240001f58:0x4:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce996e expref: -99 pid: 155092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826013.581931:0:155092:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d0f54800 type 13 mode 4 res 9663684440/4 (0 0) 00010000:00010000:1.0:1787826013.581940:0:155092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6e62d lrc: 3/1,0 mode: PR/PR res: [0x240001f58:0x4:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce996e expref: -99 pid: 155092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826013.581950:0:155092:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6e62d lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x4:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce996e expref: -99 pid: 155092 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826013.582005:0:155092:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1b90:0x0], updating i_size 8192 00000080:00200000:1.0:1787826013.582010:0:155092:0:(namei.c:147:ll_iget()) got inode: [0x20001a9e3:0x1b90:0x0](ffff9181c839a1c8): rc = 0 00000080:00010000:1.0:1787826013.582023:0:155092:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x1b90:0x0](ffff9181c839a1c8) for lock 0xbef42c0221d6e61f 00000080:00010000:1.0:1787826013.582044:0:155092:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x1b90:0x0](ffff9181c839a1c8) 00000080:00010000:1.0:1787826013.582048:0:155092:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6e61f from it ffffb9750181be08 00010000:00010000:1.0:1787826013.582053:0:155092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7200/0xbef42c0221d6e61f lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1b90:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9952 expref: -99 pid: 155092 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826013.582063:0:155092:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7200/0xbef42c0221d6e61f lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1b90:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9952 expref: -99 pid: 155092 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826013.582127:0:155092:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 3 00000080:00200000:1.0:1787826013.582138:0:155092:0:(namei.c:2328:ll_rmdir()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00010000:00010000:1.0:1787826013.582179:0:155092:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7200/0xbef42c0221d6e61f lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1b90:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ce9952 expref: -99 pid: 155092 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826013.582237:0:155092:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7200/0xbef42c0221d6e61f lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x1b90:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce9952 expref: -99 pid: 155092 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826013.582244:0:155092:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826013.582247:0:155092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7200/0xbef42c0221d6e61f lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x1b90:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce9952 expref: -99 pid: 155092 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:1.0:1787826013.582272:0:155092:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787826013.582279:0:155092:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c35b1500 x1874663042851456/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826013.582315:0:155092:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155092:1874663042851456:192.168.203.147@tcp:36:rmdir.0 00000100:00100000:1.0:1787826013.582408:0:155092:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8180 going to sleep for 11 seconds 00000100:00100000:0.0:1787826013.587674:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826013.587792:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049000192 00000100:00100000:1.0:1787826013.587805:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049000192 00000100:00100000:1.0:1787826013.587870:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9141 00000100:00100000:1.0:1787826013.587882:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a5f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187905:x1874663049000192:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826013.587905:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6600/0xbef42c0221d6e626 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x4:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce9960 expref: -99 pid: 155092 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826013.588019:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826013.588027:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a5f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187905:x1874663049000192:12345-192.168.203.147@tcp:104: Request processed in 148us (366us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826013.588035:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9141 00000100:00100000:1.0:1787826013.596229:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826013.597920:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049000448 00000100:00100000:0.0:1787826013.597941:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049000448 00000100:00100000:0.0:1787826013.598020:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9142 00000100:00100000:0.0:1787826013.598033:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663049000448:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826013.598058:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6e62d lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x4:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce996e expref: -99 pid: 155092 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826013.598247:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826013.598254:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663049000448:12345-192.168.203.147@tcp:104: Request processed in 224us (2042us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826013.598262:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9142 00010000:00010000:0.0:1787826013.598288:0:15161:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826013.598296:0:15161:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6e62d lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x4:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce996e expref: -99 pid: 155092 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826013.598407:0:15161:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3481e00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787826013.598412:0:15161:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6e62d lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x4:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00ce996e expref: -99 pid: 155092 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826013.598422:0:15161:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6e62d lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x4:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00ce996e expref: -99 pid: 155092 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826013.598432:0:15161:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6e62d lrc: 3/0,0 mode: PR/PR res: [0x240001f58:0x4:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x408400000000 nid: local remote: 0x878a5eea00ce996e expref: -99 pid: 155092 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826013.598505:0:15161:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6e62d lrc: 2/0,0 mode: --/PR res: [0x240001f58:0x4:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ce996e expref: -99 pid: 155092 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826013.598513:0:15161:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826013.598549:0:15161:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6e62d lrc: 1/0,0 mode: --/PR res: [0x240001f58:0x4:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ce996e expref: -99 pid: 155092 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826013.598557:0:15161:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6e62d lrc: 0/0,0 mode: --/PR res: [0x240001f58:0x4:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ce996e expref: -99 pid: 155092 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826013.598579:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826013.598581:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826013.598583:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826013.598592:0:2389:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1787826013.598610:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042851712:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1787826013.618068:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042851712:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1787826013.623376:0:155092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155092:1874663042851456:192.168.203.147@tcp:36:rmdir.0 00000080:00200000:0.0:1787826013.623439:0:155092:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1b90:0x0](ffff9181c839a1c8) 00000080:00200008:0.0:1787826013.694737:0:155103:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1787826013.694765:0:155103:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826013.694793:0:155103:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826013.694809:0:155103:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826013.694825:0:155103:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826013.694834:0:155103:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826013.694843:0:155103:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826013.694846:0:155103:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826013.694853:0:155103:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97500c97cb8 00010000:00010000:0.0:1787826013.694856:0:155103:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826013.694866:0:155103:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826013.695033:0:155103:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826013.695043:0:155103:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826013.695051:0:155103:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826013.695059:0:155103:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826013.695066:0:155103:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826013.695071:0:155103:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826013.695073:0:155103:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826013.695076:0:155103:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97500c97bf8 00010000:00010000:0.0:1787826013.695090:0:155103:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826013.695099:0:155103:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826013.695107:0:155103:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826013.695115:0:155103:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=267 00000080:00200000:0.0:1787826013.695122:0:155103:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 44 00000080:00200008:0.0:1787826013.695129:0:155103:0:(file.c:1090:ll_file_open()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00000080:00200080:0.0:1787826013.695155:0:155103:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) cmd=c03066f0 arg=7ffc7540f160 00000080:00200000:0.0:1787826013.695173:0:155103:0:(dir.c:451:ll_dir_setdirstripe()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) name=test stripe_offset=0 stripe_count=2, hash_type=0 00010000:00010000:0.0:1787826013.695210:0:155103:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481c00/0xbef42c0221d6e642 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ce99d0 expref: -99 pid: 155103 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826013.695339:0:155103:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6800/0xbef42c0221d64b3f lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9634 expref: -99 pid: 152202 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826013.695395:0:155103:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481c00/0xbef42c0221d6e642 lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce99d0 expref: -99 pid: 155103 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826013.695402:0:155103:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826013.695406:0:155103:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481c00/0xbef42c0221d6e642 lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce99d0 expref: -99 pid: 155103 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:0.0:1787826013.695428:0:155103:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787826013.695441:0:155103:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c55a6d80 x1874663042852352/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 552/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826013.695471:0:155103:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155103:1874663042852352:192.168.203.147@tcp:36:lfs.0 00000100:00100000:0.0:1787826013.695582:0:155103:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bb180 going to sleep for 11 seconds 00000100:00100000:1.0:1787826013.729612:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7528148b-9bee-4d31-b597-8aae3b0b4982:2389:1874663042852608:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826013.730003:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042852864:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826013.730041:0:2389:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1787826013.730046:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042852736:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826013.741913:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042852736:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826013.741930:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7528148b-9bee-4d31-b597-8aae3b0b4982:2389:1874663042852608:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826013.741940:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042852864:192.168.203.147@tcp:400:kworker.0 00000002:00010000:0.0:1787826013.765185:0:155104:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001f58:0x5:0x0]) in obj [0x240001f58:0x5:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826013.765258:0:155104:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6e657 lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 155104 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826013.765269:0:155104:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6e657 lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155104 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826013.765279:0:155104:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6e657 lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155104 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826013.765309:0:155104:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155104:1874663042853120:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787826013.765414:0:155104:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bb0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826013.773598:0:155104:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155104:1874663042853120:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:0.0:1787826013.773638:0:155104:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6e657 lrc: 4/1,0 mode: --/CR res: [0x240001f58:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9a32 expref: -99 pid: 155104 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826013.773656:0:155104:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6e657 lrc: 4/1,0 mode: --/PR res: [0x240001f58:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9a32 expref: -99 pid: 155104 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826013.773671:0:155104:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6e657 lrc: 4/1,0 mode: PR/PR res: [0x240001f58:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9a32 expref: -99 pid: 155104 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826013.773684:0:155104:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6e657 lrc: 4/1,0 mode: PR/PR res: [0x240001f58:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9a32 expref: -99 pid: 155104 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826013.773700:0:155104:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6e657 lrc: 5/2,0 mode: PR/PR res: [0x240001f58:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9a32 expref: -99 pid: 155104 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826013.773709:0:155104:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6e657 lrc: 5/2,0 mode: PR/PR res: [0x240001f58:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9a32 expref: -99 pid: 155104 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826013.773717:0:155104:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6e657 lrc: 4/1,0 mode: PR/PR res: [0x240001f58:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9a32 expref: -99 pid: 155104 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826013.773730:0:155104:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c55a5c00 x1874663042853120/t0(0) o101->lustre-MDT0001-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787826029 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787826013.774118:0:155104:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6e657 lrc: 3/1,0 mode: PR/PR res: [0x240001f58:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9a32 expref: -99 pid: 155104 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826013.774138:0:155104:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d0f54800 type 13 mode 4 res 9663684440/5 (0 0) 00010000:00010000:0.0:1787826013.774149:0:155104:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6e657 lrc: 3/1,0 mode: PR/PR res: [0x240001f58:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9a32 expref: -99 pid: 155104 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826013.774163:0:155104:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6e657 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9a32 expref: -99 pid: 155104 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826013.774207:0:155104:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1b91:0x0], updating i_size 8192 00000080:00200000:0.0:1787826013.774212:0:155104:0:(namei.c:147:ll_iget()) got inode: [0x20001a9e3:0x1b91:0x0](ffff9181c83cb248): rc = 0 00000080:00010000:0.0:1787826013.774224:0:155104:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x1b91:0x0](ffff9181c83cb248) for lock 0xbef42c0221d6e649 00000080:00010000:0.0:1787826013.774248:0:155104:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x1b91:0x0](ffff9181c83cb248) 00000080:00010000:0.0:1787826013.774252:0:155104:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6e649 from it ffffb97500c97e08 00010000:00010000:0.0:1787826013.774256:0:155104:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6e649 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1b91:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9a16 expref: -99 pid: 155104 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826013.774266:0:155104:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6e649 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1b91:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9a16 expref: -99 pid: 155104 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826013.774293:0:155104:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 3 00000080:00200000:0.0:1787826013.774303:0:155104:0:(namei.c:2328:ll_rmdir()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00010000:00010000:0.0:1787826013.774341:0:155104:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6e649 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1b91:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ce9a16 expref: -99 pid: 155104 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826013.774402:0:155104:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6e649 lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x1b91:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce9a16 expref: -99 pid: 155104 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826013.774410:0:155104:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826013.774413:0:155104:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6e649 lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x1b91:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce9a16 expref: -99 pid: 155104 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:0.0:1787826013.774455:0:155104:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787826013.774465:0:155104:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c55a5c00 x1874663042853376/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826013.774491:0:155104:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155104:1874663042853376:192.168.203.147@tcp:36:rmdir.0 00000100:00100000:0.0:1787826013.774580:0:155104:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bb0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826013.776856:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826013.776903:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049001856 00000100:00100000:1.0:1787826013.776916:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049001856 00000100:00100000:1.0:1787826013.776976:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9143 00000100:00100000:1.0:1787826013.776987:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187905:x1874663049001856:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826013.777009:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7a00/0xbef42c0221d6e650 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce9a24 expref: -99 pid: 155104 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826013.777103:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826013.777110:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187905:x1874663049001856:12345-192.168.203.147@tcp:104: Request processed in 125us (268us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826013.777118:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9143 00000100:00100000:1.0:1787826013.782939:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826013.782990:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049002112 00000100:00100000:1.0:1787826013.783003:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049002112 00000100:00100000:1.0:1787826013.783066:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9144 00000100:00100000:1.0:1787826013.783090:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b0e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663049002112:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826013.783117:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6e657 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x5:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce9a32 expref: -99 pid: 155104 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826013.783205:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826013.783212:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b0e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663049002112:12345-192.168.203.147@tcp:104: Request processed in 137us (289us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826013.783219:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9144 00000080:00200000:0.0:1787826013.822217:0:155111:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826013.822263:0:155111:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826013.822287:0:155111:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826013.822308:0:155111:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826013.822318:0:155111:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826013.822328:0:155111:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826013.822332:0:155111:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826013.822342:0:155111:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501c23b58 00010000:00010000:0.0:1787826013.822346:0:155111:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826013.822356:0:155111:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826013.822372:0:155111:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826013.822381:0:155111:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=29 00000080:00200000:0.0:1787826013.822389:0:155111:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826013.822402:0:155111:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=13 00000080:00200000:0.0:1787826013.822407:0:155111:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787826013.822423:0:155111:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787826013.822477:0:155111:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6e65e lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155111 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826013.822487:0:155111:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6e65e lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155111 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826013.822498:0:155111:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6e65e lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155111 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826013.822528:0:155111:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4a80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155111:1874663042853760:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787826013.822815:0:155111:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bb000 going to sleep for 11 seconds 00000100:00100000:0.0:1787826013.934801:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826013.934849:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049003520 00000100:00100000:0.0:1787826013.934862:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049003520 00000100:00100000:0.0:1787826013.934928:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9145 00000100:00100000:0.0:1787826013.935061:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a5180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187905:x1874663049003520:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826013.935095:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7000/0xbef42c0221d6e67a lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x6:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce9ae8 expref: -99 pid: 155112 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826013.935270:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826013.935279:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a5180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187905:x1874663049003520:12345-192.168.203.147@tcp:104: Request processed in 224us (493us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826013.935288:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9145 00000100:00100000:0.0:1787826013.953643:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826013.953715:0:33333:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:0.0:1787826013.953739:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049003776 00000100:00100000:0.0:1787826013.953765:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049003776 00000100:00100000:0.0:1787826013.953829:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9146 00000100:00100000:0.0:1787826013.953842:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181e053d450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663049003776:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826013.953865:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6e681 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x6:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce9af6 expref: -99 pid: 155112 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826013.953957:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826013.953964:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181e053d450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663049003776:12345-192.168.203.147@tcp:104: Request processed in 125us (338us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826013.953973:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9146 00010000:00010000:0.0:1787826013.954021:0:15161:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826013.954030:0:15161:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6e681 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x6:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce9af6 expref: -99 pid: 155112 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826013.954040:0:15161:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e06f6000 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787826013.954046:0:15161:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6e681 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x6:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00ce9af6 expref: -99 pid: 155112 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826013.954055:0:15161:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6e681 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x6:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00ce9af6 expref: -99 pid: 155112 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826013.954065:0:15161:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6e681 lrc: 3/0,0 mode: PR/PR res: [0x240001f58:0x6:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x408400000000 nid: local remote: 0x878a5eea00ce9af6 expref: -99 pid: 155112 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826013.954160:0:15161:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6e681 lrc: 2/0,0 mode: --/PR res: [0x240001f58:0x6:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ce9af6 expref: -99 pid: 155112 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826013.954168:0:15161:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826013.954195:0:15161:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6e681 lrc: 1/0,0 mode: --/PR res: [0x240001f58:0x6:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ce9af6 expref: -99 pid: 155112 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826013.954209:0:15161:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6e681 lrc: 0/0,0 mode: --/PR res: [0x240001f58:0x6:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ce9af6 expref: -99 pid: 155112 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826013.954234:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826013.954236:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826013.954238:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826013.954255:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042855168:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787826013.963325:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042855168:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1787826013.989133:0:155112:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c49e7b80 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155112:1874663042854912:192.168.203.147@tcp:36:rmdir.0 00000080:00200000:0.0:1787826013.989200:0:155112:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1b92:0x0](ffff9181c83cb248) 00000080:00200000:1.0:1787826014.021965:0:155113:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826014.022008:0:155113:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826014.022033:0:155113:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826014.022054:0:155113:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826014.022063:0:155113:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826014.022075:0:155113:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826014.022090:0:155113:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787826014.022100:0:155113:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb975019ebb58 00010000:00010000:1.0:1787826014.022104:0:155113:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826014.022114:0:155113:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826014.022130:0:155113:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826014.022140:0:155113:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=29 00000080:00200000:1.0:1787826014.022147:0:155113:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826014.022160:0:155113:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=13 00000080:00200000:1.0:1787826014.022165:0:155113:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:1.0:1787826014.022179:0:155113:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787826014.022235:0:155113:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e688 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155113 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826014.022244:0:155113:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e688 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155113 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826014.022255:0:155113:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e688 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155113 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826014.022284:0:155113:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155113:1874663042855296:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787826014.023357:0:155113:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8000 going to sleep for 11 seconds 00000100:00100000:0.0:1787826014.033322:0:155113:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155113:1874663042855296:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787826014.033356:0:155113:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e688 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155113 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.033375:0:155113:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e688 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155113 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.033385:0:155113:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e688 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155113 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.033392:0:155113:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e688 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155113 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.033404:0:155113:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e688 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155113 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.033412:0:155113:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b5c00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787826014.033418:0:155113:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e688 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155113 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.033426:0:155113:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e688 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155113 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.033435:0:155113:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e688 lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 155113 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.033443:0:155113:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787826014.033452:0:155113:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e688 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155113 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.033460:0:155113:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e688 lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155113 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826014.033481:0:155113:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c35b3100 x1874663042855296/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787826030 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787826014.033499:0:155113:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787826014.034307:0:155113:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787826014.034411:0:155113:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826014.034434:0:155113:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826014.034445:0:155113:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826014.034460:0:155113:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826014.034469:0:155113:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826014.034476:0:155113:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826014.034481:0:155113:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826014.034488:0:155113:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb975019ebbe8 00010000:00010000:0.0:1787826014.034492:0:155113:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826014.034502:0:155113:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826014.034517:0:155113:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826014.034537:0:155113:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=16408 00000080:00200000:0.0:1787826014.034545:0:155113:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826014.034561:0:155113:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=16392 00000080:00200000:0.0:1787826014.034566:0:155113:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787826014.034580:0:155113:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787826014.034632:0:155113:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6e68f lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155113 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.034641:0:155113:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6e68f lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155113 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.034653:0:155113:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6e68f lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155113 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826014.034681:0:155113:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155113:1874663042855424:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787826014.034817:0:155113:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bbd80 going to sleep for 11 seconds 00000080:00200000:1.0:1787826014.060977:0:155113:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x20001a9e3:0x1b93:0x0](ffff9181c8399988) 00000080:00200000:1.0:1787826014.060999:0:155113:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1b93:0x0], updating i_size 4096 00000080:00200008:1.0:1787826014.061058:0:155113:0:(file.c:479:ll_file_release()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00010000:00010000:1.0:1787826014.061120:0:155113:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513000/0xbef42c0221d64bee lrc: 2/0,0 mode: CR/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce962d expref: -99 pid: 152210 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826014.074292:0:155118:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826014.074332:0:155118:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826014.074356:0:155118:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826014.074374:0:155118:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826014.074390:0:155118:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826014.074401:0:155118:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826014.074406:0:155118:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787826014.074415:0:155118:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb975019ebc20 00010000:00010000:1.0:1787826014.074420:0:155118:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826014.074430:0:155118:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826014.074446:0:155118:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826014.074456:0:155118:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=24 00000080:00200000:1.0:1787826014.074463:0:155118:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826014.074478:0:155118:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=0 00000080:00200000:1.0:1787826014.074483:0:155118:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:1.0:1787826014.074497:0:155118:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787826014.074902:0:155118:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6e69d lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155118 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826014.074912:0:155118:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6e69d lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155118 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826014.074923:0:155118:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6e69d lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155118 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826014.074956:0:155118:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155118:1874663042855936:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:1.0:1787826014.075124:0:155118:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8b40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826014.114940:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826014.115045:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049005184 00000100:00100000:0.0:1787826014.115060:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049005184 00000100:00100000:0.0:1787826014.115141:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9147 00000100:00100000:0.0:1787826014.115154:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187905:x1874663049005184:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826014.115180:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480c00/0xbef42c0221d6e6a4 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x7:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce9bac expref: -99 pid: 155118 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826014.115270:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826014.115277:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187905:x1874663049005184:12345-192.168.203.147@tcp:104: Request processed in 126us (357us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826014.115285:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9147 00000100:00100000:1.0:1787826014.135642:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826014.135688:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049005440 00000100:00100000:1.0:1787826014.135699:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049005440 00000100:00100000:1.0:1787826014.136129:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9148 00000100:00100000:1.0:1787826014.136142:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b0e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8414:x1874663049005440:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826014.136165:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481600/0xbef42c0221d6e6ab lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x7:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce9bba expref: -99 pid: 155118 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826014.136300:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826014.136308:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b0e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8414:x1874663049005440:12345-192.168.203.147@tcp:104: Request processed in 168us (682us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826014.136316:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9148 00000080:00200000:0.0:1787826014.198301:0:155131:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826014.198347:0:155131:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826014.198374:0:155131:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826014.198392:0:155131:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826014.198401:0:155131:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826014.198412:0:155131:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826014.198416:0:155131:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826014.198425:0:155131:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb975019ebb58 00010000:00010000:0.0:1787826014.198430:0:155131:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826014.198440:0:155131:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826014.198455:0:155131:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826014.198464:0:155131:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=29 00000080:00200000:0.0:1787826014.198471:0:155131:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826014.198483:0:155131:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=13 00000080:00200000:0.0:1787826014.198488:0:155131:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787826014.198501:0:155131:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787826014.198556:0:155131:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6e6b2 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155131 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.198564:0:155131:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6e6b2 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155131 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.198575:0:155131:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6e6b2 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155131 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826014.198604:0:155131:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155131:1874663042856832:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787826014.198714:0:155131:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bb240 going to sleep for 11 seconds 00000100:00100000:0.0:1787826014.208974:0:155131:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155131:1874663042856832:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787826014.209007:0:155131:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6e6b2 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155131 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.209025:0:155131:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6e6b2 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155131 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.209034:0:155131:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6e6b2 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155131 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.209041:0:155131:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6e6b2 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155131 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.209052:0:155131:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6e6b2 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155131 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.209064:0:155131:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0513400 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787826014.209069:0:155131:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6e6b2 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155131 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.209087:0:155131:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6e6b2 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155131 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.209095:0:155131:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6e6b2 lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 155131 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.209102:0:155131:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787826014.209110:0:155131:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6e6b2 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155131 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.209116:0:155131:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6e6b2 lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155131 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826014.211518:0:155131:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c55a7480 x1874663042856832/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787826030 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787826014.211543:0:155131:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787826014.211856:0:155131:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787826014.211960:0:155131:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826014.211980:0:155131:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826014.211994:0:155131:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826014.212009:0:155131:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826014.212021:0:155131:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826014.212027:0:155131:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826014.212031:0:155131:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826014.212039:0:155131:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb975019ebbe8 00010000:00010000:0.0:1787826014.212043:0:155131:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826014.212053:0:155131:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826014.212068:0:155131:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826014.212087:0:155131:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=16408 00000080:00200000:0.0:1787826014.212097:0:155131:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826014.212111:0:155131:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=16392 00000080:00200000:0.0:1787826014.212116:0:155131:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787826014.213234:0:155131:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787826014.213305:0:155131:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e6b9 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155131 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.213316:0:155131:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e6b9 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155131 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.213327:0:155131:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e6b9 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155131 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826014.213360:0:155131:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155131:1874663042856960:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787826014.213481:0:155131:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bb240 going to sleep for 11 seconds 00000100:00100000:0.0:1787826014.235706:0:155131:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c49e4a80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155131:1874663042857088:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787826014.235776:0:155131:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7000/0xbef42c0221d6e6c0 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9c1c expref: -99 pid: 155131 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.235800:0:155131:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7000/0xbef42c0221d6e6c0 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9c1c expref: -99 pid: 155131 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.235809:0:155131:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7000/0xbef42c0221d6e6c0 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9c1c expref: -99 pid: 155131 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.235823:0:155131:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7000/0xbef42c0221d6e6c0 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9c1c expref: -99 pid: 155131 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.235835:0:155131:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7000/0xbef42c0221d6e6c0 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9c1c expref: -99 pid: 155131 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.235849:0:155131:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7000/0xbef42c0221d6e6c0 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9c1c expref: -99 pid: 155131 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.235857:0:155131:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7000/0xbef42c0221d6e6c0 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9c1c expref: -99 pid: 155131 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.235865:0:155131:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7000/0xbef42c0221d6e6c0 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9c1c expref: -99 pid: 155131 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826014.235878:0:155131:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c49e4a80 x1874663042857088/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826030 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787826014.235893:0:155131:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7000/0xbef42c0221d6e6c0 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9c1c expref: -99 pid: 155131 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.235904:0:155131:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787826014.235911:0:155131:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1787826014.235957:0:155131:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x17e0:0x0], updating i_size 4096 00000080:00010000:0.0:1787826014.235965:0:155131:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) for lock 0xbef42c0221d6e6c0 00000080:00010000:0.0:1787826014.235974:0:155131:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00000080:00010000:0.0:1787826014.235977:0:155131:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6e6c0 from it ffffb975019ebcb8 00010000:00010000:0.0:1787826014.235980:0:155131:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7000/0xbef42c0221d6e6c0 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9c1c expref: -99 pid: 155131 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.235990:0:155131:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7000/0xbef42c0221d6e6c0 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9c1c expref: -99 pid: 155131 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:0.0:1787826014.237072:0:155131:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1787826014.237093:0:155131:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826014.237116:0:155131:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826014.237130:0:155131:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826014.237145:0:155131:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826014.237152:0:155131:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826014.237159:0:155131:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826014.237162:0:155131:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826014.237169:0:155131:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb975019ebcb8 00010000:00010000:0.0:1787826014.237172:0:155131:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826014.237182:0:155131:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826014.237308:0:155131:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826014.237317:0:155131:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826014.237325:0:155131:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826014.238514:0:155131:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826014.238525:0:155131:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826014.238534:0:155131:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826014.238537:0:155131:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826014.238545:0:155131:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb975019ebbf8 00010000:00010000:0.0:1787826014.238549:0:155131:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826014.238557:0:155131:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826014.238571:0:155131:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826014.238582:0:155131:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=267 00000080:00200000:0.0:1787826014.238589:0:155131:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 44 00000080:00200008:0.0:1787826014.238596:0:155131:0:(file.c:1090:ll_file_open()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00000080:00200080:0.0:1787826014.238632:0:155131:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) cmd=c03066f0 arg=7ffc3c8d8740 00000080:00200000:0.0:1787826014.238649:0:155131:0:(dir.c:451:ll_dir_setdirstripe()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) name=test stripe_offset=0 stripe_count=2, hash_type=0 00010000:00010000:0.0:1787826014.238686:0:155131:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7000/0xbef42c0221d6e6c0 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ce9c1c expref: -99 pid: 155131 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826014.238704:0:155131:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6800/0xbef42c0221d64b3f lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9634 expref: -99 pid: 152202 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826014.239069:0:155131:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7000/0xbef42c0221d6e6c0 lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce9c1c expref: -99 pid: 155131 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826014.239089:0:155131:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826014.239092:0:155131:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7000/0xbef42c0221d6e6c0 lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce9c1c expref: -99 pid: 155131 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:0.0:1787826014.239118:0:155131:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787826014.239133:0:155131:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c55a7480 x1874663042857344/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 552/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826014.239163:0:155131:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155131:1874663042857344:192.168.203.147@tcp:36:lfs.0 00000100:00100000:0.0:1787826014.239275:0:155131:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bb480 going to sleep for 11 seconds 00000100:00100000:0.0:1787826014.247439:0:155131:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155131:1874663042857344:192.168.203.147@tcp:36:lfs.0 00000080:00200000:0.0:1787826014.247500:0:155131:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x20001a9e3:0x1b94:0x0](ffff9181c83a1988) 00000080:00200000:0.0:1787826014.247509:0:155131:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1b94:0x0], updating i_size 4096 00000080:00200008:0.0:1787826014.247546:0:155131:0:(file.c:479:ll_file_release()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00010000:00010000:0.0:1787826014.247583:0:155131:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513000/0xbef42c0221d64bee lrc: 2/0,0 mode: CR/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce962d expref: -99 pid: 152210 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826014.259494:0:155136:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826014.259543:0:155136:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826014.259568:0:155136:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826014.259588:0:155136:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826014.259596:0:155136:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826014.259605:0:155136:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826014.259609:0:155136:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826014.259619:0:155136:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501c6bc20 00010000:00010000:0.0:1787826014.259624:0:155136:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826014.259634:0:155136:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826014.259651:0:155136:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826014.259660:0:155136:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=24 00000080:00200000:0.0:1787826014.259667:0:155136:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826014.259680:0:155136:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=0 00000080:00200000:0.0:1787826014.259685:0:155136:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787826014.259699:0:155136:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787826014.259856:0:155136:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e6c7 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155136 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.259865:0:155136:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e6c7 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155136 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.259882:0:155136:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e6c7 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155136 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826014.259915:0:155136:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155136:1874663042857472:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787826014.260072:0:155136:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bb480 going to sleep for 11 seconds 00000100:00100000:0.0:1787826014.270250:0:155136:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155136:1874663042857472:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:0.0:1787826014.270290:0:155136:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e6c7 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9c62 expref: -99 pid: 155136 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.270322:0:155136:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e6c7 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1b94:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9c62 expref: -99 pid: 155136 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.270332:0:155136:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e6c7 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1b94:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9c62 expref: -99 pid: 155136 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.270346:0:155136:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e6c7 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1b94:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9c62 expref: -99 pid: 155136 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.270359:0:155136:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e6c7 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1b94:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9c62 expref: -99 pid: 155136 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.270373:0:155136:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e6c7 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1b94:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9c62 expref: -99 pid: 155136 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.270381:0:155136:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e6c7 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1b94:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9c62 expref: -99 pid: 155136 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.270394:0:155136:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e6c7 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1b94:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9c62 expref: -99 pid: 155136 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826014.270407:0:155136:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c55a5180 x1874663042857472/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/672 e 0 to 0 dl 1787826030 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787826014.270423:0:155136:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e6c7 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1b94:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9c62 expref: -99 pid: 155136 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.270433:0:155136:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/7060 (0 0) 00000080:00200000:0.0:1787826014.270441:0:155136:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00000002:00010000:0.0:1787826014.270532:0:155136:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200026560:0x8:0x0]) in obj [0x200026560:0x8:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826014.270591:0:155136:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6e6ce lrc: 3/1,0 mode: --/CR res: [0x200026560:0x8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 155136 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.270598:0:155136:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6e6ce lrc: 3/1,0 mode: --/CR res: [0x200026560:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155136 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.270609:0:155136:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6e6ce lrc: 3/1,0 mode: --/CR res: [0x200026560:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155136 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826014.270633:0:155136:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155136:1874663042857600:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787826014.270741:0:155136:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bb480 going to sleep for 11 seconds 00000100:00100000:0.0:1787826014.296892:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826014.297002:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049006848 00000100:00100000:1.0:1787826014.297014:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049006848 00000100:00100000:1.0:1787826014.297242:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9149 00000100:00100000:1.0:1787826014.297255:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187906:x1874663049006848:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826014.297279:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6e6ce lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x8:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce9c70 expref: -99 pid: 155136 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826014.297387:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826014.297395:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187906:x1874663049006848:12345-192.168.203.147@tcp:104: Request processed in 143us (520us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826014.297403:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9149 00000100:00100000:1.0:1787826014.302115:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826014.302240:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049007104 00000100:00100000:0.0:1787826014.302251:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049007104 00000100:00100000:0.0:1787826014.302359:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9150 00000100:00100000:0.0:1787826014.302371:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663049007104:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826014.302397:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f7800/0xbef42c0221d6e6d5 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x8:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce9c7e expref: -99 pid: 155136 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826014.302511:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826014.302519:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663049007104:12345-192.168.203.147@tcp:104: Request processed in 150us (409us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826014.302527:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9150 00000080:00200000:0.0:1787826014.340382:0:155137:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826014.340424:0:155137:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826014.340449:0:155137:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826014.340468:0:155137:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826014.340478:0:155137:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826014.340490:0:155137:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826014.340494:0:155137:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826014.340503:0:155137:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501c73b58 00010000:00010000:0.0:1787826014.340508:0:155137:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826014.340519:0:155137:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826014.340536:0:155137:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826014.340546:0:155137:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=29 00000080:00200000:0.0:1787826014.340630:0:155137:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826014.340646:0:155137:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=13 00000080:00200000:0.0:1787826014.340650:0:155137:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787826014.340667:0:155137:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787826014.340873:0:155137:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6e6dc lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155137 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.340882:0:155137:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6e6dc lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155137 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.340894:0:155137:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6e6dc lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155137 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826014.340927:0:155137:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155137:1874663042858368:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787826014.342956:0:155137:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bb900 going to sleep for 11 seconds 00000100:00100000:0.0:1787826014.350517:0:155137:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155137:1874663042858368:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787826014.350626:0:155137:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6e6dc lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155137 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.350643:0:155137:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6e6dc lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155137 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.350652:0:155137:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6e6dc lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155137 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.350660:0:155137:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6e6dc lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155137 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.350670:0:155137:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6e6dc lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155137 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.350682:0:155137:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0512000 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787826014.350687:0:155137:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6e6dc lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155137 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.350912:0:155137:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6e6dc lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155137 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.350922:0:155137:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6e6dc lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 155137 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.350929:0:155137:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787826014.350937:0:155137:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6e6dc lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155137 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826014.350944:0:155137:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6e6dc lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155137 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826014.350966:0:155137:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c55a5880 x1874663042858368/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787826030 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787826014.351058:0:155137:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00000100:00100000:0.0:1787826014.433225:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826014.435268:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049008512 00000100:00100000:0.0:1787826014.435289:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049008512 00000100:00100000:0.0:1787826014.435369:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9151 00000100:00100000:0.0:1787826014.435381:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a7b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187905:x1874663049008512:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826014.435404:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7e00/0xbef42c0221d6e6f8 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x9:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce9d34 expref: -99 pid: 155138 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826014.435519:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826014.435528:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a7b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187905:x1874663049008512:12345-192.168.203.147@tcp:104: Request processed in 149us (2318us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826014.435536:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9151 00000100:00100000:0.0:1787826014.443737:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826014.444684:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049008768 00000100:00100000:1.0:1787826014.444705:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049008768 00000100:00100000:1.0:1787826014.444801:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9152 00000100:00100000:1.0:1787826014.444813:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663049008768:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826014.444836:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f7400/0xbef42c0221d6e6ff lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x9:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce9d42 expref: -99 pid: 155138 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826014.444933:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826014.444941:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663049008768:12345-192.168.203.147@tcp:104: Request processed in 130us (1221us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826014.444950:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9152 00000080:00200000:1.0:1787826014.486663:0:155139:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826014.486711:0:155139:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826014.486734:0:155139:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826014.486787:0:155139:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826014.486796:0:155139:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826014.486807:0:155139:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826014.486811:0:155139:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787826014.486819:0:155139:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501c7bb58 00010000:00010000:1.0:1787826014.486824:0:155139:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826014.486836:0:155139:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826014.486852:0:155139:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826014.487055:0:155139:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=29 00000080:00200000:1.0:1787826014.487065:0:155139:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826014.487090:0:155139:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=13 00000080:00200000:1.0:1787826014.487096:0:155139:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:1.0:1787826014.487113:0:155139:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787826014.487367:0:155139:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6e706 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155139 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826014.487377:0:155139:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6e706 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155139 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826014.487389:0:155139:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6e706 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155139 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826014.487494:0:155139:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155139:1874663042859904:192.168.203.147@tcp:101:lfs.0 00000100:00100000:1.0:1787826014.488999:0:155139:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c89c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826014.620100:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826014.621173:0:15150:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:1.0:1787826014.621204:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049010176 00000100:00100000:1.0:1787826014.621217:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049010176 00000100:00100000:1.0:1787826014.621296:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9153 00000100:00100000:1.0:1787826014.621308:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c442d850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187906:x1874663049010176:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826014.621331:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481800/0xbef42c0221d6e722 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0xa:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce9df8 expref: -99 pid: 155140 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826014.623018:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826014.623051:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c442d850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187906:x1874663049010176:12345-192.168.203.147@tcp:104: Request processed in 1735us (2966us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826014.623066:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9153 00000100:00100000:1.0:1787826014.639568:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826014.640044:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049010432 00000100:00100000:0.0:1787826014.640059:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049010432 00000100:00100000:0.0:1787826014.640381:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9154 00000100:00100000:0.0:1787826014.640394:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663049010432:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826014.640417:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480c00/0xbef42c0221d6e729 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0xa:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce9e06 expref: -99 pid: 155140 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826014.640917:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826014.640927:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663049010432:12345-192.168.203.147@tcp:104: Request processed in 535us (1375us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826014.640935:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9154 00000100:00100000:0.0:1787826014.839939:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826014.840309:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049011840 00000100:00100000:1.0:1787826014.840323:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049011840 00000100:00100000:1.0:1787826014.840392:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9155 00000100:00100000:1.0:1787826014.840405:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a6300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187906:x1874663049011840:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826014.840542:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481200/0xbef42c0221d6e74c lrc: 2/0,0 mode: PR/PR res: [0x200026560:0xb:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce9ebc expref: -99 pid: 155142 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826014.840703:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826014.840711:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a6300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187906:x1874663049011840:12345-192.168.203.147@tcp:104: Request processed in 309us (786us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826014.840720:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9155 00000100:00100000:0.0:1787826014.861764:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826014.861877:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049012096 00000100:00100000:1.0:1787826014.861887:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049012096 00000100:00100000:1.0:1787826014.861959:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9156 00000100:00100000:1.0:1787826014.861971:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8414:x1874663049012096:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826014.861994:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481800/0xbef42c0221d6e753 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0xb:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce9eca expref: -99 pid: 155142 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826014.862097:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826014.862104:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8414:x1874663049012096:12345-192.168.203.147@tcp:104: Request processed in 135us (372us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826014.862112:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9156 00000100:00100000:0.0:1787826015.001973:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826015.002045:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049013504 00000100:00100000:1.0:1787826015.002054:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049013504 00000100:00100000:1.0:1787826015.002116:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9157 00000100:00100000:1.0:1787826015.002133:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187905:x1874663049013504:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826015.002151:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7e00/0xbef42c0221d6e776 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0xc:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce9f80 expref: -99 pid: 155144 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826015.002219:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826015.002225:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187905:x1874663049013504:12345-192.168.203.147@tcp:104: Request processed in 102us (262us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826015.002233:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9157 00000100:00100000:0.0:1787826015.017588:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826015.017698:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049013760 00000100:00100000:1.0:1787826015.017707:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049013760 00000100:00100000:1.0:1787826015.017788:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9158 00000100:00100000:1.0:1787826015.017798:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a4e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663049013760:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826015.017814:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6400/0xbef42c0221d6e77d lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0xc:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ce9f8e expref: -99 pid: 155144 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826015.017874:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826015.017880:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a4e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663049013760:12345-192.168.203.147@tcp:104: Request processed in 84us (303us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826015.017888:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9158 00000080:00200000:0.0:1787826015.063510:0:155145:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826015.063540:0:155145:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826015.063559:0:155145:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826015.063578:0:155145:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826015.063585:0:155145:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826015.063593:0:155145:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826015.063596:0:155145:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826015.063603:0:155145:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501cabb58 00010000:00010000:0.0:1787826015.063607:0:155145:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826015.063615:0:155145:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826015.063628:0:155145:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826015.063634:0:155145:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=29 00000080:00200000:0.0:1787826015.063640:0:155145:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826015.063650:0:155145:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=13 00000080:00200000:0.0:1787826015.063654:0:155145:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787826015.063666:0:155145:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787826015.063706:0:155145:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6e784 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155145 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826015.063714:0:155145:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6e784 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155145 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826015.063722:0:155145:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6e784 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155145 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826015.067337:0:155145:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155145:1874663042864512:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787826015.067544:0:155145:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bb300 going to sleep for 11 seconds 00000100:00100000:1.0:1787826015.072063:0:155145:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155145:1874663042864512:192.168.203.147@tcp:101:lfs.0 00010000:00010000:1.0:1787826015.072098:0:155145:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6e784 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155145 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826015.072113:0:155145:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6e784 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155145 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826015.072120:0:155145:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6e784 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155145 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826015.072131:0:155145:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6e784 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155145 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826015.072139:0:155145:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6e784 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155145 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826015.072146:0:155145:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0513200 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:1.0:1787826015.072150:0:155145:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6e784 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155145 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826015.072156:0:155145:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6e784 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155145 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826015.072163:0:155145:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6e784 lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 155145 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826015.072170:0:155145:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787826015.072177:0:155145:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6e784 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155145 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826015.072183:0:155145:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6e784 lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155145 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826015.072200:0:155145:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c55a6680 x1874663042864512/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787826031 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787826015.072216:0:155145:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:1.0:1787826015.072241:0:155145:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:1.0:1787826015.072307:0:155145:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826015.072320:0:155145:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826015.072329:0:155145:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826015.072339:0:155145:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826015.072346:0:155145:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826015.072350:0:155145:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826015.072354:0:155145:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787826015.072359:0:155145:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501cabbe8 00010000:00010000:1.0:1787826015.072362:0:155145:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826015.072371:0:155145:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826015.072382:0:155145:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826015.073010:0:155145:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=16408 00000080:00200000:1.0:1787826015.073019:0:155145:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826015.073034:0:155145:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=16392 00000080:00200000:1.0:1787826015.073038:0:155145:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:1.0:1787826015.073051:0:155145:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787826015.073105:0:155145:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5800/0xbef42c0221d6e78b lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155145 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826015.073113:0:155145:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5800/0xbef42c0221d6e78b lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155145 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826015.073122:0:155145:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5800/0xbef42c0221d6e78b lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155145 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826015.073143:0:155145:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155145:1874663042864640:192.168.203.147@tcp:101:lfs.0 00000100:00100000:1.0:1787826015.073214:0:155145:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8d80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826015.075378:0:155145:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155145:1874663042864640:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787826015.075512:0:155145:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5800/0xbef42c0221d6e78b lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155145 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826015.075525:0:155145:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5800/0xbef42c0221d6e78b lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155145 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826015.075531:0:155145:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5800/0xbef42c0221d6e78b lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155145 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826015.075538:0:155145:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5800/0xbef42c0221d6e78b lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155145 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826015.075546:0:155145:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5800/0xbef42c0221d6e78b lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155145 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826015.075552:0:155145:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b5800 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787826015.075556:0:155145:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5800/0xbef42c0221d6e78b lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155145 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826015.075562:0:155145:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5800/0xbef42c0221d6e78b lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155145 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826015.075568:0:155145:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5800/0xbef42c0221d6e78b lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 155145 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826015.075575:0:155145:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787826015.075581:0:155145:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5800/0xbef42c0221d6e78b lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155145 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826015.075587:0:155145:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5800/0xbef42c0221d6e78b lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155145 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826015.075605:0:155145:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c35b0e00 x1874663042864640/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787826031 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787826015.075618:0:155145:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787826015.075643:0:155145:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787826015.075714:0:155145:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826015.075728:0:155145:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826015.075738:0:155145:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826015.076565:0:155145:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826015.076574:0:155145:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826015.076582:0:155145:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826015.076585:0:155145:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826015.076590:0:155145:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501cabb58 00010000:00010000:0.0:1787826015.076594:0:155145:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826015.076602:0:155145:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826015.076615:0:155145:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826015.076624:0:155145:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=13 00000080:00200008:0.0:1787826015.076636:0:155145:0:(file.c:6296:ll_getattr_dentry()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1787826015.076640:0:155145:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08),name=d71a.replay-single 00000002:00010000:0.0:1787826015.076654:0:155145:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x20001a9e3:0x17e0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826015.076662:0:155145:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00010000:00010000:0.0:1787826015.076697:0:155145:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6e792 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 155145 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826015.076706:0:155145:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6e792 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 155145 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826015.076716:0:155145:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6e792 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 155145 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826015.076736:0:155145:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155145:1874663042864768:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787826015.076852:0:155145:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bb780 going to sleep for 11 seconds 00000100:00100000:0.0:1787826015.080250:0:155145:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155145:1874663042864768:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787826015.080277:0:155145:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6e792 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9ff0 expref: -99 pid: 155145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826015.080294:0:155145:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6e792 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9ff0 expref: -99 pid: 155145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826015.080303:0:155145:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6e792 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9ff0 expref: -99 pid: 155145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826015.080316:0:155145:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6e792 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9ff0 expref: -99 pid: 155145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826015.080333:0:155145:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6e792 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9ff0 expref: -99 pid: 155145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826015.080348:0:155145:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6e792 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9ff0 expref: -99 pid: 155145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826015.080356:0:155145:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6e792 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9ff0 expref: -99 pid: 155145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826015.080363:0:155145:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6e792 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9ff0 expref: -99 pid: 155145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826015.080374:0:155145:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c55a4700 x1874663042864768/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826031 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787826015.080387:0:155145:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6e792 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9ff0 expref: -99 pid: 155145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826015.080396:0:155145:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787826015.080403:0:155145:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1787826015.080430:0:155145:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x17e0:0x0], updating i_size 4096 00000080:00010000:0.0:1787826015.080437:0:155145:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) for lock 0xbef42c0221d6e792 00000080:00010000:0.0:1787826015.080444:0:155145:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00000080:00010000:0.0:1787826015.080447:0:155145:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6e792 from it ffffb97501cabcb8 00010000:00010000:0.0:1787826015.080450:0:155145:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6e792 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9ff0 expref: -99 pid: 155145 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826015.080464:0:155145:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6e792 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9ff0 expref: -99 pid: 155145 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:0.0:1787826015.081064:0:155145:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1787826015.081081:0:155145:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826015.081097:0:155145:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826015.081108:0:155145:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826015.081119:0:155145:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826015.081127:0:155145:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826015.081132:0:155145:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826015.081134:0:155145:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826015.081139:0:155145:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501cabcb8 00010000:00010000:0.0:1787826015.081142:0:155145:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826015.081149:0:155145:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826015.081266:0:155145:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826015.081274:0:155145:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826015.081281:0:155145:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826015.081293:0:155145:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826015.081300:0:155145:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826015.081303:0:155145:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826015.081306:0:155145:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826015.081309:0:155145:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501cabbf8 00010000:00010000:0.0:1787826015.081312:0:155145:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826015.081318:0:155145:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826015.081327:0:155145:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826015.081332:0:155145:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=267 00000080:00200000:0.0:1787826015.081338:0:155145:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 44 00000080:00200008:0.0:1787826015.081343:0:155145:0:(file.c:1090:ll_file_open()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00000080:00200080:0.0:1787826015.081362:0:155145:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) cmd=c03066f0 arg=7ffdbcab5bc0 00000080:00200000:0.0:1787826015.081374:0:155145:0:(dir.c:451:ll_dir_setdirstripe()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) name=test stripe_offset=0 stripe_count=2, hash_type=0 00010000:00010000:0.0:1787826015.081398:0:155145:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6e792 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ce9ff0 expref: -99 pid: 155145 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826015.081412:0:155145:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6800/0xbef42c0221d64b3f lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9634 expref: -99 pid: 152202 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826015.081447:0:155145:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6e792 lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce9ff0 expref: -99 pid: 155145 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826015.081454:0:155145:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826015.081457:0:155145:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6e792 lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ce9ff0 expref: -99 pid: 155145 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:0.0:1787826015.081475:0:155145:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787826015.081483:0:155145:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c55a4700 x1874663042865024/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 552/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826015.081504:0:155145:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155145:1874663042865024:192.168.203.147@tcp:36:lfs.0 00000100:00100000:0.0:1787826015.081580:0:155145:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bb780 going to sleep for 11 seconds 00000100:00100000:0.0:1787826015.101974:0:155145:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155145:1874663042865024:192.168.203.147@tcp:36:lfs.0 00000080:00200000:0.0:1787826015.102028:0:155145:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x20001a9e3:0x1b99:0x0](ffff9181c83a3248) 00000080:00200000:0.0:1787826015.102035:0:155145:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1b99:0x0], updating i_size 4096 00000080:00200008:0.0:1787826015.102080:0:155145:0:(file.c:479:ll_file_release()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00010000:00010000:0.0:1787826015.102108:0:155145:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513000/0xbef42c0221d64bee lrc: 2/0,0 mode: CR/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce962d expref: -99 pid: 152210 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000100:00100000:0.0:1787826015.168874:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826015.169055:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049015168 00000100:00100000:1.0:1787826015.169079:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049015168 00000100:00100000:1.0:1787826015.169153:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9159 00000100:00100000:1.0:1787826015.169165:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187904:x1874663049015168:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826015.169189:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6400/0xbef42c0221d6e7a0 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0xd:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cea044 expref: -99 pid: 155146 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826015.169431:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826015.169439:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187904:x1874663049015168:12345-192.168.203.147@tcp:104: Request processed in 276us (578us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826015.169448:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9159 00000100:00100000:0.0:1787826015.173285:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826015.173415:0:15150:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:1.0:1787826015.173438:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049015424 00000100:00100000:1.0:1787826015.173449:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049015424 00000100:00100000:1.0:1787826015.173510:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9160 00000100:00100000:1.0:1787826015.173521:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181e053f850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663049015424:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826015.173543:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f7e00/0xbef42c0221d6e7a7 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0xd:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cea052 expref: -99 pid: 155146 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826015.173622:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826015.173634:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181e053f850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663049015424:12345-192.168.203.147@tcp:104: Request processed in 113us (364us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826015.173642:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9160 00000100:00100000:1.0:1787826015.198989:0:155146:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c49e6680 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155146:1874663042865664:192.168.203.147@tcp:36:rmdir.0 00000080:00200000:1.0:1787826015.199033:0:155146:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1b99:0x0](ffff9181c83a3248) 00000080:00200000:0.0:1787826015.222605:0:155147:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826015.222650:0:155147:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826015.222678:0:155147:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826015.225429:0:155147:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826015.225467:0:155147:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826015.225480:0:155147:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826015.225485:0:155147:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826015.225499:0:155147:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501c6bb58 00010000:00010000:0.0:1787826015.225505:0:155147:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826015.225517:0:155147:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826015.225545:0:155147:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826015.225559:0:155147:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=29 00000080:00200000:0.0:1787826015.225571:0:155147:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826015.225596:0:155147:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=13 00000080:00200000:0.0:1787826015.225601:0:155147:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787826015.225628:0:155147:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787826015.225696:0:155147:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6e7ae lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155147 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826015.225706:0:155147:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6e7ae lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155147 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826015.225718:0:155147:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6e7ae lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155147 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826015.225772:0:155147:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155147:1874663042866048:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787826015.225885:0:155147:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bb780 going to sleep for 11 seconds 00000100:00100000:1.0:1787826015.320029:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826015.320086:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049016832 00000100:00100000:1.0:1787826015.320101:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049016832 00000100:00100000:1.0:1787826015.320161:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9161 00000100:00100000:1.0:1787826015.320173:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187906:x1874663049016832:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826015.320195:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480000/0xbef42c0221d6e7ca lrc: 2/0,0 mode: PR/PR res: [0x200026560:0xe:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cea108 expref: -99 pid: 155148 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826015.320290:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826015.320297:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187906:x1874663049016832:12345-192.168.203.147@tcp:104: Request processed in 126us (284us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826015.320305:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9161 00000100:00100000:1.0:1787826015.325969:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826015.326077:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049017088 00000100:00100000:0.0:1787826015.326090:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049017088 00000100:00100000:0.0:1787826015.326152:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9162 00000100:00100000:0.0:1787826015.326164:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663049017088:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826015.326188:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6e7d1 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0xe:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cea116 expref: -99 pid: 155148 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826015.326283:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826015.326290:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663049017088:12345-192.168.203.147@tcp:104: Request processed in 128us (337us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826015.326298:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9162 00000100:00100000:1.0:1787826015.418681:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826015.418786:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049018496 00000100:00100000:0.0:1787826015.418800:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049018496 00000100:00100000:0.0:1787826015.418866:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9163 00000100:00100000:0.0:1787826015.418877:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187906:x1874663049018496:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826015.418910:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7e00/0xbef42c0221d6e7f4 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0xf:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cea1cc expref: -99 pid: 155150 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826015.419009:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826015.419017:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187906:x1874663049018496:12345-192.168.203.147@tcp:104: Request processed in 142us (353us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826015.419025:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9163 00000100:00100000:1.0:1787826015.424370:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826015.426211:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049018752 00000100:00100000:0.0:1787826015.426228:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049018752 00000100:00100000:0.0:1787826015.426294:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9164 00000100:00100000:0.0:1787826015.426307:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b0700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663049018752:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826015.426330:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6400/0xbef42c0221d6e7fb lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0xf:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cea1da expref: -99 pid: 155150 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826015.426445:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826015.426454:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b0700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663049018752:12345-192.168.203.147@tcp:104: Request processed in 149us (2099us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826015.426462:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9164 00000080:00200000:0.0:1787826015.449218:0:155150:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1b9b:0x0](ffff9181c83cb248) 00000100:00100000:1.0:1787826015.559023:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826015.559151:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049020160 00000100:00100000:0.0:1787826015.559166:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049020160 00000100:00100000:0.0:1787826015.559239:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9165 00000100:00100000:0.0:1787826015.559250:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b1880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187904:x1874663049020160:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826015.559273:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7400/0xbef42c0221d6e81e lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x10:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cea290 expref: -99 pid: 155152 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826015.559371:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826015.559378:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b1880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187904:x1874663049020160:12345-192.168.203.147@tcp:104: Request processed in 130us (369us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826015.559386:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9165 00000100:00100000:1.0:1787826015.567319:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826015.567436:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049020416 00000100:00100000:0.0:1787826015.567449:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049020416 00000100:00100000:0.0:1787826015.567517:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9166 00000100:00100000:0.0:1787826015.567530:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b2680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663049020416:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826015.567551:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481600/0xbef42c0221d6e825 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x10:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cea29e expref: -99 pid: 155152 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826015.567641:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826015.567648:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b2680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663049020416:12345-192.168.203.147@tcp:104: Request processed in 122us (344us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826015.567657:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9166 00000100:00100000:1.0:1787826015.732663:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826015.733018:0:15151:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:1.0:1787826015.733041:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049021824 00000100:00100000:1.0:1787826015.733052:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049021824 00000100:00100000:1.0:1787826015.733653:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9167 00000100:00100000:1.0:1787826015.733668:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181e053c050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187906:x1874663049021824:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826015.733878:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7e00/0xbef42c0221d6e848 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x11:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cea354 expref: -99 pid: 155156 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826015.734056:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826015.734073:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181e053c050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187906:x1874663049021824:12345-192.168.203.147@tcp:104: Request processed in 400us (1418us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826015.734082:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9167 00000100:00100000:0.0:1787826015.743458:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826015.743551:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049022080 00000100:00100000:1.0:1787826015.743625:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049022080 00000100:00100000:1.0:1787826015.743732:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9168 00000100:00100000:1.0:1787826015.743772:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663049022080:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826015.743793:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f7400/0xbef42c0221d6e84f lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x11:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cea362 expref: -99 pid: 155156 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826015.743870:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826015.743877:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663049022080:12345-192.168.203.147@tcp:104: Request processed in 136us (432us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826015.743885:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9168 00000100:00100000:1.0:1787826015.892090:0:155163:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c494e680 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155163:1874663042872960:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:1.0:1787826015.892132:0:155163:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6e872 lrc: 4/1,0 mode: --/CR res: [0x200026560:0x12:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea418 expref: -99 pid: 155163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826015.892148:0:155163:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6e872 lrc: 4/1,0 mode: --/PR res: [0x200026560:0x12:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea418 expref: -99 pid: 155163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826015.892162:0:155163:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6e872 lrc: 4/1,0 mode: PR/PR res: [0x200026560:0x12:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cea418 expref: -99 pid: 155163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826015.892181:0:155163:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6e872 lrc: 4/1,0 mode: PR/PR res: [0x200026560:0x12:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea418 expref: -99 pid: 155163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826015.892196:0:155163:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6e872 lrc: 5/2,0 mode: PR/PR res: [0x200026560:0x12:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cea418 expref: -99 pid: 155163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826015.892205:0:155163:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6e872 lrc: 5/2,0 mode: PR/PR res: [0x200026560:0x12:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cea418 expref: -99 pid: 155163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826015.892213:0:155163:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6e872 lrc: 4/1,0 mode: PR/PR res: [0x200026560:0x12:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cea418 expref: -99 pid: 155163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826015.892225:0:155163:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c494e680 x1874663042872960/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787826031 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826015.892241:0:155163:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6e872 lrc: 3/1,0 mode: PR/PR res: [0x200026560:0x12:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea418 expref: -99 pid: 155163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826015.892256:0:155163:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590091616/18 (0 0) 00010000:00010000:1.0:1787826015.892264:0:155163:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6e872 lrc: 3/1,0 mode: PR/PR res: [0x200026560:0x12:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cea418 expref: -99 pid: 155163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826015.892273:0:155163:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6e872 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x12:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea418 expref: -99 pid: 155163 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000002:00010000:1.0:1787826015.892308:0:155163:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001f58:0x12:0x0]) in obj [0x240001f58:0x12:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826015.892364:0:155163:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e879 lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x12:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 155163 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826015.892372:0:155163:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e879 lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x12:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155163 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826015.892382:0:155163:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e879 lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x12:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155163 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826015.892404:0:155163:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2300 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155163:1874663042873088:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:1.0:1787826015.892494:0:155163:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c89c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826015.915933:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826015.916401:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049023616 00000100:00100000:0.0:1787826015.916416:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049023616 00000100:00100000:1.0:1787826015.916597:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9169 00000100:00100000:1.0:1787826015.916610:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187904:x1874663049023616:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826015.916636:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6e872 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x12:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cea418 expref: -99 pid: 155163 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826015.917012:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826015.917022:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187904:x1874663049023616:12345-192.168.203.147@tcp:104: Request processed in 414us (1106us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826015.917032:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9169 00000100:00100000:0.0:1787826015.922546:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826015.922640:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049023872 00000100:00100000:1.0:1787826015.922652:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049023872 00000100:00100000:1.0:1787826015.922940:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9170 00000100:00100000:1.0:1787826015.922952:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a6300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8414:x1874663049023872:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826015.922973:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e879 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x12:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cea426 expref: -99 pid: 155163 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826015.923078:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826015.923086:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a6300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8414:x1874663049023872:12345-192.168.203.147@tcp:104: Request processed in 137us (555us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826015.923095:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9170 00000002:00010000:0.0:1787826015.974862:0:155164:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787826015.974917:0:155164:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e887 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155164 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826015.974925:0:155164:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e887 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155164 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826015.974935:0:155164:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e887 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155164 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826015.974961:0:155164:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155164:1874663042873856:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787826015.975061:0:155164:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bb900 going to sleep for 11 seconds 00000080:00200000:0.0:1787826016.023256:0:155165:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826016.023291:0:155165:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826016.023323:0:155165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826016.023340:0:155165:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826016.023348:0:155165:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826016.023357:0:155165:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826016.023360:0:155165:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826016.023368:0:155165:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501d2bc20 00010000:00010000:0.0:1787826016.023607:0:155165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826016.023620:0:155165:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826016.023635:0:155165:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826016.023642:0:155165:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=24 00000080:00200000:0.0:1787826016.023649:0:155165:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826016.023662:0:155165:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=0 00000080:00200000:0.0:1787826016.023666:0:155165:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787826016.023680:0:155165:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787826016.023722:0:155165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e895 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826016.023730:0:155165:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e895 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826016.023740:0:155165:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e895 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155165 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826016.023870:0:155165:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155165:1874663042874368:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787826016.024016:0:155165:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bb840 going to sleep for 11 seconds 00000100:00100000:0.0:1787826016.032709:0:155165:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155165:1874663042874368:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:0.0:1787826016.032736:0:155165:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e895 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea4ce expref: -99 pid: 155165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826016.032917:0:155165:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e895 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1b9f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea4ce expref: -99 pid: 155165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826016.033046:0:155165:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e895 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1b9f:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea4ce expref: -99 pid: 155165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826016.033058:0:155165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e895 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1b9f:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cea4ce expref: -99 pid: 155165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826016.033081:0:155165:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e895 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1b9f:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea4ce expref: -99 pid: 155165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826016.033096:0:155165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e895 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1b9f:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cea4ce expref: -99 pid: 155165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826016.033104:0:155165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e895 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1b9f:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cea4ce expref: -99 pid: 155165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826016.033110:0:155165:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e895 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1b9f:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cea4ce expref: -99 pid: 155165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826016.033123:0:155165:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c55a7800 x1874663042874368/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/672 e 0 to 0 dl 1787826032 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787826016.033136:0:155165:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e895 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1b9f:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea4ce expref: -99 pid: 155165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826016.033150:0:155165:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/7071 (0 0) 00000080:00200000:0.0:1787826016.033157:0:155165:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00000002:00010000:0.0:1787826016.033215:0:155165:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200026560:0x13:0x0]) in obj [0x200026560:0x13:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826016.033258:0:155165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e89c lrc: 3/1,0 mode: --/CR res: [0x200026560:0x13:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 155165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826016.033267:0:155165:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e89c lrc: 3/1,0 mode: --/CR res: [0x200026560:0x13:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826016.033275:0:155165:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e89c lrc: 3/1,0 mode: --/CR res: [0x200026560:0x13:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155165 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826016.034060:0:155165:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155165:1874663042874496:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787826016.034153:0:155165:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bb840 going to sleep for 11 seconds 00000100:00100000:0.0:1787826016.046232:0:155165:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155165:1874663042874496:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:0.0:1787826016.046308:0:155165:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e89c lrc: 4/1,0 mode: --/CR res: [0x200026560:0x13:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea4dc expref: -99 pid: 155165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826016.046322:0:155165:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e89c lrc: 4/1,0 mode: --/PR res: [0x200026560:0x13:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea4dc expref: -99 pid: 155165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826016.046332:0:155165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e89c lrc: 4/1,0 mode: PR/PR res: [0x200026560:0x13:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cea4dc expref: -99 pid: 155165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826016.046342:0:155165:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e89c lrc: 4/1,0 mode: PR/PR res: [0x200026560:0x13:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea4dc expref: -99 pid: 155165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826016.046359:0:155165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e89c lrc: 5/2,0 mode: PR/PR res: [0x200026560:0x13:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cea4dc expref: -99 pid: 155165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826016.046368:0:155165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e89c lrc: 5/2,0 mode: PR/PR res: [0x200026560:0x13:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cea4dc expref: -99 pid: 155165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826016.046375:0:155165:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e89c lrc: 4/1,0 mode: PR/PR res: [0x200026560:0x13:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cea4dc expref: -99 pid: 155165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826016.046385:0:155165:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c55a4e00 x1874663042874496/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787826032 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787826016.046399:0:155165:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e89c lrc: 3/1,0 mode: PR/PR res: [0x200026560:0x13:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea4dc expref: -99 pid: 155165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826016.046411:0:155165:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590091616/19 (0 0) 00010000:00010000:0.0:1787826016.046417:0:155165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e89c lrc: 3/1,0 mode: PR/PR res: [0x200026560:0x13:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cea4dc expref: -99 pid: 155165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826016.046426:0:155165:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e89c lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x13:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea4dc expref: -99 pid: 155165 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000002:00010000:0.0:1787826016.046450:0:155165:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001f58:0x13:0x0]) in obj [0x240001f58:0x13:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826016.047388:0:155165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6e8a3 lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x13:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 155165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826016.047401:0:155165:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6e8a3 lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x13:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826016.047411:0:155165:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6e8a3 lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x13:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155165 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826016.047432:0:155165:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155165:1874663042874624:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:1.0:1787826016.050331:0:155165:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bb840 going to sleep for 11 seconds 00000100:00100000:1.0:1787826016.083536:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826016.087906:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049025280 00000100:00100000:0.0:1787826016.087921:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049025280 00000100:00100000:0.0:1787826016.087990:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9171 00000100:00100000:0.0:1787826016.088001:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187906:x1874663049025280:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826016.088018:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6e89c lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x13:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cea4dc expref: -99 pid: 155165 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826016.088377:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826016.088384:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187906:x1874663049025280:12345-192.168.203.147@tcp:104: Request processed in 385us (4921us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826016.088391:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9171 00000100:00100000:1.0:1787826016.100034:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826016.104607:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049025536 00000100:00100000:0.0:1787826016.104623:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049025536 00000100:00100000:0.0:1787826016.104773:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9172 00000100:00100000:0.0:1787826016.104785:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b0700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663049025536:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826016.104817:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6e8a3 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x13:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cea4ea expref: -99 pid: 155165 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826016.104919:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826016.104927:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b0700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663049025536:12345-192.168.203.147@tcp:104: Request processed in 145us (4903us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826016.104934:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9172 00000100:00100000:1.0:1787826016.225341:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826016.225429:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049026944 00000100:00100000:0.0:1787826016.225440:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049026944 00000100:00100000:0.0:1787826016.225506:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9173 00000100:00100000:0.0:1787826016.225517:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b1880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187904:x1874663049026944:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826016.225535:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7a00/0xbef42c0221d6e8c6 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x14:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cea5a0 expref: -99 pid: 155167 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826016.225662:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826016.225670:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b1880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187904:x1874663049026944:12345-192.168.203.147@tcp:104: Request processed in 154us (339us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826016.225678:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9173 00000100:00100000:0.0:1787826016.239858:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826016.239971:0:33333:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:1.0:1787826016.239991:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049027200 00000100:00100000:1.0:1787826016.239999:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049027200 00000100:00100000:1.0:1787826016.240112:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9174 00000100:00100000:1.0:1787826016.240123:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181e053f050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663049027200:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826016.240141:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6e8cd lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x14:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cea5ae expref: -99 pid: 155167 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826016.240219:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826016.240225:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181e053f050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663049027200:12345-192.168.203.147@tcp:104: Request processed in 105us (379us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826016.240231:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9174 00000100:00100000:1.0:1787826016.335949:0:155168:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f2a00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155168:1874663042877056:192.168.203.147@tcp:101:lfs.0 00010000:00010000:1.0:1787826016.335975:0:155168:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6e8e2 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea610 expref: -99 pid: 155168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.335991:0:155168:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6e8e2 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea610 expref: -99 pid: 155168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.335998:0:155168:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6e8e2 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea610 expref: -99 pid: 155168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.336008:0:155168:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6e8e2 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cea610 expref: -99 pid: 155168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.336018:0:155168:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6e8e2 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea610 expref: -99 pid: 155168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.336031:0:155168:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6e8e2 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cea610 expref: -99 pid: 155168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.336042:0:155168:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6e8e2 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cea610 expref: -99 pid: 155168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.336050:0:155168:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6e8e2 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cea610 expref: -99 pid: 155168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826016.336080:0:155168:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c71f2a00 x1874663042877056/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826032 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826016.336096:0:155168:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6e8e2 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea610 expref: -99 pid: 155168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.336105:0:155168:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/6112 (0 0) 00000080:00200000:1.0:1787826016.336112:0:155168:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787826016.336147:0:155168:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x17e0:0x0], updating i_size 4096 00000080:00010000:1.0:1787826016.336154:0:155168:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) for lock 0xbef42c0221d6e8e2 00000080:00010000:1.0:1787826016.336161:0:155168:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00000080:00010000:1.0:1787826016.336164:0:155168:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6e8e2 from it ffffb97501d43cb8 00010000:00010000:1.0:1787826016.336168:0:155168:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6e8e2 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cea610 expref: -99 pid: 155168 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.336176:0:155168:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6e8e2 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea610 expref: -99 pid: 155168 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:1.0:1787826016.336807:0:155168:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), request_mask 0, flags 0, foreign 0 00000080:00200000:1.0:1787826016.336814:0:155168:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826016.336832:0:155168:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826016.336843:0:155168:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826016.336855:0:155168:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826016.336861:0:155168:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826016.336866:0:155168:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826016.336869:0:155168:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787826016.336874:0:155168:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501d43cb8 00010000:00010000:1.0:1787826016.336877:0:155168:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826016.336885:0:155168:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826016.337019:0:155168:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826016.337028:0:155168:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826016.337036:0:155168:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826016.337046:0:155168:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826016.337053:0:155168:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826016.337057:0:155168:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826016.337060:0:155168:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787826016.337075:0:155168:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501d43bf8 00010000:00010000:1.0:1787826016.337079:0:155168:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826016.337086:0:155168:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826016.337095:0:155168:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826016.337102:0:155168:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=267 00000080:00200000:1.0:1787826016.337107:0:155168:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 44 00000080:00200008:1.0:1787826016.337112:0:155168:0:(file.c:1090:ll_file_open()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00000080:00200080:1.0:1787826016.337134:0:155168:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) cmd=c03066f0 arg=7ffe66850160 00000080:00200000:1.0:1787826016.337148:0:155168:0:(dir.c:451:ll_dir_setdirstripe()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) name=test stripe_offset=0 stripe_count=2, hash_type=0 00010000:00010000:1.0:1787826016.337174:0:155168:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6e8e2 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00cea610 expref: -99 pid: 155168 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826016.337189:0:155168:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6800/0xbef42c0221d64b3f lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9634 expref: -99 pid: 152202 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826016.337228:0:155168:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6e8e2 lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cea610 expref: -99 pid: 155168 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826016.337236:0:155168:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826016.337239:0:155168:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6e8e2 lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cea610 expref: -99 pid: 155168 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:1.0:1787826016.337260:0:155168:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787826016.337269:0:155168:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c35b0a80 x1874663042877312/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 552/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826016.337291:0:155168:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155168:1874663042877312:192.168.203.147@tcp:36:lfs.0 00000100:00100000:1.0:1787826016.337363:0:155168:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8b40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826016.351076:0:155168:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155168:1874663042877312:192.168.203.147@tcp:36:lfs.0 00000080:00200000:1.0:1787826016.351200:0:155168:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x20001a9e3:0x1ba1:0x0](ffff9181c839db88) 00000080:00200000:1.0:1787826016.351208:0:155168:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1ba1:0x0], updating i_size 4096 00000080:00200008:1.0:1787826016.351246:0:155168:0:(file.c:479:ll_file_release()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00010000:00010000:1.0:1787826016.351275:0:155168:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513000/0xbef42c0221d64bee lrc: 2/0,0 mode: CR/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce962d expref: -99 pid: 152210 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826016.370581:0:155169:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826016.370611:0:155169:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826016.370630:0:155169:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826016.370648:0:155169:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826016.370657:0:155169:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826016.370666:0:155169:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826016.370670:0:155169:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826016.370677:0:155169:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501c83c20 00010000:00010000:0.0:1787826016.371049:0:155169:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826016.371071:0:155169:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826016.371086:0:155169:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826016.371094:0:155169:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=24 00000080:00200000:0.0:1787826016.371101:0:155169:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826016.371113:0:155169:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=0 00000080:00200000:0.0:1787826016.371118:0:155169:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787826016.371131:0:155169:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787826016.371175:0:155169:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6e8e9 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155169 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826016.371183:0:155169:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6e8e9 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155169 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826016.371192:0:155169:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6e8e9 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155169 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826016.371216:0:155169:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155169:1874663042877440:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787826016.371349:0:155169:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bbd80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826016.407937:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826016.408020:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049028608 00000100:00100000:0.0:1787826016.408030:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049028608 00000100:00100000:0.0:1787826016.408095:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9175 00000100:00100000:0.0:1787826016.408106:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187904:x1874663049028608:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826016.408132:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6e8f0 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x15:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cea664 expref: -99 pid: 155169 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826016.408209:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826016.408217:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187904:x1874663049028608:12345-192.168.203.147@tcp:104: Request processed in 113us (289us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826016.408227:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9175 00000100:00100000:1.0:1787826016.417111:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826016.417901:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049028864 00000100:00100000:0.0:1787826016.417910:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049028864 00000100:00100000:0.0:1787826016.417961:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9176 00000100:00100000:0.0:1787826016.417971:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b0700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663049028864:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826016.417987:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481400/0xbef42c0221d6e8f7 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x15:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cea672 expref: -99 pid: 155169 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826016.418056:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826016.418073:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b0700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663049028864:12345-192.168.203.147@tcp:104: Request processed in 93us (958us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826016.418080:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9176 00000080:00200000:0.0:1787826016.491983:0:155171:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826016.492015:0:155171:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826016.492032:0:155171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826016.493108:0:155171:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826016.493136:0:155171:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826016.493146:0:155171:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826016.493150:0:155171:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826016.493159:0:155171:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501d43b58 00010000:00010000:0.0:1787826016.493164:0:155171:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826016.493179:0:155171:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826016.493193:0:155171:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826016.493202:0:155171:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=29 00000080:00200000:0.0:1787826016.493208:0:155171:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826016.493218:0:155171:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=13 00000080:00200000:0.0:1787826016.493222:0:155171:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787826016.493238:0:155171:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787826016.493284:0:155171:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6e8fe lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155171 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826016.493292:0:155171:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6e8fe lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155171 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826016.493301:0:155171:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6e8fe lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155171 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826016.493325:0:155171:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155171:1874663042878336:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787826016.494176:0:155171:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bb0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826016.559381:0:155171:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f1500 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155171:1874663042878848:192.168.203.147@tcp:36:lfs.0 00000080:00200000:1.0:1787826016.559432:0:155171:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x20001a9e3:0x1ba2:0x0](ffff9181c839ba88) 00000080:00200000:1.0:1787826016.559439:0:155171:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1ba2:0x0], updating i_size 4096 00000080:00200008:1.0:1787826016.559472:0:155171:0:(file.c:479:ll_file_release()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00010000:00010000:1.0:1787826016.559505:0:155171:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513000/0xbef42c0221d64bee lrc: 2/0,0 mode: CR/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce962d expref: -99 pid: 152210 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826016.563990:0:155178:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826016.564020:0:155178:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826016.564038:0:155178:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826016.564054:0:155178:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826016.572078:0:155178:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826016.572090:0:155178:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826016.572095:0:155178:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787826016.572105:0:155178:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501d6bc20 00010000:00010000:1.0:1787826016.572111:0:155178:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826016.572122:0:155178:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826016.572138:0:155178:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826016.572148:0:155178:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=24 00000080:00200000:1.0:1787826016.572157:0:155178:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826016.572178:0:155178:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=0 00000080:00200000:1.0:1787826016.572183:0:155178:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:1.0:1787826016.572204:0:155178:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787826016.572256:0:155178:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e913 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155178 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.572270:0:155178:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e913 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155178 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.572281:0:155178:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e913 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155178 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826016.572309:0:155178:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1880 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155178:1874663042878976:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:1.0:1787826016.572398:0:155178:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8b40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826016.583243:0:155178:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1880 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155178:1874663042878976:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:1.0:1787826016.583270:0:155178:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e913 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea71a expref: -99 pid: 155178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.583292:0:155178:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e913 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1ba2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea71a expref: -99 pid: 155178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.583300:0:155178:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e913 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1ba2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea71a expref: -99 pid: 155178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.583312:0:155178:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e913 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1ba2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cea71a expref: -99 pid: 155178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.583323:0:155178:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e913 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1ba2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea71a expref: -99 pid: 155178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.583335:0:155178:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e913 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1ba2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cea71a expref: -99 pid: 155178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.583347:0:155178:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e913 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1ba2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cea71a expref: -99 pid: 155178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.583354:0:155178:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e913 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1ba2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cea71a expref: -99 pid: 155178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826016.583364:0:155178:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c35b1880 x1874663042878976/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/672 e 0 to 0 dl 1787826032 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826016.583376:0:155178:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e913 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1ba2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea71a expref: -99 pid: 155178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.583385:0:155178:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/7074 (0 0) 00000080:00200000:1.0:1787826016.583390:0:155178:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00000002:00010000:1.0:1787826016.583446:0:155178:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200026560:0x16:0x0]) in obj [0x200026560:0x16:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826016.583494:0:155178:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6e91a lrc: 3/1,0 mode: --/CR res: [0x200026560:0x16:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 155178 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.583502:0:155178:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6e91a lrc: 3/1,0 mode: --/CR res: [0x200026560:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155178 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.583511:0:155178:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6e91a lrc: 3/1,0 mode: --/CR res: [0x200026560:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155178 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826016.583529:0:155178:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155178:1874663042879104:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:1.0:1787826016.583601:0:155178:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8b40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826016.592931:0:155178:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155178:1874663042879104:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:1.0:1787826016.592963:0:155178:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6e91a lrc: 4/1,0 mode: --/CR res: [0x200026560:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea728 expref: -99 pid: 155178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.592974:0:155178:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6e91a lrc: 4/1,0 mode: --/PR res: [0x200026560:0x16:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea728 expref: -99 pid: 155178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.592987:0:155178:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6e91a lrc: 4/1,0 mode: PR/PR res: [0x200026560:0x16:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cea728 expref: -99 pid: 155178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.592998:0:155178:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6e91a lrc: 4/1,0 mode: PR/PR res: [0x200026560:0x16:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea728 expref: -99 pid: 155178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.593010:0:155178:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6e91a lrc: 5/2,0 mode: PR/PR res: [0x200026560:0x16:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cea728 expref: -99 pid: 155178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.593017:0:155178:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6e91a lrc: 5/2,0 mode: PR/PR res: [0x200026560:0x16:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cea728 expref: -99 pid: 155178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.593023:0:155178:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6e91a lrc: 4/1,0 mode: PR/PR res: [0x200026560:0x16:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cea728 expref: -99 pid: 155178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826016.593034:0:155178:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c35b2680 x1874663042879104/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787826032 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826016.593047:0:155178:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6e91a lrc: 3/1,0 mode: PR/PR res: [0x200026560:0x16:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea728 expref: -99 pid: 155178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.593068:0:155178:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590091616/22 (0 0) 00010000:00010000:1.0:1787826016.593077:0:155178:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6e91a lrc: 3/1,0 mode: PR/PR res: [0x200026560:0x16:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cea728 expref: -99 pid: 155178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.593086:0:155178:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6e91a lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x16:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea728 expref: -99 pid: 155178 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000002:00010000:1.0:1787826016.593122:0:155178:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001f58:0x16:0x0]) in obj [0x240001f58:0x16:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826016.593180:0:155178:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6e921 lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x16:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 155178 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.593187:0:155178:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6e921 lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155178 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.593196:0:155178:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6e921 lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155178 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826016.593242:0:155178:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155178:1874663042879232:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:1.0:1787826016.593312:0:155178:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8b40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826016.619523:0:155178:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155178:1874663042879232:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:1.0:1787826016.619556:0:155178:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6e921 lrc: 4/1,0 mode: --/CR res: [0x240001f58:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea736 expref: -99 pid: 155178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.619571:0:155178:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6e921 lrc: 4/1,0 mode: --/PR res: [0x240001f58:0x16:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea736 expref: -99 pid: 155178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.619586:0:155178:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6e921 lrc: 4/1,0 mode: PR/PR res: [0x240001f58:0x16:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cea736 expref: -99 pid: 155178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.619602:0:155178:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6e921 lrc: 4/1,0 mode: PR/PR res: [0x240001f58:0x16:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea736 expref: -99 pid: 155178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.619619:0:155178:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6e921 lrc: 5/2,0 mode: PR/PR res: [0x240001f58:0x16:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cea736 expref: -99 pid: 155178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.619630:0:155178:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6e921 lrc: 5/2,0 mode: PR/PR res: [0x240001f58:0x16:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cea736 expref: -99 pid: 155178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.619637:0:155178:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6e921 lrc: 4/1,0 mode: PR/PR res: [0x240001f58:0x16:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cea736 expref: -99 pid: 155178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826016.619648:0:155178:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c35b2680 x1874663042879232/t0(0) o101->lustre-MDT0001-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787826032 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826016.619663:0:155178:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6e921 lrc: 3/1,0 mode: PR/PR res: [0x240001f58:0x16:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea736 expref: -99 pid: 155178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.619675:0:155178:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d0f54800 type 13 mode 4 res 9663684440/22 (0 0) 00010000:00010000:1.0:1787826016.619683:0:155178:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6e921 lrc: 3/1,0 mode: PR/PR res: [0x240001f58:0x16:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cea736 expref: -99 pid: 155178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.619691:0:155178:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6e921 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x16:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea736 expref: -99 pid: 155178 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826016.619724:0:155178:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1ba2:0x0], updating i_size 8192 00000080:00200000:1.0:1787826016.619728:0:155178:0:(namei.c:147:ll_iget()) got inode: [0x20001a9e3:0x1ba2:0x0](ffff9181c839ba88): rc = 0 00000080:00010000:1.0:1787826016.619738:0:155178:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x1ba2:0x0](ffff9181c839ba88) for lock 0xbef42c0221d6e913 00000080:00010000:1.0:1787826016.619815:0:155178:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x1ba2:0x0](ffff9181c839ba88) 00000080:00010000:1.0:1787826016.619821:0:155178:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6e913 from it ffffb97501d6be08 00010000:00010000:1.0:1787826016.619828:0:155178:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e913 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1ba2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cea71a expref: -99 pid: 155178 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.619840:0:155178:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e913 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1ba2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea71a expref: -99 pid: 155178 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826016.619866:0:155178:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 3 00000080:00200000:1.0:1787826016.619878:0:155178:0:(namei.c:2328:ll_rmdir()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00010000:00010000:1.0:1787826016.619913:0:155178:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e913 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1ba2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00cea71a expref: -99 pid: 155178 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826016.619964:0:155178:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e913 lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x1ba2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cea71a expref: -99 pid: 155178 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826016.619971:0:155178:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826016.619975:0:155178:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e913 lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x1ba2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cea71a expref: -99 pid: 155178 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:1.0:1787826016.620001:0:155178:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787826016.620008:0:155178:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c35b1880 x1874663042879488/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826016.620030:0:155178:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1880 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155178:1874663042879488:192.168.203.147@tcp:36:rmdir.0 00000100:00100000:1.0:1787826016.620112:0:155178:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8b40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826016.633574:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826016.633666:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049030272 00000100:00100000:1.0:1787826016.633675:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049030272 00000100:00100000:1.0:1787826016.633733:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9177 00000100:00100000:1.0:1787826016.633782:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187904:x1874663049030272:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826016.633803:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6e91a lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x16:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cea728 expref: -99 pid: 155178 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826016.633869:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826016.633875:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187904:x1874663049030272:12345-192.168.203.147@tcp:104: Request processed in 134us (310us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826016.633882:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9177 00000100:00100000:0.0:1787826016.665247:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826016.665407:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049030528 00000100:00100000:1.0:1787826016.665417:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049030528 00000100:00100000:1.0:1787826016.665474:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9178 00000100:00100000:1.0:1787826016.665484:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a7b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8414:x1874663049030528:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826016.665501:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6e921 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x16:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cea736 expref: -99 pid: 155178 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826016.665579:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826016.665587:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a7b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8414:x1874663049030528:12345-192.168.203.147@tcp:104: Request processed in 105us (350us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826016.665594:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9178 00000100:00100000:1.0:1787826016.697702:0:155178:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1880 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155178:1874663042879488:192.168.203.147@tcp:36:rmdir.0 00000080:00200000:1.0:1787826016.697799:0:155178:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1ba2:0x0](ffff9181c839ba88) 00000080:00200000:0.0:1787826016.714289:0:155179:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826016.714333:0:155179:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826016.714357:0:155179:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826016.714384:0:155179:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826016.714393:0:155179:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826016.714401:0:155179:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826016.714406:0:155179:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826016.714414:0:155179:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501d73b58 00010000:00010000:0.0:1787826016.714419:0:155179:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826016.714428:0:155179:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826016.714444:0:155179:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826016.714454:0:155179:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=29 00000080:00200000:0.0:1787826016.714467:0:155179:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826016.714480:0:155179:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=13 00000080:00200000:0.0:1787826016.714485:0:155179:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787826016.714499:0:155179:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787826016.714555:0:155179:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e928 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155179 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826016.714563:0:155179:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e928 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155179 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826016.714575:0:155179:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6e928 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155179 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826016.714612:0:155179:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6a00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155179:1874663042879872:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787826016.714986:0:155179:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bb180 going to sleep for 11 seconds 00000100:00100000:0.0:1787826016.786257:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826016.788107:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049031936 00000100:00100000:1.0:1787826016.788195:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049031936 00000100:00100000:1.0:1787826016.788327:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9179 00000100:00100000:1.0:1787826016.788341:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a5880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187906:x1874663049031936:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826016.788366:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7200/0xbef42c0221d6e944 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x17:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cea7ec expref: -99 pid: 155180 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826016.788527:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826016.788535:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a5880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187906:x1874663049031936:12345-192.168.203.147@tcp:104: Request processed in 198us (2291us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826016.788543:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9179 00000100:00100000:0.0:1787826016.797326:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826016.797464:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049032192 00000100:00100000:1.0:1787826016.797473:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049032192 00000100:00100000:1.0:1787826016.797568:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9180 00000100:00100000:1.0:1787826016.797580:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a5180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663049032192:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826016.797600:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6600/0xbef42c0221d6e94b lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x17:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cea7fa expref: -99 pid: 155180 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826016.797693:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826016.797701:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a5180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663049032192:12345-192.168.203.147@tcp:104: Request processed in 123us (376us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826016.797710:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9180 00000080:00200000:0.0:1787826016.841116:0:155181:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826016.841147:0:155181:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826016.841165:0:155181:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826016.841182:0:155181:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826016.841190:0:155181:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826016.841198:0:155181:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826016.841202:0:155181:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826016.841209:0:155181:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501d83b58 00010000:00010000:0.0:1787826016.841213:0:155181:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826016.841223:0:155181:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826016.841234:0:155181:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826016.841240:0:155181:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=29 00000080:00200000:0.0:1787826016.841245:0:155181:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826016.841254:0:155181:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=13 00000080:00200000:0.0:1787826016.841258:0:155181:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787826016.841439:0:155181:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787826016.841483:0:155181:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e952 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155181 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826016.841491:0:155181:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e952 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155181 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826016.841501:0:155181:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e952 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155181 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826016.841526:0:155181:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155181:1874663042881408:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787826016.841973:0:155181:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bb6c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826016.845253:0:155181:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155181:1874663042881408:192.168.203.147@tcp:101:lfs.0 00010000:00010000:1.0:1787826016.845284:0:155181:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e952 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155181 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.845300:0:155181:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e952 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155181 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.845309:0:155181:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e952 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155181 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.845316:0:155181:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e952 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155181 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.845327:0:155181:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e952 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155181 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.845334:0:155181:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0512800 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:1.0:1787826016.845340:0:155181:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e952 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155181 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.845348:0:155181:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e952 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155181 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.845355:0:155181:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e952 lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 155181 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.845363:0:155181:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787826016.845382:0:155181:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e952 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155181 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.845390:0:155181:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6e952 lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155181 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826016.845409:0:155181:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c55a5f80 x1874663042881408/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787826032 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787826016.845426:0:155181:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:1.0:1787826016.845461:0:155181:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:1.0:1787826016.845552:0:155181:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826016.845571:0:155181:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826016.845582:0:155181:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826016.845599:0:155181:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826016.845606:0:155181:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826016.845613:0:155181:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826016.845617:0:155181:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787826016.845623:0:155181:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501d83be8 00010000:00010000:1.0:1787826016.845626:0:155181:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826016.845635:0:155181:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826016.845649:0:155181:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826016.845658:0:155181:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=16408 00000080:00200000:1.0:1787826016.845663:0:155181:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826016.845676:0:155181:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=16392 00000080:00200000:1.0:1787826016.845680:0:155181:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:1.0:1787826016.845694:0:155181:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787826016.845939:0:155181:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6e959 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155181 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.845947:0:155181:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6e959 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155181 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.845958:0:155181:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6e959 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155181 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826016.845988:0:155181:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155181:1874663042881536:192.168.203.147@tcp:101:lfs.0 00000100:00100000:1.0:1787826016.846100:0:155181:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8000 going to sleep for 11 seconds 00000100:00100000:1.0:1787826016.857204:0:155181:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155181:1874663042881536:192.168.203.147@tcp:101:lfs.0 00010000:00010000:1.0:1787826016.857235:0:155181:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6e959 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155181 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.857252:0:155181:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6e959 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155181 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.857260:0:155181:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6e959 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155181 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.857268:0:155181:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6e959 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155181 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.857278:0:155181:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6e959 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155181 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.857286:0:155181:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4c00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:1.0:1787826016.857291:0:155181:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6e959 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155181 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.857299:0:155181:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6e959 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155181 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.857307:0:155181:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6e959 lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 155181 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.857314:0:155181:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787826016.857334:0:155181:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6e959 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155181 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.857342:0:155181:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6e959 lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155181 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826016.857363:0:155181:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c35b2680 x1874663042881536/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787826032 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787826016.857500:0:155181:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:1.0:1787826016.857538:0:155181:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:1.0:1787826016.857657:0:155181:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826016.857678:0:155181:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826016.857687:0:155181:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826016.857705:0:155181:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826016.857713:0:155181:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826016.857718:0:155181:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826016.857722:0:155181:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787826016.857738:0:155181:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501d83b58 00010000:00010000:1.0:1787826016.861327:0:155181:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826016.861354:0:155181:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826016.861378:0:155181:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826016.861396:0:155181:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=13 00000080:00200008:1.0:1787826016.861422:0:155181:0:(file.c:6296:ll_getattr_dentry()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), request_mask 0, flags 0, foreign 0 00000080:00200000:1.0:1787826016.861431:0:155181:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08),name=d71a.replay-single 00000002:00010000:1.0:1787826016.861468:0:155181:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x20001a9e3:0x17e0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826016.861484:0:155181:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00010000:00010000:1.0:1787826016.861542:0:155181:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5000/0xbef42c0221d6e960 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 155181 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.861550:0:155181:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5000/0xbef42c0221d6e960 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 155181 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.861560:0:155181:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5000/0xbef42c0221d6e960 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 155181 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826016.861589:0:155181:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155181:1874663042881664:192.168.203.147@tcp:101:lfs.0 00000100:00100000:1.0:1787826016.861698:0:155181:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8000 going to sleep for 11 seconds 00000100:00100000:1.0:1787826016.868005:0:155181:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155181:1874663042881664:192.168.203.147@tcp:101:lfs.0 00010000:00010000:1.0:1787826016.868043:0:155181:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5000/0xbef42c0221d6e960 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea85c expref: -99 pid: 155181 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.868073:0:155181:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5000/0xbef42c0221d6e960 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea85c expref: -99 pid: 155181 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.868082:0:155181:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5000/0xbef42c0221d6e960 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea85c expref: -99 pid: 155181 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.868096:0:155181:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5000/0xbef42c0221d6e960 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cea85c expref: -99 pid: 155181 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.868183:0:155181:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5000/0xbef42c0221d6e960 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea85c expref: -99 pid: 155181 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.868201:0:155181:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5000/0xbef42c0221d6e960 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cea85c expref: -99 pid: 155181 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.868209:0:155181:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5000/0xbef42c0221d6e960 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cea85c expref: -99 pid: 155181 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.868217:0:155181:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5000/0xbef42c0221d6e960 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cea85c expref: -99 pid: 155181 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826016.868231:0:155181:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c35b2680 x1874663042881664/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826032 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826016.868246:0:155181:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5000/0xbef42c0221d6e960 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea85c expref: -99 pid: 155181 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.868259:0:155181:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/6112 (0 0) 00000080:00200000:1.0:1787826016.868267:0:155181:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787826016.868304:0:155181:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x17e0:0x0], updating i_size 4096 00000080:00010000:1.0:1787826016.868312:0:155181:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) for lock 0xbef42c0221d6e960 00000080:00010000:1.0:1787826016.868322:0:155181:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00000080:00010000:1.0:1787826016.868393:0:155181:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6e960 from it ffffb97501d83cb8 00010000:00010000:1.0:1787826016.868398:0:155181:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5000/0xbef42c0221d6e960 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cea85c expref: -99 pid: 155181 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.868413:0:155181:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5000/0xbef42c0221d6e960 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea85c expref: -99 pid: 155181 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:0.0:1787826016.870674:0:155181:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1787826016.870687:0:155181:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826016.870714:0:155181:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826016.871013:0:155181:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826016.871035:0:155181:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826016.871044:0:155181:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826016.871055:0:155181:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826016.871068:0:155181:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826016.871078:0:155181:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501d83cb8 00010000:00010000:0.0:1787826016.871082:0:155181:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826016.871097:0:155181:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826016.871612:0:155181:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826016.871626:0:155181:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826016.871636:0:155181:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826016.871645:0:155181:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826016.871652:0:155181:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826016.871657:0:155181:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826016.871660:0:155181:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826016.871663:0:155181:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501d83bf8 00010000:00010000:0.0:1787826016.871666:0:155181:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826016.871673:0:155181:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826016.871683:0:155181:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826016.871691:0:155181:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=267 00000080:00200000:0.0:1787826016.871701:0:155181:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 44 00000080:00200008:0.0:1787826016.871710:0:155181:0:(file.c:1090:ll_file_open()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00000080:00200080:0.0:1787826016.872028:0:155181:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) cmd=c03066f0 arg=7fffba89eec0 00000080:00200000:0.0:1787826016.872049:0:155181:0:(dir.c:451:ll_dir_setdirstripe()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) name=test stripe_offset=0 stripe_count=2, hash_type=0 00010000:00010000:0.0:1787826016.872095:0:155181:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5000/0xbef42c0221d6e960 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00cea85c expref: -99 pid: 155181 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826016.872124:0:155181:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6800/0xbef42c0221d64b3f lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9634 expref: -99 pid: 152202 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826016.872184:0:155181:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5000/0xbef42c0221d6e960 lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cea85c expref: -99 pid: 155181 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826016.872192:0:155181:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826016.872194:0:155181:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5000/0xbef42c0221d6e960 lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cea85c expref: -99 pid: 155181 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:0.0:1787826016.872215:0:155181:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787826016.872232:0:155181:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c55a5f80 x1874663042881920/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 552/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826016.872261:0:155181:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155181:1874663042881920:192.168.203.147@tcp:36:lfs.0 00000100:00100000:0.0:1787826016.872393:0:155181:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bbb40 going to sleep for 11 seconds 00000080:00200000:1.0:1787826016.912660:0:155182:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826016.912798:0:155182:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826016.912825:0:155182:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826016.912845:0:155182:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826016.912854:0:155182:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826016.912866:0:155182:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826016.912869:0:155182:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787826016.912879:0:155182:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501d8bc20 00010000:00010000:1.0:1787826016.912882:0:155182:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826016.912891:0:155182:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826016.912907:0:155182:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826016.912918:0:155182:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=24 00000080:00200000:1.0:1787826016.912926:0:155182:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826016.912940:0:155182:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=0 00000080:00200000:1.0:1787826016.912945:0:155182:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:1.0:1787826016.912960:0:155182:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787826016.913019:0:155182:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6e967 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155182 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.913027:0:155182:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6e967 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155182 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.913038:0:155182:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6e967 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155182 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826016.913177:0:155182:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155182:1874663042882048:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:1.0:1787826016.913305:0:155182:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8000 going to sleep for 11 seconds 00000100:00100000:1.0:1787826016.923477:0:155182:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155182:1874663042882048:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:1.0:1787826016.923514:0:155182:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6e967 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea8a2 expref: -99 pid: 155182 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.923541:0:155182:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6e967 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1ba4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea8a2 expref: -99 pid: 155182 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.923550:0:155182:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6e967 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1ba4:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea8a2 expref: -99 pid: 155182 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.923564:0:155182:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6e967 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1ba4:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cea8a2 expref: -99 pid: 155182 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.923577:0:155182:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6e967 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1ba4:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea8a2 expref: -99 pid: 155182 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.923590:0:155182:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6e967 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1ba4:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cea8a2 expref: -99 pid: 155182 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.923599:0:155182:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6e967 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1ba4:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cea8a2 expref: -99 pid: 155182 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.923607:0:155182:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6e967 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1ba4:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cea8a2 expref: -99 pid: 155182 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826016.923619:0:155182:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c35b2680 x1874663042882048/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/672 e 0 to 0 dl 1787826032 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826016.923635:0:155182:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6e967 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1ba4:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea8a2 expref: -99 pid: 155182 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.923646:0:155182:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/7076 (0 0) 00000080:00200000:1.0:1787826016.923654:0:155182:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00000002:00010000:1.0:1787826016.924032:0:155182:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200026560:0x18:0x0]) in obj [0x200026560:0x18:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826016.924100:0:155182:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6e96e lrc: 3/1,0 mode: --/CR res: [0x200026560:0x18:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 155182 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.924117:0:155182:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6e96e lrc: 3/1,0 mode: --/CR res: [0x200026560:0x18:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155182 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.924126:0:155182:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6e96e lrc: 3/1,0 mode: --/CR res: [0x200026560:0x18:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155182 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826016.924148:0:155182:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155182:1874663042882176:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:1.0:1787826016.924229:0:155182:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8000 going to sleep for 11 seconds 00000100:00100000:1.0:1787826016.934395:0:155182:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155182:1874663042882176:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:1.0:1787826016.934433:0:155182:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6e96e lrc: 4/1,0 mode: --/CR res: [0x200026560:0x18:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea8b0 expref: -99 pid: 155182 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.934447:0:155182:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6e96e lrc: 4/1,0 mode: --/PR res: [0x200026560:0x18:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea8b0 expref: -99 pid: 155182 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.934460:0:155182:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6e96e lrc: 4/1,0 mode: PR/PR res: [0x200026560:0x18:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cea8b0 expref: -99 pid: 155182 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.934473:0:155182:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6e96e lrc: 4/1,0 mode: PR/PR res: [0x200026560:0x18:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea8b0 expref: -99 pid: 155182 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.934486:0:155182:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6e96e lrc: 5/2,0 mode: PR/PR res: [0x200026560:0x18:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cea8b0 expref: -99 pid: 155182 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.934494:0:155182:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6e96e lrc: 5/2,0 mode: PR/PR res: [0x200026560:0x18:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cea8b0 expref: -99 pid: 155182 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.934502:0:155182:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6e96e lrc: 4/1,0 mode: PR/PR res: [0x200026560:0x18:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cea8b0 expref: -99 pid: 155182 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826016.934514:0:155182:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c35b1180 x1874663042882176/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787826032 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826016.934530:0:155182:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6e96e lrc: 3/1,0 mode: PR/PR res: [0x200026560:0x18:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea8b0 expref: -99 pid: 155182 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.934545:0:155182:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590091616/24 (0 0) 00010000:00010000:1.0:1787826016.934553:0:155182:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6e96e lrc: 3/1,0 mode: PR/PR res: [0x200026560:0x18:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cea8b0 expref: -99 pid: 155182 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.934561:0:155182:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6e96e lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x18:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea8b0 expref: -99 pid: 155182 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000002:00010000:1.0:1787826016.934594:0:155182:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001f58:0x18:0x0]) in obj [0x240001f58:0x18:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826016.934652:0:155182:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e975 lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x18:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 155182 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.934660:0:155182:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e975 lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x18:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155182 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.934670:0:155182:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e975 lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x18:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155182 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826016.935282:0:155182:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155182:1874663042882304:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:1.0:1787826016.935387:0:155182:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8000 going to sleep for 11 seconds 00000100:00100000:1.0:1787826016.973341:0:155182:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155182:1874663042882304:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:1.0:1787826016.973377:0:155182:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e975 lrc: 4/1,0 mode: --/CR res: [0x240001f58:0x18:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea8be expref: -99 pid: 155182 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.973392:0:155182:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e975 lrc: 4/1,0 mode: --/PR res: [0x240001f58:0x18:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea8be expref: -99 pid: 155182 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.973406:0:155182:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e975 lrc: 4/1,0 mode: PR/PR res: [0x240001f58:0x18:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cea8be expref: -99 pid: 155182 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.973417:0:155182:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e975 lrc: 4/1,0 mode: PR/PR res: [0x240001f58:0x18:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea8be expref: -99 pid: 155182 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.973433:0:155182:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e975 lrc: 5/2,0 mode: PR/PR res: [0x240001f58:0x18:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cea8be expref: -99 pid: 155182 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.973442:0:155182:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e975 lrc: 5/2,0 mode: PR/PR res: [0x240001f58:0x18:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cea8be expref: -99 pid: 155182 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.973450:0:155182:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e975 lrc: 4/1,0 mode: PR/PR res: [0x240001f58:0x18:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cea8be expref: -99 pid: 155182 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826016.973460:0:155182:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c35b1180 x1874663042882304/t0(0) o101->lustre-MDT0001-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787826032 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826016.973476:0:155182:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e975 lrc: 3/1,0 mode: PR/PR res: [0x240001f58:0x18:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea8be expref: -99 pid: 155182 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.973489:0:155182:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d0f54800 type 13 mode 4 res 9663684440/24 (0 0) 00010000:00010000:1.0:1787826016.973497:0:155182:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e975 lrc: 3/1,0 mode: PR/PR res: [0x240001f58:0x18:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cea8be expref: -99 pid: 155182 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.973505:0:155182:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e975 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x18:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea8be expref: -99 pid: 155182 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826016.973547:0:155182:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1ba4:0x0], updating i_size 8192 00000080:00200000:1.0:1787826016.973552:0:155182:0:(namei.c:147:ll_iget()) got inode: [0x20001a9e3:0x1ba4:0x0](ffff9181c83d42c8): rc = 0 00000080:00010000:1.0:1787826016.973563:0:155182:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x1ba4:0x0](ffff9181c83d42c8) for lock 0xbef42c0221d6e967 00000080:00010000:1.0:1787826016.973585:0:155182:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x1ba4:0x0](ffff9181c83d42c8) 00000080:00010000:1.0:1787826016.973589:0:155182:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6e967 from it ffffb97501d8be08 00010000:00010000:1.0:1787826016.973594:0:155182:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6e967 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1ba4:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cea8a2 expref: -99 pid: 155182 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826016.973603:0:155182:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6e967 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1ba4:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cea8a2 expref: -99 pid: 155182 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826016.973627:0:155182:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 3 00000080:00200000:1.0:1787826016.973637:0:155182:0:(namei.c:2328:ll_rmdir()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00010000:00010000:1.0:1787826016.973674:0:155182:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6e967 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1ba4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00cea8a2 expref: -99 pid: 155182 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826016.973792:0:155182:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6e967 lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x1ba4:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cea8a2 expref: -99 pid: 155182 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826016.973800:0:155182:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826016.973804:0:155182:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6e967 lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x1ba4:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cea8a2 expref: -99 pid: 155182 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:1.0:1787826016.973833:0:155182:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787826016.973841:0:155182:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c35b2680 x1874663042882560/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826016.973866:0:155182:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155182:1874663042882560:192.168.203.147@tcp:36:rmdir.0 00000100:00100000:1.0:1787826016.973946:0:155182:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8000 going to sleep for 11 seconds 00000100:00100000:0.0:1787826016.981312:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826016.981425:0:33333:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:1.0:1787826016.981451:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049033600 00000100:00100000:1.0:1787826016.981463:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049033600 00000100:00100000:1.0:1787826016.981537:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9181 00000100:00100000:1.0:1787826016.981550:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181e053c450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187904:x1874663049033600:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826016.981807:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6e96e lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x18:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cea8b0 expref: -99 pid: 155182 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826016.981959:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826016.981970:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181e053c450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187904:x1874663049033600:12345-192.168.203.147@tcp:104: Request processed in 420us (671us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826016.981978:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9181 00000100:00100000:0.0:1787826017.024983:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826017.025074:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049033856 00000100:00100000:1.0:1787826017.025089:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049033856 00000100:00100000:1.0:1787826017.025164:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9182 00000100:00100000:1.0:1787826017.025176:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663049033856:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826017.025200:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6e975 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x18:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cea8be expref: -99 pid: 155182 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826017.025296:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826017.025305:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663049033856:12345-192.168.203.147@tcp:104: Request processed in 130us (337us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826017.025314:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9182 00000100:00100000:1.0:1787826017.050514:0:155182:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155182:1874663042882560:192.168.203.147@tcp:36:rmdir.0 00000080:00200000:1.0:1787826017.050577:0:155182:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1ba4:0x0](ffff9181c83d42c8) 00000100:00100000:0.0:1787826017.173201:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826017.173265:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049035264 00000100:00100000:1.0:1787826017.173275:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049035264 00000100:00100000:1.0:1787826017.173333:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9183 00000100:00100000:1.0:1787826017.173342:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a6680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187906:x1874663049035264:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826017.173360:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481400/0xbef42c0221d6e998 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x19:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cea974 expref: -99 pid: 155184 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826017.173436:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826017.173443:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a6680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187906:x1874663049035264:12345-192.168.203.147@tcp:104: Request processed in 102us (252us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826017.173450:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9183 00010000:00010000:1.0:1787826017.175074:0:33329:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826017.175078:0:33329:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826017.175080:0:33329:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826017.194603:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826017.195048:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049035520 00000100:00100000:0.0:1787826017.195071:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049035520 00000100:00100000:0.0:1787826017.195165:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9184 00000100:00100000:0.0:1787826017.195177:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8414:x1874663049035520:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826017.195194:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6e99f lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x19:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cea982 expref: -99 pid: 155184 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826017.195279:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826017.195286:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8414:x1874663049035520:12345-192.168.203.147@tcp:104: Request processed in 113us (692us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826017.195293:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9184 00000080:00200000:0.0:1787826017.313182:0:155186:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826017.313210:0:155186:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826017.313228:0:155186:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826017.313245:0:155186:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826017.313253:0:155186:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826017.313261:0:155186:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826017.313265:0:155186:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826017.313273:0:155186:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501c6bc20 00010000:00010000:0.0:1787826017.313276:0:155186:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826017.313286:0:155186:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826017.313299:0:155186:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826017.313306:0:155186:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=24 00000080:00200000:0.0:1787826017.313311:0:155186:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826017.313321:0:155186:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=0 00000080:00200000:0.0:1787826017.313326:0:155186:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787826017.313337:0:155186:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787826017.313378:0:155186:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6e9bb lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155186 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.313386:0:155186:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6e9bb lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155186 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.313395:0:155186:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6e9bb lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155186 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826017.313418:0:155186:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155186:1874663042885120:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787826017.313502:0:155186:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bbb40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826017.328956:0:155186:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155186:1874663042885120:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:0.0:1787826017.328983:0:155186:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6e9bb lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceaa2a expref: -99 pid: 155186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.329018:0:155186:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6e9bb lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1ba6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceaa2a expref: -99 pid: 155186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.329027:0:155186:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6e9bb lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1ba6:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceaa2a expref: -99 pid: 155186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.329038:0:155186:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6e9bb lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1ba6:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceaa2a expref: -99 pid: 155186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.329048:0:155186:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6e9bb lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1ba6:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceaa2a expref: -99 pid: 155186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.329075:0:155186:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6e9bb lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1ba6:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceaa2a expref: -99 pid: 155186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.329082:0:155186:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6e9bb lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1ba6:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceaa2a expref: -99 pid: 155186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.329089:0:155186:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6e9bb lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1ba6:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceaa2a expref: -99 pid: 155186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826017.329100:0:155186:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c55a4380 x1874663042885120/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/672 e 0 to 0 dl 1787826033 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787826017.329113:0:155186:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6e9bb lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1ba6:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceaa2a expref: -99 pid: 155186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.329124:0:155186:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/7078 (0 0) 00000080:00200000:0.0:1787826017.329130:0:155186:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00000002:00010000:0.0:1787826017.329186:0:155186:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200026560:0x1a:0x0]) in obj [0x200026560:0x1a:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826017.329226:0:155186:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e9c2 lrc: 3/1,0 mode: --/CR res: [0x200026560:0x1a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 155186 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.329233:0:155186:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e9c2 lrc: 3/1,0 mode: --/CR res: [0x200026560:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155186 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.329242:0:155186:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e9c2 lrc: 3/1,0 mode: --/CR res: [0x200026560:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155186 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826017.329259:0:155186:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155186:1874663042885248:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787826017.329333:0:155186:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bbb40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826017.344798:0:155186:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155186:1874663042885248:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:0.0:1787826017.344838:0:155186:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e9c2 lrc: 4/1,0 mode: --/CR res: [0x200026560:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceaa38 expref: -99 pid: 155186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.344851:0:155186:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e9c2 lrc: 4/1,0 mode: --/PR res: [0x200026560:0x1a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceaa38 expref: -99 pid: 155186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.344864:0:155186:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e9c2 lrc: 4/1,0 mode: PR/PR res: [0x200026560:0x1a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceaa38 expref: -99 pid: 155186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.344878:0:155186:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e9c2 lrc: 4/1,0 mode: PR/PR res: [0x200026560:0x1a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceaa38 expref: -99 pid: 155186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.344891:0:155186:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e9c2 lrc: 5/2,0 mode: PR/PR res: [0x200026560:0x1a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceaa38 expref: -99 pid: 155186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.344900:0:155186:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e9c2 lrc: 5/2,0 mode: PR/PR res: [0x200026560:0x1a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceaa38 expref: -99 pid: 155186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.344907:0:155186:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e9c2 lrc: 4/1,0 mode: PR/PR res: [0x200026560:0x1a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceaa38 expref: -99 pid: 155186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826017.344918:0:155186:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c55a7100 x1874663042885248/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787826033 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787826017.344932:0:155186:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e9c2 lrc: 3/1,0 mode: PR/PR res: [0x200026560:0x1a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceaa38 expref: -99 pid: 155186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.344949:0:155186:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590091616/26 (0 0) 00010000:00010000:0.0:1787826017.344957:0:155186:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e9c2 lrc: 3/1,0 mode: PR/PR res: [0x200026560:0x1a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceaa38 expref: -99 pid: 155186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.344965:0:155186:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e9c2 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x1a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceaa38 expref: -99 pid: 155186 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000002:00010000:1.0:1787826017.345402:0:155186:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001f58:0x1a:0x0]) in obj [0x240001f58:0x1a:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826017.345452:0:155186:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6e9c9 lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x1a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 155186 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826017.345462:0:155186:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6e9c9 lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155186 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826017.345472:0:155186:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6e9c9 lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155186 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826017.345495:0:155186:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155186:1874663042885376:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:1.0:1787826017.345577:0:155186:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8b40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826017.370732:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826017.370912:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049036928 00000100:00100000:0.0:1787826017.370922:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049036928 00000100:00100000:0.0:1787826017.370983:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9185 00000100:00100000:0.0:1787826017.370993:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b1180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187906:x1874663049036928:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826017.371023:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6e9c2 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x1a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceaa38 expref: -99 pid: 155186 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826017.371107:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826017.371114:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b1180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187906:x1874663049036928:12345-192.168.203.147@tcp:104: Request processed in 122us (391us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826017.371121:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9185 00000100:00100000:0.0:1787826017.380733:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826017.380960:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049037184 00000100:00100000:1.0:1787826017.380967:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049037184 00000100:00100000:1.0:1787826017.381027:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9186 00000100:00100000:1.0:1787826017.381036:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a6300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663049037184:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826017.381052:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6e9c9 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x1a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceaa46 expref: -99 pid: 155186 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826017.381144:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826017.381151:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a6300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663049037184:12345-192.168.203.147@tcp:104: Request processed in 116us (427us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826017.381159:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9186 00000100:00100000:0.0:1787826017.487589:0:155187:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5cb6f00 going to sleep for 11 seconds 00000080:00200000:0.0:1787826017.536315:0:155195:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826017.536349:0:155195:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826017.536373:0:155195:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826017.536389:0:155195:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826017.536399:0:155195:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826017.536407:0:155195:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826017.536411:0:155195:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826017.536418:0:155195:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb975022bbc20 00010000:00010000:0.0:1787826017.536422:0:155195:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826017.536431:0:155195:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826017.536445:0:155195:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826017.536464:0:155195:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=24 00000080:00200000:0.0:1787826017.536471:0:155195:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826017.536485:0:155195:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=0 00000080:00200000:0.0:1787826017.536489:0:155195:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787826017.536501:0:155195:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787826017.536545:0:155195:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512a00/0xbef42c0221d6e9e5 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155195 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.536553:0:155195:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512a00/0xbef42c0221d6e9e5 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155195 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.536563:0:155195:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512a00/0xbef42c0221d6e9e5 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155195 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826017.536586:0:155195:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155195:1874663042886656:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787826017.537246:0:155195:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bb600 going to sleep for 11 seconds 00000100:00100000:1.0:1787826017.577605:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826017.577655:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049038592 00000100:00100000:1.0:1787826017.577666:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049038592 00000100:00100000:1.0:1787826017.577724:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9187 00000100:00100000:1.0:1787826017.577736:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187904:x1874663049038592:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826017.578125:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7a00/0xbef42c0221d6e9ec lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x1b:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceaafc expref: -99 pid: 155195 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826017.578207:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826017.578215:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187904:x1874663049038592:12345-192.168.203.147@tcp:104: Request processed in 482us (618us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826017.578222:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9187 00000100:00100000:1.0:1787826017.591680:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826017.591755:0:15150:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:1.0:1787826017.591773:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049038848 00000100:00100000:1.0:1787826017.591782:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049038848 00000100:00100000:1.0:1787826017.591831:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9188 00000100:00100000:1.0:1787826017.591840:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181e053c050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663049038848:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826017.591858:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6e9f3 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x1b:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceab0a expref: -99 pid: 155195 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826017.591935:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826017.591942:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181e053c050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663049038848:12345-192.168.203.147@tcp:104: Request processed in 103us (273us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826017.591949:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9188 00000100:00100000:0.0:1787826017.628020:0:155195:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f2680 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155195:1874663042887168:192.168.203.147@tcp:36:rmdir.0 00000080:00200000:0.0:1787826017.628074:0:155195:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1ba7:0x0](ffff9181c83ddb88) 00000080:00200000:0.0:1787826017.648422:0:155199:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826017.648451:0:155199:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826017.648468:0:155199:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826017.648484:0:155199:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826017.648492:0:155199:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826017.648502:0:155199:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826017.648506:0:155199:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826017.648512:0:155199:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501d73b58 00010000:00010000:0.0:1787826017.648516:0:155199:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826017.648526:0:155199:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826017.648539:0:155199:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826017.648545:0:155199:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=29 00000080:00200000:0.0:1787826017.648551:0:155199:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826017.648563:0:155199:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=13 00000080:00200000:0.0:1787826017.648567:0:155199:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787826017.648579:0:155199:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787826017.648621:0:155199:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6e9fa lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155199 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.648953:0:155199:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6e9fa lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155199 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.648969:0:155199:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6e9fa lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155199 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826017.648996:0:155199:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155199:1874663042887552:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787826017.649453:0:155199:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bb480 going to sleep for 11 seconds 00000100:00100000:0.0:1787826017.673583:0:155199:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155199:1874663042887552:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787826017.673606:0:155199:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6e9fa lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155199 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.673620:0:155199:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6e9fa lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155199 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.673626:0:155199:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6e9fa lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155199 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.673632:0:155199:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6e9fa lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155199 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.673640:0:155199:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6e9fa lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155199 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.673646:0:155199:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0512e00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787826017.673650:0:155199:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6e9fa lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155199 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.673657:0:155199:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6e9fa lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155199 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.673664:0:155199:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6e9fa lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 155199 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.673671:0:155199:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787826017.673685:0:155199:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6e9fa lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155199 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.673691:0:155199:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6e9fa lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155199 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826017.673717:0:155199:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c55a4e00 x1874663042887552/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787826033 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787826017.673731:0:155199:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787826017.673793:0:155199:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787826017.673860:0:155199:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826017.673873:0:155199:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826017.673882:0:155199:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826017.673904:0:155199:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826017.673911:0:155199:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826017.673916:0:155199:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826017.673919:0:155199:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826017.673924:0:155199:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501d73be8 00010000:00010000:0.0:1787826017.673927:0:155199:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826017.673936:0:155199:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826017.673947:0:155199:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826017.673954:0:155199:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=16408 00000080:00200000:0.0:1787826017.673958:0:155199:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826017.673971:0:155199:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=16392 00000080:00200000:0.0:1787826017.673975:0:155199:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787826017.673985:0:155199:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787826017.674021:0:155199:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6ea01 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155199 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.674028:0:155199:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6ea01 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155199 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.674036:0:155199:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6ea01 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155199 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826017.674066:0:155199:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155199:1874663042887680:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787826017.674257:0:155199:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bb480 going to sleep for 11 seconds 00000100:00100000:0.0:1787826017.681609:0:155199:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155199:1874663042887680:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787826017.681633:0:155199:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6ea01 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155199 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.681647:0:155199:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6ea01 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155199 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.681659:0:155199:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6ea01 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155199 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.681666:0:155199:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6ea01 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155199 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.681685:0:155199:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6ea01 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155199 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.681691:0:155199:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0512600 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787826017.681701:0:155199:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6ea01 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155199 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.681708:0:155199:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6ea01 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155199 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.681715:0:155199:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6ea01 lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 155199 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.681722:0:155199:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787826017.681728:0:155199:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6ea01 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155199 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.681735:0:155199:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6ea01 lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155199 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826017.681775:0:155199:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c55a4e00 x1874663042887680/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787826033 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787826017.681789:0:155199:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787826017.681818:0:155199:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787826017.681904:0:155199:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826017.681919:0:155199:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826017.681928:0:155199:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826017.681938:0:155199:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826017.681944:0:155199:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826017.681949:0:155199:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826017.681952:0:155199:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826017.681957:0:155199:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501d73b58 00010000:00010000:0.0:1787826017.681960:0:155199:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826017.681967:0:155199:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826017.681979:0:155199:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826017.681985:0:155199:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=13 00000080:00200008:0.0:1787826017.681994:0:155199:0:(file.c:6296:ll_getattr_dentry()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1787826017.681999:0:155199:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08),name=d71a.replay-single 00000002:00010000:0.0:1787826017.682010:0:155199:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x20001a9e3:0x17e0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826017.682016:0:155199:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00010000:00010000:0.0:1787826017.682050:0:155199:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6ea08 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 155199 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.682073:0:155199:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6ea08 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 155199 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.682081:0:155199:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6ea08 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 155199 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826017.682099:0:155199:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155199:1874663042887808:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787826017.682175:0:155199:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bb480 going to sleep for 11 seconds 00000100:00100000:0.0:1787826017.695950:0:155199:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155199:1874663042887808:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787826017.695975:0:155199:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6ea08 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceab6c expref: -99 pid: 155199 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.695993:0:155199:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6ea08 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceab6c expref: -99 pid: 155199 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.696001:0:155199:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6ea08 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceab6c expref: -99 pid: 155199 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.696011:0:155199:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6ea08 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceab6c expref: -99 pid: 155199 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.696023:0:155199:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6ea08 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceab6c expref: -99 pid: 155199 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.696035:0:155199:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6ea08 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceab6c expref: -99 pid: 155199 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.696049:0:155199:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6ea08 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceab6c expref: -99 pid: 155199 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.696066:0:155199:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6ea08 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceab6c expref: -99 pid: 155199 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826017.696078:0:155199:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c55a4e00 x1874663042887808/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826033 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787826017.696092:0:155199:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6ea08 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceab6c expref: -99 pid: 155199 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.696102:0:155199:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787826017.696116:0:155199:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1787826017.696145:0:155199:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x17e0:0x0], updating i_size 4096 00000080:00010000:0.0:1787826017.696153:0:155199:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) for lock 0xbef42c0221d6ea08 00000080:00010000:0.0:1787826017.696161:0:155199:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00000080:00010000:0.0:1787826017.696164:0:155199:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6ea08 from it ffffb97501d73cb8 00010000:00010000:0.0:1787826017.696168:0:155199:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6ea08 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceab6c expref: -99 pid: 155199 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.696176:0:155199:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6ea08 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceab6c expref: -99 pid: 155199 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:0.0:1787826017.696703:0:155199:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1787826017.696709:0:155199:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826017.696726:0:155199:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826017.696736:0:155199:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826017.696777:0:155199:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826017.696784:0:155199:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826017.696789:0:155199:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826017.696792:0:155199:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826017.696796:0:155199:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501d73cb8 00010000:00010000:0.0:1787826017.696799:0:155199:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826017.696805:0:155199:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826017.696919:0:155199:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826017.696928:0:155199:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826017.696935:0:155199:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826017.696942:0:155199:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826017.696948:0:155199:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826017.696952:0:155199:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826017.696954:0:155199:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826017.696958:0:155199:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501d73bf8 00010000:00010000:0.0:1787826017.696961:0:155199:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826017.696967:0:155199:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826017.696976:0:155199:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826017.696981:0:155199:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=267 00000080:00200000:0.0:1787826017.696985:0:155199:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 44 00000080:00200008:0.0:1787826017.696990:0:155199:0:(file.c:1090:ll_file_open()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00000080:00200080:0.0:1787826017.697010:0:155199:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) cmd=c03066f0 arg=7fffccfcc470 00000080:00200000:0.0:1787826017.697021:0:155199:0:(dir.c:451:ll_dir_setdirstripe()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) name=test stripe_offset=0 stripe_count=2, hash_type=0 00010000:00010000:0.0:1787826017.697044:0:155199:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6ea08 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ceab6c expref: -99 pid: 155199 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826017.697070:0:155199:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6800/0xbef42c0221d64b3f lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9634 expref: -99 pid: 152202 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826017.697106:0:155199:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6ea08 lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ceab6c expref: -99 pid: 155199 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826017.697113:0:155199:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826017.697115:0:155199:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6ea08 lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ceab6c expref: -99 pid: 155199 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:0.0:1787826017.697132:0:155199:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787826017.697141:0:155199:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c55a4e00 x1874663042888064/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 552/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826017.697166:0:155199:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155199:1874663042888064:192.168.203.147@tcp:36:lfs.0 00000100:00100000:0.0:1787826017.697234:0:155199:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bb480 going to sleep for 11 seconds 00000100:00100000:0.0:1787826017.716967:0:155199:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155199:1874663042888064:192.168.203.147@tcp:36:lfs.0 00000080:00200000:0.0:1787826017.717028:0:155199:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x20001a9e3:0x1ba8:0x0](ffff9181c83a0908) 00000080:00200000:0.0:1787826017.717035:0:155199:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1ba8:0x0], updating i_size 4096 00000080:00200008:0.0:1787826017.717079:0:155199:0:(file.c:479:ll_file_release()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00010000:00010000:0.0:1787826017.717112:0:155199:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513000/0xbef42c0221d64bee lrc: 2/0,0 mode: CR/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce962d expref: -99 pid: 152210 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000100:00100000:0.0:1787826017.794531:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826017.794571:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049040256 00000100:00100000:0.0:1787826017.794580:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049040256 00000100:00100000:0.0:1787826017.794628:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9189 00000100:00100000:0.0:1787826017.794638:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187904:x1874663049040256:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826017.794654:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480200/0xbef42c0221d6ea16 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x1c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceabc0 expref: -99 pid: 155203 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826017.794788:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826017.794796:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187904:x1874663049040256:12345-192.168.203.147@tcp:104: Request processed in 159us (273us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826017.794802:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9189 00000100:00100000:0.0:1787826017.805352:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826017.805454:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049040512 00000100:00100000:1.0:1787826017.805464:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049040512 00000100:00100000:1.0:1787826017.805519:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9190 00000100:00100000:1.0:1787826017.805530:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8414:x1874663049040512:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826017.805546:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481000/0xbef42c0221d6ea1d lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x1c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceabce expref: -99 pid: 155203 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826017.805624:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826017.805631:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8414:x1874663049040512:12345-192.168.203.147@tcp:104: Request processed in 103us (289us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826017.805638:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9190 00010000:00010000:1.0:1787826017.805661:0:15161:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826017.805667:0:15161:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481000/0xbef42c0221d6ea1d lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x1c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceabce expref: -99 pid: 155203 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826017.805675:0:15161:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3481000 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:1.0:1787826017.805679:0:15161:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481000/0xbef42c0221d6ea1d lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x1c:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00ceabce expref: -99 pid: 155203 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826017.805685:0:15161:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481000/0xbef42c0221d6ea1d lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x1c:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00ceabce expref: -99 pid: 155203 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826017.805698:0:15161:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481000/0xbef42c0221d6ea1d lrc: 3/0,0 mode: PR/PR res: [0x240001f58:0x1c:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x408400000000 nid: local remote: 0x878a5eea00ceabce expref: -99 pid: 155203 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826017.805765:0:15161:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481000/0xbef42c0221d6ea1d lrc: 2/0,0 mode: --/PR res: [0x240001f58:0x1c:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ceabce expref: -99 pid: 155203 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826017.805773:0:15161:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826017.805793:0:15161:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481000/0xbef42c0221d6ea1d lrc: 1/0,0 mode: --/PR res: [0x240001f58:0x1c:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ceabce expref: -99 pid: 155203 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826017.805800:0:15161:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481000/0xbef42c0221d6ea1d lrc: 0/0,0 mode: --/PR res: [0x240001f58:0x1c:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ceabce expref: -99 pid: 155203 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826017.805819:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826017.805821:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826017.805822:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826017.805836:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042888960:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1787826017.821831:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042888960:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787826017.901471:0:155209:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c494f800 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155209:1874663042889344:192.168.203.147@tcp:101:lfs.0 00010000:00010000:1.0:1787826017.901499:0:155209:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480600/0xbef42c0221d6ea32 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceac30 expref: -99 pid: 155209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826017.901516:0:155209:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480600/0xbef42c0221d6ea32 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceac30 expref: -99 pid: 155209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826017.901524:0:155209:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480600/0xbef42c0221d6ea32 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceac30 expref: -99 pid: 155209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826017.901541:0:155209:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480600/0xbef42c0221d6ea32 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceac30 expref: -99 pid: 155209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826017.901551:0:155209:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480600/0xbef42c0221d6ea32 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceac30 expref: -99 pid: 155209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826017.901563:0:155209:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480600/0xbef42c0221d6ea32 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceac30 expref: -99 pid: 155209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826017.901571:0:155209:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480600/0xbef42c0221d6ea32 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceac30 expref: -99 pid: 155209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826017.901578:0:155209:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480600/0xbef42c0221d6ea32 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceac30 expref: -99 pid: 155209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826017.901590:0:155209:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c494f800 x1874663042889344/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826033 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826017.901604:0:155209:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480600/0xbef42c0221d6ea32 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceac30 expref: -99 pid: 155209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826017.901614:0:155209:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/6112 (0 0) 00000080:00200000:1.0:1787826017.901620:0:155209:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787826017.901646:0:155209:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x17e0:0x0], updating i_size 4096 00000080:00010000:1.0:1787826017.901652:0:155209:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) for lock 0xbef42c0221d6ea32 00000080:00010000:1.0:1787826017.901659:0:155209:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00000080:00010000:1.0:1787826017.901662:0:155209:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6ea32 from it ffffb975022bbcb8 00010000:00010000:1.0:1787826017.901666:0:155209:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480600/0xbef42c0221d6ea32 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceac30 expref: -99 pid: 155209 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826017.901675:0:155209:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480600/0xbef42c0221d6ea32 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceac30 expref: -99 pid: 155209 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:1.0:1787826017.902443:0:155209:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), request_mask 0, flags 0, foreign 0 00000080:00200000:1.0:1787826017.902451:0:155209:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826017.902468:0:155209:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826017.902479:0:155209:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826017.902491:0:155209:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826017.902498:0:155209:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826017.902502:0:155209:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826017.902505:0:155209:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787826017.902509:0:155209:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb975022bbcb8 00010000:00010000:1.0:1787826017.902513:0:155209:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826017.902522:0:155209:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826017.911817:0:155209:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826017.911844:0:155209:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826017.911861:0:155209:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826017.911877:0:155209:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826017.911886:0:155209:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826017.911893:0:155209:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826017.911897:0:155209:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787826017.911905:0:155209:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb975022bbbf8 00010000:00010000:1.0:1787826017.911909:0:155209:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826017.911917:0:155209:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826017.911933:0:155209:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826017.911941:0:155209:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=267 00000080:00200000:1.0:1787826017.911947:0:155209:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 44 00000080:00200008:1.0:1787826017.911952:0:155209:0:(file.c:1090:ll_file_open()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00000080:00200080:1.0:1787826017.911972:0:155209:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) cmd=c03066f0 arg=7ffe4f417d00 00000080:00200000:1.0:1787826017.911984:0:155209:0:(dir.c:451:ll_dir_setdirstripe()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) name=test stripe_offset=0 stripe_count=2, hash_type=0 00010000:00010000:1.0:1787826017.912009:0:155209:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480600/0xbef42c0221d6ea32 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ceac30 expref: -99 pid: 155209 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826017.912023:0:155209:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6800/0xbef42c0221d64b3f lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9634 expref: -99 pid: 152202 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826017.912076:0:155209:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480600/0xbef42c0221d6ea32 lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ceac30 expref: -99 pid: 155209 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826017.912085:0:155209:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826017.912088:0:155209:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480600/0xbef42c0221d6ea32 lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ceac30 expref: -99 pid: 155209 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:1.0:1787826017.912116:0:155209:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787826017.912130:0:155209:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c35b1180 x1874663042889600/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 552/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826017.912153:0:155209:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155209:1874663042889600:192.168.203.147@tcp:36:lfs.0 00000100:00100000:1.0:1787826017.912239:0:155209:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826017.928815:0:155209:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155209:1874663042889600:192.168.203.147@tcp:36:lfs.0 00000080:00200000:1.0:1787826017.928866:0:155209:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x20001a9e3:0x1ba9:0x0](ffff9181c839a1c8) 00000080:00200000:1.0:1787826017.928872:0:155209:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1ba9:0x0], updating i_size 4096 00000080:00200008:1.0:1787826017.928896:0:155209:0:(file.c:479:ll_file_release()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00010000:00010000:1.0:1787826017.928923:0:155209:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513000/0xbef42c0221d64bee lrc: 2/0,0 mode: CR/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce962d expref: -99 pid: 152210 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826017.935328:0:155216:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826017.935358:0:155216:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826017.935383:0:155216:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826017.935402:0:155216:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826017.935411:0:155216:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826017.935422:0:155216:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826017.935427:0:155216:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826017.935436:0:155216:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501e03c20 00010000:00010000:0.0:1787826017.935442:0:155216:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826017.935452:0:155216:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826017.935465:0:155216:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826017.935472:0:155216:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=24 00000080:00200000:0.0:1787826017.935477:0:155216:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826017.935488:0:155216:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=0 00000080:00200000:0.0:1787826017.935492:0:155216:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787826017.935503:0:155216:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787826017.935545:0:155216:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6ea39 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155216 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.935553:0:155216:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6ea39 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155216 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826017.935562:0:155216:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6ea39 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155216 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826017.935595:0:155216:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155216:1874663042889728:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787826017.935893:0:155216:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bb840 going to sleep for 11 seconds 00000100:00100000:1.0:1787826017.982007:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826017.982059:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049041920 00000100:00100000:1.0:1787826017.982069:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049041920 00000100:00100000:1.0:1787826017.982130:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9191 00000100:00100000:1.0:1787826017.982141:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187904:x1874663049041920:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826017.982159:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6ea40 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x1d:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceac84 expref: -99 pid: 155216 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826017.982232:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826017.982240:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187904:x1874663049041920:12345-192.168.203.147@tcp:104: Request processed in 101us (241us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826017.982247:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9191 00000100:00100000:1.0:1787826017.990427:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826017.990542:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049042176 00000100:00100000:0.0:1787826017.990551:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049042176 00000100:00100000:0.0:1787826017.990604:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9192 00000100:00100000:0.0:1787826017.990614:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b3480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663049042176:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826017.990629:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6ea47 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x1d:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceac92 expref: -99 pid: 155216 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826017.990716:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826017.990724:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b3480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663049042176:12345-192.168.203.147@tcp:104: Request processed in 111us (304us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826017.990730:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9192 00010000:00010000:0.0:1787826017.990773:0:15160:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826017.990779:0:15160:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6ea47 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x1d:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceac92 expref: -99 pid: 155216 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826017.990787:0:15160:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e06f6000 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787826017.990792:0:15160:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6ea47 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x1d:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00ceac92 expref: -99 pid: 155216 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826017.990801:0:15160:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6ea47 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x1d:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00ceac92 expref: -99 pid: 155216 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826017.990810:0:15160:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6ea47 lrc: 3/0,0 mode: PR/PR res: [0x240001f58:0x1d:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x408400000000 nid: local remote: 0x878a5eea00ceac92 expref: -99 pid: 155216 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826017.990869:0:15160:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6ea47 lrc: 2/0,0 mode: --/PR res: [0x240001f58:0x1d:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ceac92 expref: -99 pid: 155216 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826017.990877:0:15160:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826017.990898:0:15160:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6ea47 lrc: 1/0,0 mode: --/PR res: [0x240001f58:0x1d:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ceac92 expref: -99 pid: 155216 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826017.990905:0:15160:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6ea47 lrc: 0/0,0 mode: --/PR res: [0x240001f58:0x1d:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ceac92 expref: -99 pid: 155216 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826017.990925:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826017.990926:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826017.990928:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826017.990942:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042890496:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1787826018.010471:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042890496:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787826018.092014:0:155233:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f3480 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155233:1874663042891264:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:1.0:1787826018.092039:0:155233:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6ea63 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cead3a expref: -99 pid: 155233 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826018.092074:0:155233:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6ea63 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1baa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cead3a expref: -99 pid: 155233 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826018.092083:0:155233:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6ea63 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1baa:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cead3a expref: -99 pid: 155233 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826018.092093:0:155233:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6ea63 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1baa:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cead3a expref: -99 pid: 155233 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826018.092102:0:155233:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6ea63 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1baa:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cead3a expref: -99 pid: 155233 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826018.092115:0:155233:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6ea63 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1baa:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cead3a expref: -99 pid: 155233 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826018.092123:0:155233:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6ea63 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1baa:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cead3a expref: -99 pid: 155233 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826018.092130:0:155233:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6ea63 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1baa:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cead3a expref: -99 pid: 155233 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826018.092142:0:155233:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c71f3480 x1874663042891264/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/672 e 0 to 0 dl 1787826034 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826018.092155:0:155233:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6ea63 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1baa:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cead3a expref: -99 pid: 155233 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826018.092165:0:155233:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/7082 (0 0) 00000080:00200000:1.0:1787826018.092170:0:155233:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00000002:00010000:1.0:1787826018.092244:0:155233:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200026560:0x1e:0x0]) in obj [0x200026560:0x1e:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826018.092298:0:155233:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4200/0xbef42c0221d6ea6a lrc: 3/1,0 mode: --/CR res: [0x200026560:0x1e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 155233 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826018.092306:0:155233:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4200/0xbef42c0221d6ea6a lrc: 3/1,0 mode: --/CR res: [0x200026560:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155233 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826018.092313:0:155233:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4200/0xbef42c0221d6ea6a lrc: 3/1,0 mode: --/CR res: [0x200026560:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155233 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826018.092331:0:155233:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155233:1874663042891392:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:1.0:1787826018.092491:0:155233:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8d80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826018.129395:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826018.129450:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049043584 00000100:00100000:0.0:1787826018.129460:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049043584 00000100:00100000:0.0:1787826018.129509:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9193 00000100:00100000:0.0:1787826018.129518:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a6300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187904:x1874663049043584:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826018.129535:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4200/0xbef42c0221d6ea6a lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x1e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cead48 expref: -99 pid: 155233 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826018.129618:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826018.129633:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a6300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187904:x1874663049043584:12345-192.168.203.147@tcp:104: Request processed in 108us (240us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826018.129641:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9193 00000100:00100000:0.0:1787826018.142905:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826018.142946:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049043840 00000100:00100000:0.0:1787826018.142952:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049043840 00000100:00100000:0.0:1787826018.143013:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9194 00000100:00100000:0.0:1787826018.143023:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8414:x1874663049043840:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826018.143040:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6ea71 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x1e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cead56 expref: -99 pid: 155233 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826018.143112:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826018.143119:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8414:x1874663049043840:12345-192.168.203.147@tcp:104: Request processed in 98us (225us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826018.143126:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9194 00000100:00100000:1.0:1787826018.199975:0:155233:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9181c4d2b000: committing for last_committed 335007449145 gen 11 00000100:00100000:1.0:1787826018.200155:0:155233:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c494d880 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155233:1874663042891776:192.168.203.147@tcp:36:rmdir.0 00000080:00200000:1.0:1787826018.200197:0:155233:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1baa:0x0](ffff9181c83d63c8) 00000080:00200000:1.0:1787826018.524015:0:155258:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826018.524059:0:155258:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826018.524078:0:155258:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826018.524101:0:155258:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826018.524109:0:155258:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826018.524117:0:155258:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826018.524121:0:155258:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787826018.524129:0:155258:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501eabc20 00010000:00010000:1.0:1787826018.524133:0:155258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826018.524142:0:155258:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826018.524157:0:155258:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826018.524165:0:155258:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=24 00000080:00200000:1.0:1787826018.524171:0:155258:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826018.524183:0:155258:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=0 00000080:00200000:1.0:1787826018.524187:0:155258:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:1.0:1787826018.524200:0:155258:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787826018.524247:0:155258:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6ea8d lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155258 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826018.524256:0:155258:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6ea8d lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155258 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826018.524266:0:155258:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6ea8d lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155258 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826018.524296:0:155258:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155258:1874663042892800:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:1.0:1787826018.524438:0:155258:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8e40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826018.559391:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826018.559555:0:15151:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:1.0:1787826018.559573:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049131136 00000100:00100000:1.0:1787826018.559581:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049131136 00000100:00100000:1.0:1787826018.559630:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9195 00000100:00100000:1.0:1787826018.559640:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181e053f050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:188682:x1874663049131136:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826018.559658:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6ea94 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x1f:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceae0c expref: -99 pid: 155258 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826018.560377:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826018.560389:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181e053f050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:188682:x1874663049131136:12345-192.168.203.147@tcp:104: Request processed in 748us (1006us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826018.560398:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9195 00010000:00010000:0.0:1787826018.560612:0:15160:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826018.560621:0:15160:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6ea94 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x1f:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceae0c expref: -99 pid: 155258 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826018.560631:0:15160:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3480a00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787826018.560635:0:15160:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6ea94 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x1f:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00ceae0c expref: -99 pid: 155258 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826018.560644:0:15160:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6ea94 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x1f:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00ceae0c expref: -99 pid: 155258 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826018.560653:0:15160:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6ea94 lrc: 3/0,0 mode: PR/PR res: [0x200026560:0x1f:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x408400000000 nid: local remote: 0x878a5eea00ceae0c expref: -99 pid: 155258 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826018.560891:0:15160:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6ea94 lrc: 2/0,0 mode: --/PR res: [0x200026560:0x1f:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ceae0c expref: -99 pid: 155258 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826018.560899:0:15160:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826018.560921:0:15160:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6ea94 lrc: 1/0,0 mode: --/PR res: [0x200026560:0x1f:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ceae0c expref: -99 pid: 155258 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826018.560929:0:15160:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6ea94 lrc: 0/0,0 mode: --/PR res: [0x200026560:0x1f:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ceae0c expref: -99 pid: 155258 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826018.560947:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826018.560949:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826018.560951:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826018.560967:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042893440:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787826018.578812:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826018.578850:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049131392 00000100:00100000:1.0:1787826018.578858:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049131392 00000100:00100000:1.0:1787826018.578906:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9196 00000100:00100000:1.0:1787826018.578916:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663049131392:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826018.578933:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6ea9b lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x1f:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceae1a expref: -99 pid: 155258 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826018.579003:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826018.579011:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663049131392:12345-192.168.203.147@tcp:104: Request processed in 96us (206us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826018.579018:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9196 00000100:00100000:0.0:1787826018.579605:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042893440:192.168.203.147@tcp:103:ldlm_bl.0 00000080:00200000:1.0:1787826018.636610:0:155266:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826018.636642:0:155266:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826018.636663:0:155266:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826018.636883:0:155266:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826018.636893:0:155266:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826018.636902:0:155266:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826018.636905:0:155266:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787826018.636912:0:155266:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501e9bb58 00010000:00010000:1.0:1787826018.636916:0:155266:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826018.636925:0:155266:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826018.636939:0:155266:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826018.636947:0:155266:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=29 00000080:00200000:1.0:1787826018.636952:0:155266:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826018.637216:0:155266:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=13 00000080:00200000:1.0:1787826018.637222:0:155266:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:1.0:1787826018.637237:0:155266:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787826018.637281:0:155266:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6eaa2 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155266 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826018.637295:0:155266:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6eaa2 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155266 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826018.637304:0:155266:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6eaa2 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155266 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826018.637328:0:155266:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155266:1874663042893696:192.168.203.147@tcp:101:lfs.0 00000100:00100000:1.0:1787826018.637431:0:155266:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826018.705373:0:155274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f0e00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155274:1874663042894336:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:1.0:1787826018.705395:0:155274:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7800/0xbef42c0221d6eab7 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceaec2 expref: -99 pid: 155274 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826018.705413:0:155274:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7800/0xbef42c0221d6eab7 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1bac:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceaec2 expref: -99 pid: 155274 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826018.705419:0:155274:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7800/0xbef42c0221d6eab7 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1bac:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceaec2 expref: -99 pid: 155274 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826018.705427:0:155274:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7800/0xbef42c0221d6eab7 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bac:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceaec2 expref: -99 pid: 155274 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826018.705436:0:155274:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7800/0xbef42c0221d6eab7 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bac:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceaec2 expref: -99 pid: 155274 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826018.705446:0:155274:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7800/0xbef42c0221d6eab7 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1bac:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceaec2 expref: -99 pid: 155274 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826018.705509:0:155274:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7800/0xbef42c0221d6eab7 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1bac:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceaec2 expref: -99 pid: 155274 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826018.705514:0:155274:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7800/0xbef42c0221d6eab7 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bac:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceaec2 expref: -99 pid: 155274 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826018.705523:0:155274:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c71f0e00 x1874663042894336/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/672 e 0 to 0 dl 1787826034 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826018.705536:0:155274:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7800/0xbef42c0221d6eab7 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1bac:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceaec2 expref: -99 pid: 155274 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826018.705546:0:155274:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/7084 (0 0) 00000080:00200000:1.0:1787826018.705562:0:155274:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00000002:00010000:1.0:1787826018.705612:0:155274:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200026560:0x20:0x0]) in obj [0x200026560:0x20:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826018.705781:0:155274:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6eabe lrc: 3/1,0 mode: --/CR res: [0x200026560:0x20:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 155274 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826018.705790:0:155274:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6eabe lrc: 3/1,0 mode: --/CR res: [0x200026560:0x20:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155274 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826018.705799:0:155274:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6eabe lrc: 3/1,0 mode: --/CR res: [0x200026560:0x20:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155274 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826018.705820:0:155274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155274:1874663042894464:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:1.0:1787826018.705894:0:155274:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8840 going to sleep for 11 seconds 00000100:00100000:1.0:1787826018.739021:0:155274:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5cf9240 going to sleep for 11 seconds 00000100:00100000:1.0:1787826018.751002:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826018.751043:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049132800 00000100:00100000:1.0:1787826018.751062:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049132800 00000100:00100000:1.0:1787826018.751107:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9197 00000100:00100000:1.0:1787826018.751117:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b1880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187904:x1874663049132800:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826018.751133:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6eabe lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x20:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceaed0 expref: -99 pid: 155274 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826018.751361:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826018.751369:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b1880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187904:x1874663049132800:12345-192.168.203.147@tcp:104: Request processed in 253us (376us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826018.751376:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9197 00000100:00100000:0.0:1787826018.764986:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826018.765027:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049133056 00000100:00100000:0.0:1787826018.765036:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049133056 00000100:00100000:0.0:1787826018.765089:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9198 00000100:00100000:0.0:1787826018.765099:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663049133056:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826018.765115:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6eac5 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x20:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceaede expref: -99 pid: 155274 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826018.765178:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826018.765184:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663049133056:12345-192.168.203.147@tcp:104: Request processed in 86us (207us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826018.765193:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9198 00010000:00010000:0.0:1787826018.765211:0:33329:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826018.765217:0:33329:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6eac5 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x20:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceaede expref: -99 pid: 155274 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826018.765225:0:33329:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e06f6e00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787826018.765230:0:33329:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6eac5 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x20:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00ceaede expref: -99 pid: 155274 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826018.765239:0:33329:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6eac5 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x20:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00ceaede expref: -99 pid: 155274 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826018.765247:0:33329:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6eac5 lrc: 3/0,0 mode: PR/PR res: [0x240001f58:0x20:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x408400000000 nid: local remote: 0x878a5eea00ceaede expref: -99 pid: 155274 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826018.765296:0:33329:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6eac5 lrc: 2/0,0 mode: --/PR res: [0x240001f58:0x20:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ceaede expref: -99 pid: 155274 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826018.765302:0:33329:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826018.765319:0:33329:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6eac5 lrc: 1/0,0 mode: --/PR res: [0x240001f58:0x20:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ceaede expref: -99 pid: 155274 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826018.765326:0:33329:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6eac5 lrc: 0/0,0 mode: --/PR res: [0x240001f58:0x20:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ceaede expref: -99 pid: 155274 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826018.765343:0:33329:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826018.765345:0:33329:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826018.765346:0:33329:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826018.765359:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042895104:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787826018.781939:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042895104:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787826018.793296:0:155274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f0e00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155274:1874663042894848:192.168.203.147@tcp:36:rmdir.0 00000080:00200000:1.0:1787826018.793352:0:155274:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1bac:0x0](ffff9181c83d8908) 00000080:00200000:1.0:1787826018.806776:0:155291:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826018.806805:0:155291:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826018.806823:0:155291:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826018.806840:0:155291:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826018.806847:0:155291:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826018.806855:0:155291:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826018.806859:0:155291:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787826018.806866:0:155291:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501ef3b58 00010000:00010000:1.0:1787826018.806871:0:155291:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826018.806879:0:155291:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826018.806893:0:155291:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826018.806901:0:155291:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=29 00000080:00200000:1.0:1787826018.806906:0:155291:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826018.806916:0:155291:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=13 00000080:00200000:1.0:1787826018.806921:0:155291:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:1.0:1787826018.806934:0:155291:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787826018.806974:0:155291:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6eacc lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155291 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826018.806982:0:155291:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6eacc lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155291 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826018.806992:0:155291:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6eacc lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155291 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826018.807017:0:155291:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155291:1874663042895232:192.168.203.147@tcp:101:lfs.0 00000100:00100000:1.0:1787826018.807105:0:155291:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8180 going to sleep for 11 seconds 00000100:00100000:1.0:1787826018.821289:0:155291:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155291:1874663042895232:192.168.203.147@tcp:101:lfs.0 00010000:00010000:1.0:1787826018.821309:0:155291:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6eacc lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155291 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826018.821323:0:155291:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6eacc lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155291 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826018.821335:0:155291:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6eacc lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155291 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826018.821343:0:155291:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6eacc lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155291 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826018.821351:0:155291:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6eacc lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155291 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826018.821357:0:155291:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4a00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:1.0:1787826018.821361:0:155291:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6eacc lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155291 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826018.821367:0:155291:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6eacc lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155291 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826018.821374:0:155291:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6eacc lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 155291 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826018.821381:0:155291:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787826018.821386:0:155291:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6eacc lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155291 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826018.821393:0:155291:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6eacc lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155291 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826018.821406:0:155291:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c35b0a80 x1874663042895232/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787826034 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787826018.821420:0:155291:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:1.0:1787826018.821506:0:155291:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:1.0:1787826018.821565:0:155291:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826018.821577:0:155291:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826018.821585:0:155291:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826018.821597:0:155291:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826018.821603:0:155291:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826018.821608:0:155291:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826018.821611:0:155291:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787826018.821616:0:155291:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501ef3be8 00010000:00010000:1.0:1787826018.821619:0:155291:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826018.821627:0:155291:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826018.821638:0:155291:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826018.821644:0:155291:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=16408 00000080:00200000:1.0:1787826018.821650:0:155291:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826018.821658:0:155291:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=16392 00000080:00200000:1.0:1787826018.821662:0:155291:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:1.0:1787826018.821671:0:155291:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787826018.821702:0:155291:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6ead3 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155291 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826018.821710:0:155291:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6ead3 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155291 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826018.821719:0:155291:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6ead3 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155291 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826018.821736:0:155291:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155291:1874663042895360:192.168.203.147@tcp:101:lfs.0 00000100:00100000:1.0:1787826018.821818:0:155291:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8180 going to sleep for 11 seconds 00000100:00100000:1.0:1787826018.851338:0:155291:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f1180 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155291:1874663042895488:192.168.203.147@tcp:101:lfs.0 00010000:00010000:1.0:1787826018.851365:0:155291:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7800/0xbef42c0221d6eada lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceaf40 expref: -99 pid: 155291 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826018.851383:0:155291:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7800/0xbef42c0221d6eada lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceaf40 expref: -99 pid: 155291 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826018.851392:0:155291:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7800/0xbef42c0221d6eada lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceaf40 expref: -99 pid: 155291 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826018.851403:0:155291:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7800/0xbef42c0221d6eada lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceaf40 expref: -99 pid: 155291 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826018.851414:0:155291:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7800/0xbef42c0221d6eada lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceaf40 expref: -99 pid: 155291 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826018.851427:0:155291:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7800/0xbef42c0221d6eada lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceaf40 expref: -99 pid: 155291 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826018.851435:0:155291:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7800/0xbef42c0221d6eada lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceaf40 expref: -99 pid: 155291 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826018.851446:0:155291:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7800/0xbef42c0221d6eada lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceaf40 expref: -99 pid: 155291 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826018.851458:0:155291:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c71f1180 x1874663042895488/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826034 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826018.851471:0:155291:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7800/0xbef42c0221d6eada lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceaf40 expref: -99 pid: 155291 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826018.851481:0:155291:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/6112 (0 0) 00000080:00200000:1.0:1787826018.851488:0:155291:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787826018.851514:0:155291:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x17e0:0x0], updating i_size 4096 00000080:00010000:1.0:1787826018.851520:0:155291:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) for lock 0xbef42c0221d6eada 00000080:00010000:1.0:1787826018.851526:0:155291:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00000080:00010000:1.0:1787826018.851529:0:155291:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6eada from it ffffb97501ef3cb8 00010000:00010000:1.0:1787826018.851532:0:155291:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7800/0xbef42c0221d6eada lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceaf40 expref: -99 pid: 155291 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826018.851539:0:155291:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7800/0xbef42c0221d6eada lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceaf40 expref: -99 pid: 155291 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:1.0:1787826018.855356:0:155291:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), request_mask 0, flags 0, foreign 0 00000080:00200000:1.0:1787826018.855367:0:155291:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826018.855388:0:155291:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826018.855405:0:155291:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826018.855424:0:155291:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826018.855431:0:155291:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826018.855439:0:155291:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826018.855442:0:155291:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787826018.855450:0:155291:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501ef3cb8 00010000:00010000:1.0:1787826018.855454:0:155291:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826018.855462:0:155291:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826018.855591:0:155291:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826018.855601:0:155291:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826018.855608:0:155291:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826018.855618:0:155291:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826018.855624:0:155291:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826018.855629:0:155291:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826018.855631:0:155291:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787826018.855636:0:155291:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501ef3bf8 00010000:00010000:1.0:1787826018.855638:0:155291:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826018.855646:0:155291:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826018.855654:0:155291:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826018.855661:0:155291:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=267 00000080:00200000:1.0:1787826018.855668:0:155291:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 44 00000080:00200008:1.0:1787826018.855673:0:155291:0:(file.c:1090:ll_file_open()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00000080:00200080:1.0:1787826018.855693:0:155291:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) cmd=c03066f0 arg=7fffb56538c0 00000080:00200000:1.0:1787826018.855706:0:155291:0:(dir.c:451:ll_dir_setdirstripe()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) name=test stripe_offset=0 stripe_count=2, hash_type=0 00010000:00010000:1.0:1787826018.855730:0:155291:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7800/0xbef42c0221d6eada lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ceaf40 expref: -99 pid: 155291 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826018.857140:0:155291:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6800/0xbef42c0221d64b3f lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9634 expref: -99 pid: 152202 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826018.857185:0:155291:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7800/0xbef42c0221d6eada lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ceaf40 expref: -99 pid: 155291 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826018.857192:0:155291:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826018.857195:0:155291:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7800/0xbef42c0221d6eada lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ceaf40 expref: -99 pid: 155291 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:1.0:1787826018.857215:0:155291:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787826018.857224:0:155291:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c35b1500 x1874663042896128/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 552/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826018.857243:0:155291:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155291:1874663042896128:192.168.203.147@tcp:36:lfs.0 00000100:00100000:1.0:1787826018.857291:0:155291:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8840 going to sleep for 11 seconds 00000100:00100000:0.0:1787826018.947500:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826018.950890:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049134464 00000100:00100000:1.0:1787826018.950908:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049134464 00000100:00100000:1.0:1787826018.950979:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9199 00000100:00100000:1.0:1787826018.950990:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:188682:x1874663049134464:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826018.951009:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6eae8 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x21:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceaf94 expref: -99 pid: 155306 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826018.951120:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826018.951127:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:188682:x1874663049134464:12345-192.168.203.147@tcp:104: Request processed in 140us (3638us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826018.951134:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9199 00010000:00010000:1.0:1787826018.951268:0:15160:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826018.951275:0:15160:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6eae8 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x21:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceaf94 expref: -99 pid: 155306 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826018.951283:0:15160:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3480e00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:1.0:1787826018.951287:0:15160:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6eae8 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x21:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00ceaf94 expref: -99 pid: 155306 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826018.951296:0:15160:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6eae8 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x21:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00ceaf94 expref: -99 pid: 155306 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826018.951303:0:15160:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6eae8 lrc: 3/0,0 mode: PR/PR res: [0x200026560:0x21:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x408400000000 nid: local remote: 0x878a5eea00ceaf94 expref: -99 pid: 155306 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826018.951367:0:15160:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6eae8 lrc: 2/0,0 mode: --/PR res: [0x200026560:0x21:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ceaf94 expref: -99 pid: 155306 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826018.951375:0:15160:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826018.951394:0:15160:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6eae8 lrc: 1/0,0 mode: --/PR res: [0x200026560:0x21:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ceaf94 expref: -99 pid: 155306 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826018.951401:0:15160:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6eae8 lrc: 0/0,0 mode: --/PR res: [0x200026560:0x21:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ceaf94 expref: -99 pid: 155306 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826018.951419:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826018.951420:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826018.951422:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826018.951435:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042896896:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1787826018.959991:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826018.960063:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042896896:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1787826018.960103:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049134720 00000100:00100000:0.0:1787826018.960110:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049134720 00000100:00100000:0.0:1787826018.960158:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9200 00000100:00100000:0.0:1787826018.960167:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a7b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663049134720:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826018.960183:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6eaef lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x21:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceafa2 expref: -99 pid: 155306 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826018.960255:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826018.960262:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a7b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663049134720:12345-192.168.203.147@tcp:104: Request processed in 96us (283us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826018.960269:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9200 00010000:00010000:0.0:1787826018.966443:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826018.966446:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826018.966449:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200000:0.0:1787826019.019849:0:155324:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826019.019878:0:155324:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826019.019898:0:155324:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826019.019915:0:155324:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826019.019923:0:155324:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826019.019933:0:155324:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826019.019937:0:155324:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826019.019944:0:155324:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501f2bb58 00010000:00010000:0.0:1787826019.019948:0:155324:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826019.019958:0:155324:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826019.019971:0:155324:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826019.019978:0:155324:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=29 00000080:00200000:0.0:1787826019.019984:0:155324:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826019.019995:0:155324:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=13 00000080:00200000:0.0:1787826019.019999:0:155324:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787826019.020013:0:155324:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787826019.020067:0:155324:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6eaf6 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155324 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.020075:0:155324:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6eaf6 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155324 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.020084:0:155324:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6eaf6 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155324 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826019.020109:0:155324:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155324:1874663042897152:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787826019.020309:0:155324:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bb180 going to sleep for 11 seconds 00000100:00100000:0.0:1787826019.028253:0:155324:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155324:1874663042897152:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787826019.028275:0:155324:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6eaf6 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155324 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.028288:0:155324:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6eaf6 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155324 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.028296:0:155324:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6eaf6 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155324 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.028303:0:155324:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6eaf6 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155324 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.028311:0:155324:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6eaf6 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155324 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.028317:0:155324:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0513400 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787826019.028321:0:155324:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6eaf6 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155324 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.028328:0:155324:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6eaf6 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155324 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.028334:0:155324:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6eaf6 lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 155324 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.028341:0:155324:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787826019.028347:0:155324:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6eaf6 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155324 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.028354:0:155324:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6eaf6 lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155324 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826019.028369:0:155324:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c55a5880 x1874663042897152/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787826035 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787826019.028383:0:155324:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787826019.028409:0:155324:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787826019.028471:0:155324:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826019.028484:0:155324:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826019.028493:0:155324:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826019.028503:0:155324:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826019.028510:0:155324:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826019.028515:0:155324:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826019.028518:0:155324:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826019.028524:0:155324:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501f2bbe8 00010000:00010000:0.0:1787826019.028527:0:155324:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826019.028535:0:155324:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826019.028545:0:155324:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826019.028552:0:155324:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=16408 00000080:00200000:0.0:1787826019.028556:0:155324:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826019.028566:0:155324:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=16392 00000080:00200000:0.0:1787826019.028570:0:155324:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787826019.028581:0:155324:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787826019.028616:0:155324:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6eafd lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155324 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.028624:0:155324:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6eafd lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155324 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.028633:0:155324:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6eafd lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155324 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826019.028652:0:155324:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155324:1874663042897280:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787826019.028726:0:155324:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bb180 going to sleep for 11 seconds 00000100:00100000:0.0:1787826019.031506:0:155324:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155324:1874663042897280:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787826019.031527:0:155324:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6eafd lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155324 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.031539:0:155324:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6eafd lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155324 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.031547:0:155324:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6eafd lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155324 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.031554:0:155324:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6eafd lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155324 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.031562:0:155324:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6eafd lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155324 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.031568:0:155324:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0513a00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787826019.031572:0:155324:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6eafd lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155324 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.031579:0:155324:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6eafd lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155324 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.031587:0:155324:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6eafd lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 155324 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.031594:0:155324:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787826019.031599:0:155324:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6eafd lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155324 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.031605:0:155324:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6eafd lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155324 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826019.031620:0:155324:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c55a5880 x1874663042897280/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787826035 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787826019.031633:0:155324:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787826019.031654:0:155324:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787826019.031844:0:155324:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826019.031857:0:155324:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826019.031865:0:155324:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826019.031876:0:155324:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826019.031882:0:155324:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826019.031887:0:155324:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826019.031890:0:155324:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826019.031895:0:155324:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501f2bb58 00010000:00010000:0.0:1787826019.031899:0:155324:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826019.031908:0:155324:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826019.031919:0:155324:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826019.031925:0:155324:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=13 00000080:00200008:0.0:1787826019.031934:0:155324:0:(file.c:6296:ll_getattr_dentry()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1787826019.031938:0:155324:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08),name=d71a.replay-single 00000002:00010000:0.0:1787826019.031948:0:155324:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x20001a9e3:0x17e0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826019.031952:0:155324:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00010000:00010000:0.0:1787826019.031982:0:155324:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6eb04 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 155324 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.031989:0:155324:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6eb04 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 155324 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.031997:0:155324:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6eb04 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 155324 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826019.032014:0:155324:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155324:1874663042897408:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787826019.032088:0:155324:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bb180 going to sleep for 11 seconds 00000100:00100000:0.0:1787826019.036246:0:155324:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155324:1874663042897408:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787826019.036275:0:155324:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6eb04 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb004 expref: -99 pid: 155324 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.036291:0:155324:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6eb04 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb004 expref: -99 pid: 155324 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.036305:0:155324:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6eb04 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb004 expref: -99 pid: 155324 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.036318:0:155324:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6eb04 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb004 expref: -99 pid: 155324 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.036328:0:155324:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6eb04 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb004 expref: -99 pid: 155324 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.036339:0:155324:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6eb04 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb004 expref: -99 pid: 155324 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.036346:0:155324:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6eb04 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb004 expref: -99 pid: 155324 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.036352:0:155324:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6eb04 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb004 expref: -99 pid: 155324 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826019.036361:0:155324:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c55a5880 x1874663042897408/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826035 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787826019.036374:0:155324:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6eb04 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb004 expref: -99 pid: 155324 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.036385:0:155324:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787826019.036392:0:155324:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1787826019.036427:0:155324:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x17e0:0x0], updating i_size 4096 00000080:00010000:0.0:1787826019.036435:0:155324:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) for lock 0xbef42c0221d6eb04 00000080:00010000:0.0:1787826019.036442:0:155324:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00000080:00010000:0.0:1787826019.036446:0:155324:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6eb04 from it ffffb97501f2bcb8 00010000:00010000:0.0:1787826019.036449:0:155324:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6eb04 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb004 expref: -99 pid: 155324 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.036458:0:155324:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6eb04 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb004 expref: -99 pid: 155324 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:0.0:1787826019.037058:0:155324:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1787826019.037065:0:155324:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826019.037082:0:155324:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826019.037228:0:155324:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826019.037244:0:155324:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826019.037252:0:155324:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826019.037257:0:155324:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826019.037260:0:155324:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826019.037265:0:155324:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501f2bcb8 00010000:00010000:0.0:1787826019.037269:0:155324:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826019.037277:0:155324:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826019.037416:0:155324:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826019.037429:0:155324:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826019.037436:0:155324:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826019.037445:0:155324:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826019.037450:0:155324:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826019.037454:0:155324:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826019.037457:0:155324:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826019.037460:0:155324:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501f2bbf8 00010000:00010000:0.0:1787826019.037463:0:155324:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826019.037469:0:155324:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826019.037478:0:155324:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826019.037483:0:155324:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=267 00000080:00200000:0.0:1787826019.037488:0:155324:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 44 00000080:00200008:0.0:1787826019.037494:0:155324:0:(file.c:1090:ll_file_open()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00000080:00200080:0.0:1787826019.037516:0:155324:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) cmd=c03066f0 arg=7ffd6d2c5ca0 00000080:00200000:0.0:1787826019.037530:0:155324:0:(dir.c:451:ll_dir_setdirstripe()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) name=test stripe_offset=0 stripe_count=2, hash_type=0 00010000:00010000:0.0:1787826019.037555:0:155324:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6eb04 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ceb004 expref: -99 pid: 155324 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826019.037576:0:155324:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6800/0xbef42c0221d64b3f lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9634 expref: -99 pid: 152202 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826019.037615:0:155324:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6eb04 lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ceb004 expref: -99 pid: 155324 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826019.037623:0:155324:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826019.037626:0:155324:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6eb04 lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ceb004 expref: -99 pid: 155324 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:0.0:1787826019.037644:0:155324:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787826019.037654:0:155324:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c55a5880 x1874663042897664/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 552/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826019.037676:0:155324:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155324:1874663042897664:192.168.203.147@tcp:36:lfs.0 00000100:00100000:0.0:1787826019.038522:0:155324:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bb180 going to sleep for 11 seconds 00000100:00100000:0.0:1787826019.059251:0:155324:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155324:1874663042897664:192.168.203.147@tcp:36:lfs.0 00000080:00200000:0.0:1787826019.059305:0:155324:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x20001a9e3:0x1bae:0x0](ffff9181c83a4b08) 00000080:00200000:0.0:1787826019.059311:0:155324:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1bae:0x0], updating i_size 4096 00000080:00200008:0.0:1787826019.059339:0:155324:0:(file.c:479:ll_file_release()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00010000:00010000:0.0:1787826019.059367:0:155324:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513000/0xbef42c0221d64bee lrc: 2/0,0 mode: CR/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce962d expref: -99 pid: 152210 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826019.064671:0:155333:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826019.064704:0:155333:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826019.064722:0:155333:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826019.064739:0:155333:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826019.064773:0:155333:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826019.064782:0:155333:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826019.064785:0:155333:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826019.064793:0:155333:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501f73c20 00010000:00010000:0.0:1787826019.064797:0:155333:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826019.064806:0:155333:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826019.064819:0:155333:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826019.064826:0:155333:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=24 00000080:00200000:0.0:1787826019.064832:0:155333:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826019.064842:0:155333:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=0 00000080:00200000:0.0:1787826019.064846:0:155333:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787826019.064859:0:155333:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787826019.064900:0:155333:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6eb0b lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155333 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.064908:0:155333:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6eb0b lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155333 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.064918:0:155333:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6eb0b lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155333 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826019.064947:0:155333:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155333:1874663042897792:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787826019.065037:0:155333:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bb180 going to sleep for 11 seconds 00000100:00100000:1.0:1787826019.087725:0:155333:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f0380 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155333:1874663042897920:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:1.0:1787826019.088173:0:155333:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7c00/0xbef42c0221d6eb12 lrc: 4/1,0 mode: --/CR res: [0x200026560:0x22:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb058 expref: -99 pid: 155333 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826019.088190:0:155333:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7c00/0xbef42c0221d6eb12 lrc: 4/1,0 mode: --/PR res: [0x200026560:0x22:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb058 expref: -99 pid: 155333 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826019.088201:0:155333:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7c00/0xbef42c0221d6eb12 lrc: 4/1,0 mode: PR/PR res: [0x200026560:0x22:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb058 expref: -99 pid: 155333 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826019.088213:0:155333:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7c00/0xbef42c0221d6eb12 lrc: 4/1,0 mode: PR/PR res: [0x200026560:0x22:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb058 expref: -99 pid: 155333 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826019.088227:0:155333:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7c00/0xbef42c0221d6eb12 lrc: 5/2,0 mode: PR/PR res: [0x200026560:0x22:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb058 expref: -99 pid: 155333 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826019.088241:0:155333:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7c00/0xbef42c0221d6eb12 lrc: 5/2,0 mode: PR/PR res: [0x200026560:0x22:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb058 expref: -99 pid: 155333 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826019.088249:0:155333:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7c00/0xbef42c0221d6eb12 lrc: 4/1,0 mode: PR/PR res: [0x200026560:0x22:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb058 expref: -99 pid: 155333 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826019.088263:0:155333:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c71f0380 x1874663042897920/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787826035 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826019.088279:0:155333:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7c00/0xbef42c0221d6eb12 lrc: 3/1,0 mode: PR/PR res: [0x200026560:0x22:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb058 expref: -99 pid: 155333 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826019.088292:0:155333:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590091616/34 (0 0) 00010000:00010000:1.0:1787826019.088300:0:155333:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7c00/0xbef42c0221d6eb12 lrc: 3/1,0 mode: PR/PR res: [0x200026560:0x22:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb058 expref: -99 pid: 155333 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826019.088309:0:155333:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7c00/0xbef42c0221d6eb12 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x22:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb058 expref: -99 pid: 155333 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000002:00010000:1.0:1787826019.088338:0:155333:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001f58:0x22:0x0]) in obj [0x240001f58:0x22:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826019.088386:0:155333:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6eb19 lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x22:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 155333 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826019.088393:0:155333:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6eb19 lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x22:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155333 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826019.088401:0:155333:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6eb19 lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x22:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155333 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826019.088425:0:155333:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155333:1874663042898048:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:1.0:1787826019.090190:0:155333:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8a80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826019.116607:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826019.116646:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049136128 00000100:00100000:1.0:1787826019.116655:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049136128 00000100:00100000:1.0:1787826019.116985:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9201 00000100:00100000:1.0:1787826019.116996:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b0380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:188682:x1874663049136128:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826019.117013:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7c00/0xbef42c0221d6eb12 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x22:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceb058 expref: -99 pid: 155333 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826019.117133:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826019.117142:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b0380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:188682:x1874663049136128:12345-192.168.203.147@tcp:104: Request processed in 148us (543us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826019.117149:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9201 00000100:00100000:1.0:1787826019.122671:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826019.122725:0:33333:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:1.0:1787826019.122739:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049136384 00000100:00100000:1.0:1787826019.122764:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049136384 00000100:00100000:1.0:1787826019.122807:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9202 00000100:00100000:1.0:1787826019.122817:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181e053ec50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663049136384:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826019.122832:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6eb19 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x22:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceb066 expref: -99 pid: 155333 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826019.122892:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826019.122898:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181e053ec50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663049136384:12345-192.168.203.147@tcp:104: Request processed in 83us (235us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826019.122906:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9202 00010000:00010000:1.0:1787826019.122951:0:33329:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826019.122957:0:33329:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6eb19 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x22:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceb066 expref: -99 pid: 155333 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826019.122969:0:33329:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4600 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:1.0:1787826019.122974:0:33329:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6eb19 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x22:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00ceb066 expref: -99 pid: 155333 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826019.122982:0:33329:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6eb19 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x22:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00ceb066 expref: -99 pid: 155333 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826019.122990:0:33329:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6eb19 lrc: 3/0,0 mode: PR/PR res: [0x240001f58:0x22:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x408400000000 nid: local remote: 0x878a5eea00ceb066 expref: -99 pid: 155333 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826019.123055:0:33329:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6eb19 lrc: 2/0,0 mode: --/PR res: [0x240001f58:0x22:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ceb066 expref: -99 pid: 155333 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826019.123062:0:33329:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826019.123083:0:33329:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6eb19 lrc: 1/0,0 mode: --/PR res: [0x240001f58:0x22:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ceb066 expref: -99 pid: 155333 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826019.123090:0:33329:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6eb19 lrc: 0/0,0 mode: --/PR res: [0x240001f58:0x22:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ceb066 expref: -99 pid: 155333 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826019.123105:0:33329:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826019.123107:0:33329:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826019.123108:0:33329:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826019.123122:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042898560:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1787826019.135830:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042898560:192.168.203.147@tcp:103:ldlm_bl.0 00000080:00200000:0.0:1787826019.203510:0:155359:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826019.203540:0:155359:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826019.203558:0:155359:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826019.203575:0:155359:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826019.203583:0:155359:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826019.203591:0:155359:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826019.203595:0:155359:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826019.203602:0:155359:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501edbc20 00010000:00010000:0.0:1787826019.203606:0:155359:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826019.203616:0:155359:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826019.203629:0:155359:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826019.203637:0:155359:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=24 00000080:00200000:0.0:1787826019.203643:0:155359:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826019.203654:0:155359:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=0 00000080:00200000:0.0:1787826019.203658:0:155359:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787826019.203739:0:155359:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787826019.203850:0:155359:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6eb35 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155359 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.203859:0:155359:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6eb35 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155359 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.203868:0:155359:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6eb35 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155359 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826019.203890:0:155359:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155359:1874663042899328:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787826019.203978:0:155359:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bbe40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826019.212135:0:155359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155359:1874663042899328:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:0.0:1787826019.212163:0:155359:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6eb35 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb10e expref: -99 pid: 155359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826019.247772:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826019.247878:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049137792 00000100:00100000:0.0:1787826019.247890:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049137792 00000100:00100000:0.0:1787826019.247954:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9203 00000100:00100000:0.0:1787826019.247963:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b1880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:188682:x1874663049137792:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826019.247980:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7e00/0xbef42c0221d6eb3c lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x23:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceb11c expref: -99 pid: 155359 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826019.248059:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826019.248066:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b1880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:188682:x1874663049137792:12345-192.168.203.147@tcp:104: Request processed in 105us (334us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826019.248074:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9203 00010000:00010000:0.0:1787826019.248095:0:33350:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826019.248102:0:33350:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7e00/0xbef42c0221d6eb3c lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x23:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceb11c expref: -99 pid: 155359 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826019.248110:0:33350:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e06f7e00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787826019.248116:0:33350:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7e00/0xbef42c0221d6eb3c lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x23:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00ceb11c expref: -99 pid: 155359 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826019.248124:0:33350:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7e00/0xbef42c0221d6eb3c lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x23:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00ceb11c expref: -99 pid: 155359 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826019.248132:0:33350:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7e00/0xbef42c0221d6eb3c lrc: 3/0,0 mode: PR/PR res: [0x200026560:0x23:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x408400000000 nid: local remote: 0x878a5eea00ceb11c expref: -99 pid: 155359 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826019.248185:0:33350:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7e00/0xbef42c0221d6eb3c lrc: 2/0,0 mode: --/PR res: [0x200026560:0x23:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ceb11c expref: -99 pid: 155359 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826019.248192:0:33350:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826019.248212:0:33350:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7e00/0xbef42c0221d6eb3c lrc: 1/0,0 mode: --/PR res: [0x200026560:0x23:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ceb11c expref: -99 pid: 155359 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826019.248219:0:33350:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7e00/0xbef42c0221d6eb3c lrc: 0/0,0 mode: --/PR res: [0x200026560:0x23:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ceb11c expref: -99 pid: 155359 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826019.248237:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826019.248239:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826019.248241:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826019.248248:0:2390:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1787826019.248260:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042899968:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1787826019.255113:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042899968:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787826019.259513:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826019.259557:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049138048 00000100:00100000:1.0:1787826019.259566:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049138048 00000100:00100000:1.0:1787826019.259610:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9204 00000100:00100000:1.0:1787826019.259619:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b0380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663049138048:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826019.259636:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481000/0xbef42c0221d6eb43 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x23:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceb12a expref: -99 pid: 155359 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826019.259720:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826019.259727:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b0380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663049138048:12345-192.168.203.147@tcp:104: Request processed in 109us (222us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826019.259735:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9204 00000100:00100000:1.0:1787826019.290655:0:155359:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c494e680 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155359:1874663042899840:192.168.203.147@tcp:36:rmdir.0 00000080:00200000:1.0:1787826019.290716:0:155359:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1baf:0x0](ffff9181c83d42c8) 00000080:00200000:1.0:1787826019.314349:0:155365:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826019.314378:0:155365:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826019.314396:0:155365:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826019.314411:0:155365:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826019.314419:0:155365:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826019.314427:0:155365:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826019.314430:0:155365:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787826019.314438:0:155365:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501ff3b58 00010000:00010000:1.0:1787826019.314443:0:155365:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826019.314451:0:155365:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826019.315066:0:155365:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826019.315075:0:155365:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=29 00000080:00200000:1.0:1787826019.315081:0:155365:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826019.315092:0:155365:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=13 00000080:00200000:1.0:1787826019.315097:0:155365:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:1.0:1787826019.315112:0:155365:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787826019.315158:0:155365:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5000/0xbef42c0221d6eb4a lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155365 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826019.315167:0:155365:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5000/0xbef42c0221d6eb4a lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155365 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826019.315176:0:155365:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5000/0xbef42c0221d6eb4a lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155365 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826019.315199:0:155365:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155365:1874663042900224:192.168.203.147@tcp:101:lfs.0 00000100:00100000:1.0:1787826019.315416:0:155365:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8e40 going to sleep for 11 seconds 00010000:00010000:1.0:1787826019.325807:0:155365:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:1.0:1787826019.325873:0:155365:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826019.325889:0:155365:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826019.325899:0:155365:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826019.325915:0:155365:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826019.325923:0:155365:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826019.325929:0:155365:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826019.325933:0:155365:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787826019.325950:0:155365:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501ff3b58 00010000:00010000:1.0:1787826019.325954:0:155365:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826019.325963:0:155365:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826019.325975:0:155365:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826019.325982:0:155365:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=13 00000080:00200008:1.0:1787826019.325991:0:155365:0:(file.c:6296:ll_getattr_dentry()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), request_mask 0, flags 0, foreign 0 00000080:00200000:1.0:1787826019.325996:0:155365:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08),name=d71a.replay-single 00000002:00010000:1.0:1787826019.326008:0:155365:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x20001a9e3:0x17e0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826019.326014:0:155365:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00010000:00010000:1.0:1787826019.326059:0:155365:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5800/0xbef42c0221d6eb58 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 155365 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826019.326067:0:155365:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5800/0xbef42c0221d6eb58 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 155365 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826019.326088:0:155365:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5800/0xbef42c0221d6eb58 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 155365 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826019.326113:0:155365:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155365:1874663042900480:192.168.203.147@tcp:101:lfs.0 00000100:00100000:1.0:1787826019.326298:0:155365:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c86c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826019.411616:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826019.411668:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049139456 00000100:00100000:1.0:1787826019.411676:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049139456 00000100:00100000:1.0:1787826019.411725:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9205 00000100:00100000:1.0:1787826019.411736:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:188682:x1874663049139456:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826019.411772:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480200/0xbef42c0221d6eb66 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x24:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceb1e0 expref: -99 pid: 155368 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826019.411836:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826019.411842:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:188682:x1874663049139456:12345-192.168.203.147@tcp:104: Request processed in 109us (235us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826019.411849:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9205 00010000:00010000:1.0:1787826019.411884:0:33329:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826019.411890:0:33329:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480200/0xbef42c0221d6eb66 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x24:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceb1e0 expref: -99 pid: 155368 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826019.411900:0:33329:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3480200 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:1.0:1787826019.411905:0:33329:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480200/0xbef42c0221d6eb66 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x24:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00ceb1e0 expref: -99 pid: 155368 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826019.411913:0:33329:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480200/0xbef42c0221d6eb66 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x24:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00ceb1e0 expref: -99 pid: 155368 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826019.411921:0:33329:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480200/0xbef42c0221d6eb66 lrc: 3/0,0 mode: PR/PR res: [0x200026560:0x24:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x408400000000 nid: local remote: 0x878a5eea00ceb1e0 expref: -99 pid: 155368 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826019.411974:0:33329:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480200/0xbef42c0221d6eb66 lrc: 2/0,0 mode: --/PR res: [0x200026560:0x24:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ceb1e0 expref: -99 pid: 155368 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826019.411982:0:33329:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826019.412003:0:33329:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480200/0xbef42c0221d6eb66 lrc: 1/0,0 mode: --/PR res: [0x200026560:0x24:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ceb1e0 expref: -99 pid: 155368 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826019.412011:0:33329:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480200/0xbef42c0221d6eb66 lrc: 0/0,0 mode: --/PR res: [0x200026560:0x24:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ceb1e0 expref: -99 pid: 155368 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826019.412030:0:33329:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826019.412032:0:33329:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826019.412034:0:33329:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826019.412059:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042901504:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1787826019.421088:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826019.421251:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042901504:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787826019.421526:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049139712 00000100:00100000:1.0:1787826019.421533:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049139712 00000100:00100000:1.0:1787826019.421585:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9206 00000100:00100000:1.0:1787826019.421595:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663049139712:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826019.421611:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480400/0xbef42c0221d6eb6d lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x24:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceb1ee expref: -99 pid: 155368 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826019.421679:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826019.421685:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663049139712:12345-192.168.203.147@tcp:104: Request processed in 92us (608us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826019.421696:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9206 00010000:00010000:1.0:1787826019.421715:0:15160:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826019.421721:0:15160:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480400/0xbef42c0221d6eb6d lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x24:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceb1ee expref: -99 pid: 155368 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826019.421729:0:15160:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3480400 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:1.0:1787826019.421734:0:15160:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480400/0xbef42c0221d6eb6d lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x24:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00ceb1ee expref: -99 pid: 155368 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826019.421761:0:15160:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480400/0xbef42c0221d6eb6d lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x24:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00ceb1ee expref: -99 pid: 155368 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826019.421771:0:15160:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480400/0xbef42c0221d6eb6d lrc: 3/0,0 mode: PR/PR res: [0x240001f58:0x24:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x408400000000 nid: local remote: 0x878a5eea00ceb1ee expref: -99 pid: 155368 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826019.421821:0:15160:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480400/0xbef42c0221d6eb6d lrc: 2/0,0 mode: --/PR res: [0x240001f58:0x24:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ceb1ee expref: -99 pid: 155368 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826019.421829:0:15160:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826019.421845:0:15160:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480400/0xbef42c0221d6eb6d lrc: 1/0,0 mode: --/PR res: [0x240001f58:0x24:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ceb1ee expref: -99 pid: 155368 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826019.421852:0:15160:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480400/0xbef42c0221d6eb6d lrc: 0/0,0 mode: --/PR res: [0x240001f58:0x24:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ceb1ee expref: -99 pid: 155368 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826019.425693:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826019.425699:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826019.425701:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826019.425724:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042901632:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787826019.444121:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042901632:192.168.203.147@tcp:103:ldlm_bl.0 00000080:00200000:1.0:1787826019.510719:0:155371:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826019.510762:0:155371:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826019.510781:0:155371:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826019.510798:0:155371:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826019.510806:0:155371:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826019.510814:0:155371:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826019.510817:0:155371:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787826019.510825:0:155371:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97502003b58 00010000:00010000:1.0:1787826019.510829:0:155371:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826019.510837:0:155371:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826019.510851:0:155371:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826019.510859:0:155371:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=29 00000080:00200000:1.0:1787826019.510865:0:155371:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826019.510876:0:155371:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=13 00000080:00200000:1.0:1787826019.510879:0:155371:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:1.0:1787826019.510889:0:155371:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787826019.510926:0:155371:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4200/0xbef42c0221d6eb74 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155371 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826019.510932:0:155371:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4200/0xbef42c0221d6eb74 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155371 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826019.510939:0:155371:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4200/0xbef42c0221d6eb74 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155371 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826019.510957:0:155371:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155371:1874663042901760:192.168.203.147@tcp:101:lfs.0 00000100:00100000:1.0:1787826019.511018:0:155371:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8cc0 going to sleep for 11 seconds 00010000:00010000:0.0:1787826019.536002:0:155371:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787826019.536098:0:155371:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826019.536113:0:155371:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826019.536125:0:155371:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826019.536139:0:155371:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826019.536148:0:155371:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826019.536154:0:155371:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826019.536158:0:155371:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826019.536165:0:155371:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97502003be8 00010000:00010000:0.0:1787826019.536170:0:155371:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826019.536191:0:155371:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826019.536205:0:155371:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826019.536212:0:155371:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=16408 00000080:00200000:0.0:1787826019.536217:0:155371:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826019.536230:0:155371:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=16392 00000080:00200000:0.0:1787826019.536235:0:155371:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787826019.536246:0:155371:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787826019.536288:0:155371:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6eb7b lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155371 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.536298:0:155371:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6eb7b lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155371 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.536477:0:155371:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6eb7b lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155371 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826019.536504:0:155371:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155371:1874663042901888:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787826019.536600:0:155371:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bbe40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826019.684472:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826019.684539:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049141120 00000100:00100000:0.0:1787826019.684549:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049141120 00000100:00100000:0.0:1787826019.684602:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9207 00000100:00100000:0.0:1787826019.684613:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a5180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187904:x1874663049141120:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826019.684631:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481400/0xbef42c0221d6eb90 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x25:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceb2a4 expref: -99 pid: 155375 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826019.684713:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826019.684721:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a5180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187904:x1874663049141120:12345-192.168.203.147@tcp:104: Request processed in 110us (258us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826019.684729:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9207 00000100:00100000:0.0:1787826019.699856:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826019.699907:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049141376 00000100:00100000:1.0:1787826019.699917:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049141376 00000100:00100000:1.0:1787826019.699980:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9208 00000100:00100000:1.0:1787826019.699992:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663049141376:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826019.700014:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6eb97 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x25:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceb2b2 expref: -99 pid: 155375 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826019.700122:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826019.700129:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663049141376:12345-192.168.203.147@tcp:104: Request processed in 140us (275us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826019.700142:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9208 00000080:00200000:0.0:1787826019.760309:0:155380:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826019.760354:0:155380:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826019.760378:0:155380:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826019.760400:0:155380:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826019.760410:0:155380:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826019.760421:0:155380:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826019.760425:0:155380:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826019.760435:0:155380:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97500d7fb58 00010000:00010000:0.0:1787826019.760440:0:155380:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826019.760449:0:155380:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826019.760466:0:155380:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826019.760475:0:155380:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=29 00000080:00200000:0.0:1787826019.760483:0:155380:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826019.760496:0:155380:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=13 00000080:00200000:0.0:1787826019.760500:0:155380:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787826019.760515:0:155380:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787826019.760574:0:155380:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6eb9e lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.760582:0:155380:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6eb9e lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.760591:0:155380:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6eb9e lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155380 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826019.760623:0:155380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155380:1874663042903296:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787826019.761020:0:155380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bb240 going to sleep for 11 seconds 00010000:00010000:0.0:1787826019.767240:0:155380:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787826019.767320:0:155380:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826019.767337:0:155380:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826019.767347:0:155380:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826019.767360:0:155380:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826019.767366:0:155380:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826019.767371:0:155380:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826019.767375:0:155380:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826019.767381:0:155380:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97500d7fbe8 00010000:00010000:0.0:1787826019.767384:0:155380:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826019.767393:0:155380:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826019.767408:0:155380:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826019.767416:0:155380:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=16408 00000080:00200000:0.0:1787826019.767421:0:155380:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826019.767441:0:155380:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=16392 00000080:00200000:0.0:1787826019.767446:0:155380:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787826019.767462:0:155380:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787826019.767511:0:155380:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6eba5 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.767518:0:155380:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6eba5 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155380 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.767528:0:155380:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6eba5 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155380 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826019.767553:0:155380:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155380:1874663042903424:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787826019.767837:0:155380:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bb240 going to sleep for 11 seconds 00000080:00200000:0.0:1787826019.833270:0:155383:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826019.833304:0:155383:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826019.833322:0:155383:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826019.833337:0:155383:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826019.833346:0:155383:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826019.833354:0:155383:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826019.833358:0:155383:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826019.833365:0:155383:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97500da3c20 00010000:00010000:0.0:1787826019.833370:0:155383:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826019.833378:0:155383:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826019.833391:0:155383:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826019.833398:0:155383:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=24 00000080:00200000:0.0:1787826019.833404:0:155383:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826019.833417:0:155383:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=0 00000080:00200000:0.0:1787826019.833421:0:155383:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787826019.833432:0:155383:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787826019.833473:0:155383:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6ebb3 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155383 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.833480:0:155383:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6ebb3 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155383 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.833489:0:155383:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6ebb3 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155383 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826019.833513:0:155383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6a00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155383:1874663042903936:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787826019.833595:0:155383:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bb540 going to sleep for 11 seconds 00000100:00100000:0.0:1787826019.846877:0:155383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6a00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155383:1874663042903936:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:0.0:1787826019.846904:0:155383:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6ebb3 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb35a expref: -99 pid: 155383 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.846926:0:155383:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6ebb3 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1bb2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb35a expref: -99 pid: 155383 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.846935:0:155383:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6ebb3 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1bb2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb35a expref: -99 pid: 155383 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.846946:0:155383:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6ebb3 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bb2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb35a expref: -99 pid: 155383 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.846957:0:155383:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6ebb3 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bb2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb35a expref: -99 pid: 155383 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.846974:0:155383:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6ebb3 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1bb2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb35a expref: -99 pid: 155383 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.846985:0:155383:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6ebb3 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1bb2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb35a expref: -99 pid: 155383 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.846992:0:155383:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6ebb3 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bb2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb35a expref: -99 pid: 155383 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826019.847002:0:155383:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c55a6a00 x1874663042903936/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/672 e 0 to 0 dl 1787826035 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787826019.847016:0:155383:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6ebb3 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1bb2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb35a expref: -99 pid: 155383 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.847025:0:155383:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/7090 (0 0) 00000080:00200000:0.0:1787826019.847032:0:155383:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00000002:00010000:0.0:1787826019.848220:0:155383:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200026560:0x26:0x0]) in obj [0x200026560:0x26:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826019.848279:0:155383:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6ebba lrc: 3/1,0 mode: --/CR res: [0x200026560:0x26:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 155383 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.848290:0:155383:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6ebba lrc: 3/1,0 mode: --/CR res: [0x200026560:0x26:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155383 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.848299:0:155383:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6ebba lrc: 3/1,0 mode: --/CR res: [0x200026560:0x26:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155383 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826019.848322:0:155383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6300 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155383:1874663042904064:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787826019.848398:0:155383:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bb540 going to sleep for 11 seconds 00000100:00100000:0.0:1787826019.870231:0:155383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f3b80 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155383:1874663042904192:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:0.0:1787826019.870256:0:155383:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f7000/0xbef42c0221d6ebc1 lrc: 4/1,0 mode: --/CR res: [0x240001f58:0x26:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb376 expref: -99 pid: 155383 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.870268:0:155383:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f7000/0xbef42c0221d6ebc1 lrc: 4/1,0 mode: --/PR res: [0x240001f58:0x26:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb376 expref: -99 pid: 155383 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.870279:0:155383:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f7000/0xbef42c0221d6ebc1 lrc: 4/1,0 mode: PR/PR res: [0x240001f58:0x26:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb376 expref: -99 pid: 155383 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.870288:0:155383:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f7000/0xbef42c0221d6ebc1 lrc: 4/1,0 mode: PR/PR res: [0x240001f58:0x26:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb376 expref: -99 pid: 155383 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.870300:0:155383:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f7000/0xbef42c0221d6ebc1 lrc: 5/2,0 mode: PR/PR res: [0x240001f58:0x26:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb376 expref: -99 pid: 155383 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.870307:0:155383:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f7000/0xbef42c0221d6ebc1 lrc: 5/2,0 mode: PR/PR res: [0x240001f58:0x26:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb376 expref: -99 pid: 155383 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.870313:0:155383:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f7000/0xbef42c0221d6ebc1 lrc: 4/1,0 mode: PR/PR res: [0x240001f58:0x26:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb376 expref: -99 pid: 155383 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826019.870323:0:155383:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c71f3b80 x1874663042904192/t0(0) o101->lustre-MDT0001-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787826035 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787826019.870336:0:155383:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f7000/0xbef42c0221d6ebc1 lrc: 3/1,0 mode: PR/PR res: [0x240001f58:0x26:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb376 expref: -99 pid: 155383 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.870353:0:155383:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d0f54800 type 13 mode 4 res 9663684440/38 (0 0) 00010000:00010000:0.0:1787826019.870361:0:155383:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f7000/0xbef42c0221d6ebc1 lrc: 3/1,0 mode: PR/PR res: [0x240001f58:0x26:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb376 expref: -99 pid: 155383 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.870369:0:155383:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f7000/0xbef42c0221d6ebc1 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x26:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb376 expref: -99 pid: 155383 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826019.870400:0:155383:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1bb2:0x0], updating i_size 8192 00000080:00200000:0.0:1787826019.870404:0:155383:0:(namei.c:147:ll_iget()) got inode: [0x20001a9e3:0x1bb2:0x0](ffff9181c83e9148): rc = 0 00000080:00010000:0.0:1787826019.870413:0:155383:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x1bb2:0x0](ffff9181c83e9148) for lock 0xbef42c0221d6ebb3 00000080:00010000:0.0:1787826019.870430:0:155383:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x1bb2:0x0](ffff9181c83e9148) 00000080:00010000:0.0:1787826019.870433:0:155383:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6ebb3 from it ffffb97500da3e08 00010000:00010000:0.0:1787826019.870437:0:155383:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6ebb3 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1bb2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb35a expref: -99 pid: 155383 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.870446:0:155383:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6ebb3 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1bb2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb35a expref: -99 pid: 155383 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826019.870470:0:155383:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 3 00000080:00200000:0.0:1787826019.870480:0:155383:0:(namei.c:2328:ll_rmdir()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00010000:00010000:0.0:1787826019.870513:0:155383:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6ebb3 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1bb2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ceb35a expref: -99 pid: 155383 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826019.870565:0:155383:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6ebb3 lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x1bb2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ceb35a expref: -99 pid: 155383 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826019.870574:0:155383:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826019.870578:0:155383:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6ebb3 lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x1bb2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ceb35a expref: -99 pid: 155383 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:0.0:1787826019.870604:0:155383:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787826019.870613:0:155383:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c55a6a00 x1874663042904448/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826019.870637:0:155383:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6a00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155383:1874663042904448:192.168.203.147@tcp:36:rmdir.0 00000100:00100000:0.0:1787826019.870722:0:155383:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bbe40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826019.888029:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826019.888104:0:15151:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:1.0:1787826019.888120:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049142784 00000100:00100000:1.0:1787826019.888129:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049142784 00000100:00100000:1.0:1787826019.888177:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9209 00000100:00100000:1.0:1787826019.888186:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181e053d850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:188682:x1874663049142784:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826019.888203:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6ebba lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x26:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceb368 expref: -99 pid: 155383 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826019.888284:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826019.888291:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181e053d850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:188682:x1874663049142784:12345-192.168.203.147@tcp:104: Request processed in 106us (271us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826019.888298:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9209 00010000:00010000:1.0:1787826019.888437:0:33329:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826019.888444:0:33329:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6ebba lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x26:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceb368 expref: -99 pid: 155383 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826019.888453:0:33329:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0513e00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:1.0:1787826019.888458:0:33329:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6ebba lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x26:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00ceb368 expref: -99 pid: 155383 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826019.888466:0:33329:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6ebba lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x26:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00ceb368 expref: -99 pid: 155383 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826019.888473:0:33329:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6ebba lrc: 3/0,0 mode: PR/PR res: [0x200026560:0x26:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x408400000000 nid: local remote: 0x878a5eea00ceb368 expref: -99 pid: 155383 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826019.888527:0:33329:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6ebba lrc: 2/0,0 mode: --/PR res: [0x200026560:0x26:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ceb368 expref: -99 pid: 155383 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826019.888533:0:33329:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826019.888552:0:33329:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6ebba lrc: 1/0,0 mode: --/PR res: [0x200026560:0x26:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ceb368 expref: -99 pid: 155383 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826019.888558:0:33329:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6ebba lrc: 0/0,0 mode: --/PR res: [0x200026560:0x26:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ceb368 expref: -99 pid: 155383 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826019.888577:0:33329:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826019.888579:0:33329:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826019.888580:0:33329:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826019.888593:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042904576:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787826019.903066:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042904576:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787826019.911740:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826019.911797:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049143040 00000100:00100000:1.0:1787826019.911816:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049143040 00000100:00100000:1.0:1787826019.911862:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9210 00000100:00100000:1.0:1787826019.911871:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b1880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663049143040:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826019.911887:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f7000/0xbef42c0221d6ebc1 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x26:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceb376 expref: -99 pid: 155383 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826019.911950:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826019.911956:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b1880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663049143040:12345-192.168.203.147@tcp:104: Request processed in 87us (226us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826019.911962:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9210 00010000:00010000:1.0:1787826019.911983:0:15160:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826019.911988:0:15160:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f7000/0xbef42c0221d6ebc1 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x26:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceb376 expref: -99 pid: 155383 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826019.911996:0:15160:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e06f7000 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:1.0:1787826019.912001:0:15160:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f7000/0xbef42c0221d6ebc1 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x26:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00ceb376 expref: -99 pid: 155383 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826019.912014:0:15160:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f7000/0xbef42c0221d6ebc1 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x26:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00ceb376 expref: -99 pid: 155383 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826019.912022:0:15160:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f7000/0xbef42c0221d6ebc1 lrc: 3/0,0 mode: PR/PR res: [0x240001f58:0x26:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x408400000000 nid: local remote: 0x878a5eea00ceb376 expref: -99 pid: 155383 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826019.912087:0:15160:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f7000/0xbef42c0221d6ebc1 lrc: 2/0,0 mode: --/PR res: [0x240001f58:0x26:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ceb376 expref: -99 pid: 155383 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826019.912094:0:15160:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826019.912111:0:15160:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f7000/0xbef42c0221d6ebc1 lrc: 1/0,0 mode: --/PR res: [0x240001f58:0x26:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ceb376 expref: -99 pid: 155383 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826019.912118:0:15160:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f7000/0xbef42c0221d6ebc1 lrc: 0/0,0 mode: --/PR res: [0x240001f58:0x26:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ceb376 expref: -99 pid: 155383 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826019.912138:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826019.912140:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826019.912141:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826019.912154:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042904704:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787826019.916651:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042904704:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1787826019.940398:0:155383:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6a00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155383:1874663042904448:192.168.203.147@tcp:36:rmdir.0 00000080:00200000:0.0:1787826019.940443:0:155383:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1bb2:0x0](ffff9181c83e9148) 00000100:00100000:0.0:1787826019.962320:0:155391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c494f100 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155391:1874663042904832:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787826019.962341:0:155391:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6ebc8 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155391 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.962354:0:155391:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6ebc8 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155391 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.962360:0:155391:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6ebc8 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155391 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.962367:0:155391:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6ebc8 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155391 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.962376:0:155391:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6ebc8 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155391 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.962382:0:155391:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3481e00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787826019.962386:0:155391:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6ebc8 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155391 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.962393:0:155391:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6ebc8 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155391 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.962400:0:155391:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6ebc8 lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 155391 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.962407:0:155391:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787826019.962412:0:155391:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6ebc8 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155391 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.962417:0:155391:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6ebc8 lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155391 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826019.962432:0:155391:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c494f100 x1874663042904832/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787826035 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787826019.962446:0:155391:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787826019.962473:0:155391:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787826019.962538:0:155391:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826019.962554:0:155391:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826019.962563:0:155391:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826019.962575:0:155391:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826019.962582:0:155391:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826019.962588:0:155391:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826019.962591:0:155391:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826019.962598:0:155391:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97502003be8 00010000:00010000:0.0:1787826019.962602:0:155391:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826019.962610:0:155391:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826019.962623:0:155391:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826019.962630:0:155391:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=16408 00000080:00200000:0.0:1787826019.962634:0:155391:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826019.962649:0:155391:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=16392 00000080:00200000:0.0:1787826019.962653:0:155391:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787826019.962676:0:155391:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787826019.962710:0:155391:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6ebcf lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155391 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.962718:0:155391:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6ebcf lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155391 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.962726:0:155391:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6ebcf lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155391 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826019.962786:0:155391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155391:1874663042904960:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787826019.963019:0:155391:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bbe40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826019.981079:0:155391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c494df80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155391:1874663042905088:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787826019.981117:0:155391:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6ebd6 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb3d8 expref: -99 pid: 155391 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.981140:0:155391:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6ebd6 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb3d8 expref: -99 pid: 155391 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.981148:0:155391:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6ebd6 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb3d8 expref: -99 pid: 155391 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.981160:0:155391:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6ebd6 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb3d8 expref: -99 pid: 155391 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.981171:0:155391:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6ebd6 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb3d8 expref: -99 pid: 155391 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.981183:0:155391:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6ebd6 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb3d8 expref: -99 pid: 155391 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.981190:0:155391:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6ebd6 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb3d8 expref: -99 pid: 155391 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.981196:0:155391:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6ebd6 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb3d8 expref: -99 pid: 155391 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826019.981207:0:155391:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c494df80 x1874663042905088/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826035 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787826019.981221:0:155391:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6ebd6 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb3d8 expref: -99 pid: 155391 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.981234:0:155391:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787826019.981240:0:155391:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1787826019.981265:0:155391:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x17e0:0x0], updating i_size 4096 00000080:00010000:0.0:1787826019.981273:0:155391:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) for lock 0xbef42c0221d6ebd6 00000080:00010000:0.0:1787826019.981279:0:155391:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00000080:00010000:0.0:1787826019.981282:0:155391:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6ebd6 from it ffffb97502003cb8 00010000:00010000:0.0:1787826019.981286:0:155391:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6ebd6 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb3d8 expref: -99 pid: 155391 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826019.981293:0:155391:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6ebd6 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb3d8 expref: -99 pid: 155391 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:0.0:1787826019.982001:0:155391:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1787826019.982009:0:155391:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826019.982025:0:155391:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826019.982048:0:155391:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826019.982060:0:155391:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826019.982067:0:155391:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826019.982073:0:155391:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826019.982076:0:155391:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826019.982080:0:155391:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97502003cb8 00010000:00010000:0.0:1787826019.982873:0:155391:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826019.982888:0:155391:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826019.983016:0:155391:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826019.983029:0:155391:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826019.983048:0:155391:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826019.983059:0:155391:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826019.983066:0:155391:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826019.983071:0:155391:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826019.983074:0:155391:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826019.983079:0:155391:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97502003bf8 00010000:00010000:0.0:1787826019.983082:0:155391:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826019.983239:0:155391:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826019.983250:0:155391:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826019.983257:0:155391:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=267 00000080:00200000:1.0:1787826019.985245:0:155391:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 44 00000080:00200008:1.0:1787826019.985262:0:155391:0:(file.c:1090:ll_file_open()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00000080:00200080:1.0:1787826019.985294:0:155391:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) cmd=c03066f0 arg=7fffc2eab7e0 00000080:00200000:1.0:1787826019.985308:0:155391:0:(dir.c:451:ll_dir_setdirstripe()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) name=test stripe_offset=0 stripe_count=2, hash_type=0 00010000:00010000:1.0:1787826019.985344:0:155391:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6ebd6 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ceb3d8 expref: -99 pid: 155391 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826019.985370:0:155391:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6800/0xbef42c0221d64b3f lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9634 expref: -99 pid: 152202 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826019.985416:0:155391:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6ebd6 lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ceb3d8 expref: -99 pid: 155391 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826019.985424:0:155391:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826019.985427:0:155391:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6ebd6 lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ceb3d8 expref: -99 pid: 155391 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:1.0:1787826019.985447:0:155391:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787826019.985459:0:155391:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c35b0380 x1874663042905344/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 552/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826019.985482:0:155391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155391:1874663042905344:192.168.203.147@tcp:36:lfs.0 00000100:00100000:1.0:1787826019.985614:0:155391:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c80c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826020.001174:0:155391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155391:1874663042905344:192.168.203.147@tcp:36:lfs.0 00000080:00200000:0.0:1787826020.001223:0:155391:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x20001a9e3:0x1bb3:0x0](ffff9181c83cb248) 00000080:00200000:0.0:1787826020.001230:0:155391:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1bb3:0x0], updating i_size 4096 00000080:00200008:0.0:1787826020.001260:0:155391:0:(file.c:479:ll_file_release()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00010000:00010000:0.0:1787826020.001286:0:155391:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513000/0xbef42c0221d64bee lrc: 2/0,0 mode: CR/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce962d expref: -99 pid: 152210 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826020.018327:0:155399:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826020.018355:0:155399:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826020.018373:0:155399:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826020.018390:0:155399:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826020.018399:0:155399:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826020.018406:0:155399:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826020.018410:0:155399:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826020.018417:0:155399:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97502003c20 00010000:00010000:0.0:1787826020.018420:0:155399:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826020.018429:0:155399:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826020.018441:0:155399:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826020.018448:0:155399:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=24 00000080:00200000:0.0:1787826020.018454:0:155399:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826020.018466:0:155399:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=0 00000080:00200000:0.0:1787826020.018470:0:155399:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787826020.018482:0:155399:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787826020.018528:0:155399:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6ebdd lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155399 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826020.018535:0:155399:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6ebdd lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155399 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826020.018546:0:155399:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6ebdd lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155399 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826020.018573:0:155399:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155399:1874663042905472:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787826020.024132:0:155399:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bbe40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826020.073724:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826020.073847:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049144448 00000100:00100000:0.0:1787826020.073858:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049144448 00000100:00100000:0.0:1787826020.073921:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9211 00000100:00100000:0.0:1787826020.073931:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:188682:x1874663049144448:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826020.073949:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481600/0xbef42c0221d6ebe4 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x27:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceb42c expref: -99 pid: 155399 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826020.074017:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826020.074023:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:188682:x1874663049144448:12345-192.168.203.147@tcp:104: Request processed in 94us (311us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826020.074031:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9211 00000100:00100000:1.0:1787826020.088199:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826020.088327:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049144704 00000100:00100000:0.0:1787826020.088336:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049144704 00000100:00100000:0.0:1787826020.088389:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9212 00000100:00100000:0.0:1787826020.088400:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b2680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663049144704:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826020.088417:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480000/0xbef42c0221d6ebeb lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x27:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceb43a expref: -99 pid: 155399 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826020.088497:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826020.088504:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b2680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663049144704:12345-192.168.203.147@tcp:104: Request processed in 106us (315us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826020.088511:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9212 00010000:00010000:0.0:1787826020.088531:0:15161:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826020.088537:0:15161:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480000/0xbef42c0221d6ebeb lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x27:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceb43a expref: -99 pid: 155399 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826020.088546:0:15161:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3480000 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787826020.088551:0:15161:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480000/0xbef42c0221d6ebeb lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x27:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00ceb43a expref: -99 pid: 155399 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826020.088559:0:15161:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480000/0xbef42c0221d6ebeb lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x27:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00ceb43a expref: -99 pid: 155399 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826020.088568:0:15161:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480000/0xbef42c0221d6ebeb lrc: 3/0,0 mode: PR/PR res: [0x240001f58:0x27:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x408400000000 nid: local remote: 0x878a5eea00ceb43a expref: -99 pid: 155399 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826020.088621:0:15161:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480000/0xbef42c0221d6ebeb lrc: 2/0,0 mode: --/PR res: [0x240001f58:0x27:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ceb43a expref: -99 pid: 155399 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826020.088629:0:15161:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826020.088650:0:15161:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480000/0xbef42c0221d6ebeb lrc: 1/0,0 mode: --/PR res: [0x240001f58:0x27:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ceb43a expref: -99 pid: 155399 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826020.088669:0:15161:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480000/0xbef42c0221d6ebeb lrc: 0/0,0 mode: --/PR res: [0x240001f58:0x27:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ceb43a expref: -99 pid: 155399 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826020.088706:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826020.088709:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826020.088711:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826020.088726:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042906240:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787826020.102350:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042906240:192.168.203.147@tcp:103:ldlm_bl.0 00000001:02000400:0.0:1787826020.158486:0:155405:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: oleg347-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 00000100:00100000:1.0:1787826020.207447:0:155406:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f2d80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155406:1874663042906624:192.168.203.147@tcp:101:lfs.0 00010000:00010000:1.0:1787826020.207473:0:155406:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6ec00 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb49c expref: -99 pid: 155406 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.207489:0:155406:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6ec00 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb49c expref: -99 pid: 155406 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.207499:0:155406:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6ec00 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb49c expref: -99 pid: 155406 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.207511:0:155406:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6ec00 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb49c expref: -99 pid: 155406 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.207522:0:155406:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6ec00 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb49c expref: -99 pid: 155406 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.207535:0:155406:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6ec00 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb49c expref: -99 pid: 155406 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.207544:0:155406:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6ec00 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb49c expref: -99 pid: 155406 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.207551:0:155406:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6ec00 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb49c expref: -99 pid: 155406 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826020.207562:0:155406:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c71f2d80 x1874663042906624/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826036 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826020.207574:0:155406:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6ec00 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb49c expref: -99 pid: 155406 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.207583:0:155406:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/6112 (0 0) 00000080:00200000:1.0:1787826020.207589:0:155406:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787826020.207614:0:155406:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x17e0:0x0], updating i_size 4096 00000080:00010000:1.0:1787826020.207620:0:155406:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) for lock 0xbef42c0221d6ec00 00000080:00010000:1.0:1787826020.207661:0:155406:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00000080:00010000:1.0:1787826020.207664:0:155406:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6ec00 from it ffffb9750150bcb8 00010000:00010000:1.0:1787826020.207668:0:155406:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6ec00 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb49c expref: -99 pid: 155406 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.207677:0:155406:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6ec00 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb49c expref: -99 pid: 155406 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:0.0:1787826020.210801:0:155406:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1787826020.210811:0:155406:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826020.210834:0:155406:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826020.210853:0:155406:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826020.210868:0:155406:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826020.210876:0:155406:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826020.210885:0:155406:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826020.210889:0:155406:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826020.210896:0:155406:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb9750150bcb8 00010000:00010000:0.0:1787826020.210901:0:155406:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826020.210910:0:155406:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826020.211051:0:155406:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826020.211062:0:155406:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826020.211069:0:155406:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826020.211079:0:155406:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826020.211085:0:155406:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826020.211089:0:155406:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826020.211092:0:155406:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826020.211096:0:155406:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb9750150bbf8 00010000:00010000:0.0:1787826020.211100:0:155406:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826020.211108:0:155406:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826020.211118:0:155406:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826020.211124:0:155406:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=267 00000080:00200000:0.0:1787826020.211130:0:155406:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 44 00000080:00200008:0.0:1787826020.211137:0:155406:0:(file.c:1090:ll_file_open()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00000080:00200080:0.0:1787826020.211158:0:155406:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) cmd=c03066f0 arg=7ffcae198410 00000080:00200000:0.0:1787826020.211171:0:155406:0:(dir.c:451:ll_dir_setdirstripe()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) name=test stripe_offset=0 stripe_count=2, hash_type=0 00010000:00010000:0.0:1787826020.211210:0:155406:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6ec00 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ceb49c expref: -99 pid: 155406 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826020.211227:0:155406:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6800/0xbef42c0221d64b3f lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9634 expref: -99 pid: 152202 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826020.211275:0:155406:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6ec00 lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ceb49c expref: -99 pid: 155406 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826020.211283:0:155406:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826020.211286:0:155406:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6ec00 lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ceb49c expref: -99 pid: 155406 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:0.0:1787826020.212258:0:155406:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787826020.212275:0:155406:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c55a4700 x1874663042906880/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 552/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826020.212305:0:155406:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155406:1874663042906880:192.168.203.147@tcp:36:lfs.0 00000100:00100000:0.0:1787826020.212524:0:155406:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bb240 going to sleep for 11 seconds 00000100:00100000:0.0:1787826020.248586:0:155406:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155406:1874663042906880:192.168.203.147@tcp:36:lfs.0 00000080:00200000:0.0:1787826020.248634:0:155406:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x20001a9e3:0x1bb4:0x0](ffff9181c83cb248) 00000080:00200000:0.0:1787826020.248641:0:155406:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1bb4:0x0], updating i_size 4096 00000080:00200008:0.0:1787826020.248740:0:155406:0:(file.c:479:ll_file_release()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00010000:00010000:0.0:1787826020.248797:0:155406:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513000/0xbef42c0221d64bee lrc: 2/0,0 mode: CR/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce962d expref: -99 pid: 152210 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826020.256900:0:155414:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826020.256930:0:155414:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826020.256948:0:155414:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826020.256964:0:155414:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826020.256973:0:155414:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826020.256982:0:155414:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826020.256986:0:155414:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826020.256994:0:155414:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501fe3c20 00010000:00010000:0.0:1787826020.256998:0:155414:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826020.257008:0:155414:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826020.257021:0:155414:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826020.257028:0:155414:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=24 00000080:00200000:0.0:1787826020.257045:0:155414:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826020.257058:0:155414:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=0 00000080:00200000:0.0:1787826020.257062:0:155414:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787826020.257072:0:155414:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787826020.257112:0:155414:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6ec07 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155414 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826020.257119:0:155414:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6ec07 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155414 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826020.257128:0:155414:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6ec07 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155414 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826020.257153:0:155414:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155414:1874663042907008:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787826020.257249:0:155414:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bb240 going to sleep for 11 seconds 00000100:00100000:0.0:1787826020.277959:0:155414:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155414:1874663042907008:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:0.0:1787826020.277987:0:155414:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6ec07 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb4e2 expref: -99 pid: 155414 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826020.278011:0:155414:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6ec07 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1bb4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb4e2 expref: -99 pid: 155414 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826020.278020:0:155414:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6ec07 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1bb4:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb4e2 expref: -99 pid: 155414 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826020.278031:0:155414:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6ec07 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bb4:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb4e2 expref: -99 pid: 155414 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826020.278056:0:155414:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6ec07 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bb4:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb4e2 expref: -99 pid: 155414 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826020.278069:0:155414:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6ec07 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1bb4:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb4e2 expref: -99 pid: 155414 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826020.278077:0:155414:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6ec07 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1bb4:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb4e2 expref: -99 pid: 155414 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826020.278084:0:155414:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6ec07 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bb4:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb4e2 expref: -99 pid: 155414 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826020.278106:0:155414:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c55a7480 x1874663042907008/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/672 e 0 to 0 dl 1787826036 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787826020.278120:0:155414:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6ec07 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1bb4:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb4e2 expref: -99 pid: 155414 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826020.278130:0:155414:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/7092 (0 0) 00000080:00200000:0.0:1787826020.278136:0:155414:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00000002:00010000:0.0:1787826020.278202:0:155414:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200026560:0x28:0x0]) in obj [0x200026560:0x28:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826020.278245:0:155414:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512a00/0xbef42c0221d6ec0e lrc: 3/1,0 mode: --/CR res: [0x200026560:0x28:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 155414 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826020.278254:0:155414:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512a00/0xbef42c0221d6ec0e lrc: 3/1,0 mode: --/CR res: [0x200026560:0x28:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155414 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826020.278262:0:155414:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512a00/0xbef42c0221d6ec0e lrc: 3/1,0 mode: --/CR res: [0x200026560:0x28:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155414 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826020.278302:0:155414:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155414:1874663042907136:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787826020.278456:0:155414:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bb240 going to sleep for 11 seconds 00000100:00100000:0.0:1787826020.299158:0:155414:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155414:1874663042907136:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:0.0:1787826020.299185:0:155414:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512a00/0xbef42c0221d6ec0e lrc: 4/1,0 mode: --/CR res: [0x200026560:0x28:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb4f0 expref: -99 pid: 155414 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826020.299197:0:155414:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512a00/0xbef42c0221d6ec0e lrc: 4/1,0 mode: --/PR res: [0x200026560:0x28:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb4f0 expref: -99 pid: 155414 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826020.299210:0:155414:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512a00/0xbef42c0221d6ec0e lrc: 4/1,0 mode: PR/PR res: [0x200026560:0x28:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb4f0 expref: -99 pid: 155414 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826020.299222:0:155414:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512a00/0xbef42c0221d6ec0e lrc: 4/1,0 mode: PR/PR res: [0x200026560:0x28:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb4f0 expref: -99 pid: 155414 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826020.299239:0:155414:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512a00/0xbef42c0221d6ec0e lrc: 5/2,0 mode: PR/PR res: [0x200026560:0x28:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb4f0 expref: -99 pid: 155414 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826020.299246:0:155414:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512a00/0xbef42c0221d6ec0e lrc: 5/2,0 mode: PR/PR res: [0x200026560:0x28:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb4f0 expref: -99 pid: 155414 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826020.299253:0:155414:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512a00/0xbef42c0221d6ec0e lrc: 4/1,0 mode: PR/PR res: [0x200026560:0x28:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb4f0 expref: -99 pid: 155414 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826020.299264:0:155414:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c55a5c00 x1874663042907136/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787826036 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787826020.299277:0:155414:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512a00/0xbef42c0221d6ec0e lrc: 3/1,0 mode: PR/PR res: [0x200026560:0x28:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb4f0 expref: -99 pid: 155414 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826020.299289:0:155414:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590091616/40 (0 0) 00010000:00010000:0.0:1787826020.299296:0:155414:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512a00/0xbef42c0221d6ec0e lrc: 3/1,0 mode: PR/PR res: [0x200026560:0x28:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb4f0 expref: -99 pid: 155414 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826020.299304:0:155414:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512a00/0xbef42c0221d6ec0e lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x28:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb4f0 expref: -99 pid: 155414 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000002:00010000:0.0:1787826020.299330:0:155414:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001f58:0x28:0x0]) in obj [0x240001f58:0x28:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826020.299374:0:155414:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6ec15 lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x28:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 155414 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826020.299382:0:155414:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6ec15 lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x28:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155414 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826020.299391:0:155414:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6ec15 lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x28:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155414 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826020.299410:0:155414:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155414:1874663042907264:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787826020.300216:0:155414:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bb240 going to sleep for 11 seconds 00000100:00100000:0.0:1787826020.317289:0:155414:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155414:1874663042907264:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:0.0:1787826020.317379:0:155414:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6ec15 lrc: 4/1,0 mode: --/CR res: [0x240001f58:0x28:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb4fe expref: -99 pid: 155414 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826020.317392:0:155414:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6ec15 lrc: 4/1,0 mode: --/PR res: [0x240001f58:0x28:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb4fe expref: -99 pid: 155414 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826020.317403:0:155414:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6ec15 lrc: 4/1,0 mode: PR/PR res: [0x240001f58:0x28:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb4fe expref: -99 pid: 155414 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826020.317414:0:155414:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6ec15 lrc: 4/1,0 mode: PR/PR res: [0x240001f58:0x28:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb4fe expref: -99 pid: 155414 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826020.317427:0:155414:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6ec15 lrc: 5/2,0 mode: PR/PR res: [0x240001f58:0x28:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb4fe expref: -99 pid: 155414 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826020.317435:0:155414:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6ec15 lrc: 5/2,0 mode: PR/PR res: [0x240001f58:0x28:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb4fe expref: -99 pid: 155414 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826020.317442:0:155414:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6ec15 lrc: 4/1,0 mode: PR/PR res: [0x240001f58:0x28:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb4fe expref: -99 pid: 155414 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826020.317457:0:155414:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c55a5c00 x1874663042907264/t0(0) o101->lustre-MDT0001-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787826036 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787826020.317471:0:155414:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6ec15 lrc: 3/1,0 mode: PR/PR res: [0x240001f58:0x28:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb4fe expref: -99 pid: 155414 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826020.317483:0:155414:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d0f54800 type 13 mode 4 res 9663684440/40 (0 0) 00010000:00010000:0.0:1787826020.317490:0:155414:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6ec15 lrc: 3/1,0 mode: PR/PR res: [0x240001f58:0x28:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb4fe expref: -99 pid: 155414 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826020.317498:0:155414:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6ec15 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x28:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb4fe expref: -99 pid: 155414 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826020.317530:0:155414:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1bb4:0x0], updating i_size 8192 00000080:00200000:0.0:1787826020.317534:0:155414:0:(namei.c:147:ll_iget()) got inode: [0x20001a9e3:0x1bb4:0x0](ffff9181c83cb248): rc = 0 00000080:00010000:0.0:1787826020.317543:0:155414:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x1bb4:0x0](ffff9181c83cb248) for lock 0xbef42c0221d6ec07 00000080:00010000:0.0:1787826020.317568:0:155414:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x1bb4:0x0](ffff9181c83cb248) 00000080:00010000:0.0:1787826020.317572:0:155414:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6ec07 from it ffffb97501fe3e08 00010000:00010000:0.0:1787826020.317575:0:155414:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6ec07 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1bb4:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb4e2 expref: -99 pid: 155414 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826020.317585:0:155414:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6ec07 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1bb4:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb4e2 expref: -99 pid: 155414 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826020.317603:0:155414:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 3 00000080:00200000:0.0:1787826020.317612:0:155414:0:(namei.c:2328:ll_rmdir()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00010000:00010000:0.0:1787826020.317639:0:155414:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6ec07 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1bb4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ceb4e2 expref: -99 pid: 155414 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826020.317682:0:155414:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6ec07 lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x1bb4:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ceb4e2 expref: -99 pid: 155414 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826020.317688:0:155414:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826020.317691:0:155414:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6ec07 lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x1bb4:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ceb4e2 expref: -99 pid: 155414 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:0.0:1787826020.318140:0:155414:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787826020.318147:0:155414:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c55a7480 x1874663042907520/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826020.318172:0:155414:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155414:1874663042907520:192.168.203.147@tcp:36:rmdir.0 00000100:00100000:0.0:1787826020.318282:0:155414:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bb240 going to sleep for 11 seconds 00000100:00100000:0.0:1787826020.327453:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826020.327521:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049146112 00000100:00100000:0.0:1787826020.327531:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049146112 00000100:00100000:0.0:1787826020.327655:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9213 00000100:00100000:0.0:1787826020.327665:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187904:x1874663049146112:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826020.327683:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512a00/0xbef42c0221d6ec0e lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x28:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceb4f0 expref: -99 pid: 155414 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826020.327886:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826020.327894:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187904:x1874663049146112:12345-192.168.203.147@tcp:104: Request processed in 230us (450us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826020.327901:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9213 00010000:00010000:0.0:1787826020.328012:0:15160:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826020.328019:0:15160:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512a00/0xbef42c0221d6ec0e lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x28:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceb4f0 expref: -99 pid: 155414 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826020.328028:0:15160:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0512a00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787826020.328045:0:15160:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512a00/0xbef42c0221d6ec0e lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x28:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00ceb4f0 expref: -99 pid: 155414 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826020.328055:0:15160:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512a00/0xbef42c0221d6ec0e lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x28:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00ceb4f0 expref: -99 pid: 155414 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826020.328064:0:15160:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512a00/0xbef42c0221d6ec0e lrc: 3/0,0 mode: PR/PR res: [0x200026560:0x28:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x408400000000 nid: local remote: 0x878a5eea00ceb4f0 expref: -99 pid: 155414 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826020.328118:0:15160:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512a00/0xbef42c0221d6ec0e lrc: 2/0,0 mode: --/PR res: [0x200026560:0x28:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ceb4f0 expref: -99 pid: 155414 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826020.328125:0:15160:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826020.328145:0:15160:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512a00/0xbef42c0221d6ec0e lrc: 1/0,0 mode: --/PR res: [0x200026560:0x28:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ceb4f0 expref: -99 pid: 155414 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826020.328153:0:15160:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512a00/0xbef42c0221d6ec0e lrc: 0/0,0 mode: --/PR res: [0x200026560:0x28:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ceb4f0 expref: -99 pid: 155414 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826020.328168:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826020.328170:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826020.328172:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826020.328187:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042907648:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1787826020.331191:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042907648:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1787826020.333885:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826020.333927:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049146368 00000100:00100000:0.0:1787826020.333938:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049146368 00000100:00100000:0.0:1787826020.333987:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9214 00000100:00100000:0.0:1787826020.333998:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663049146368:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826020.334017:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6ec15 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x28:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceb4fe expref: -99 pid: 155414 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826020.334104:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826020.334111:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663049146368:12345-192.168.203.147@tcp:104: Request processed in 115us (235us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826020.334119:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9214 00000100:00100000:0.0:1787826020.359986:0:155414:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155414:1874663042907520:192.168.203.147@tcp:36:rmdir.0 00000080:00200000:0.0:1787826020.360030:0:155414:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1bb4:0x0](ffff9181c83cb248) 00000100:00100000:1.0:1787826020.439789:0:155445:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f1880 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155445:1874663042908544:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:1.0:1787826020.439815:0:155445:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6ec31 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb5a6 expref: -99 pid: 155445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.439838:0:155445:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6ec31 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1bb5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb5a6 expref: -99 pid: 155445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.439846:0:155445:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6ec31 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1bb5:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb5a6 expref: -99 pid: 155445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.439857:0:155445:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6ec31 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bb5:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb5a6 expref: -99 pid: 155445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.439869:0:155445:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6ec31 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bb5:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb5a6 expref: -99 pid: 155445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.439882:0:155445:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6ec31 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1bb5:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb5a6 expref: -99 pid: 155445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.439895:0:155445:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6ec31 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1bb5:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb5a6 expref: -99 pid: 155445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.439903:0:155445:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6ec31 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bb5:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb5a6 expref: -99 pid: 155445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826020.439916:0:155445:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c71f1880 x1874663042908544/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/672 e 0 to 0 dl 1787826036 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826020.439930:0:155445:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6ec31 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1bb5:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb5a6 expref: -99 pid: 155445 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.439940:0:155445:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/7093 (0 0) 00000080:00200000:1.0:1787826020.439946:0:155445:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00000002:00010000:1.0:1787826020.440001:0:155445:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200026560:0x29:0x0]) in obj [0x200026560:0x29:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826020.440059:0:155445:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6ec38 lrc: 3/1,0 mode: --/CR res: [0x200026560:0x29:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 155445 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.440068:0:155445:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6ec38 lrc: 3/1,0 mode: --/CR res: [0x200026560:0x29:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155445 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.440076:0:155445:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6ec38 lrc: 3/1,0 mode: --/CR res: [0x200026560:0x29:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155445 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826020.440237:0:155445:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155445:1874663042908672:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:1.0:1787826020.440298:0:155445:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8f00 going to sleep for 11 seconds 00000100:00100000:0.0:1787826020.476928:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826020.476968:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049147776 00000100:00100000:0.0:1787826020.476976:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049147776 00000100:00100000:0.0:1787826020.477026:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9215 00000100:00100000:0.0:1787826020.477046:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a4000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:188682:x1874663049147776:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826020.477064:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6ec38 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x29:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceb5b4 expref: -99 pid: 155445 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826020.477152:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826020.477159:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a4000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:188682:x1874663049147776:12345-192.168.203.147@tcp:104: Request processed in 125us (240us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826020.477166:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9215 00000100:00100000:0.0:1787826020.494521:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826020.494570:0:33333:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:0.0:1787826020.494587:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049148032 00000100:00100000:0.0:1787826020.494596:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049148032 00000100:00100000:0.0:1787826020.494644:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9216 00000100:00100000:0.0:1787826020.494653:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181e053e050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663049148032:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826020.495029:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6c00/0xbef42c0221d6ec3f lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x29:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceb5c2 expref: -99 pid: 155445 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826020.495528:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826020.495537:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181e053e050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663049148032:12345-192.168.203.147@tcp:104: Request processed in 884us (1024us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826020.495546:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9216 00000100:00100000:1.0:1787826020.523830:0:155445:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c494f480 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155445:1874663042909056:192.168.203.147@tcp:36:rmdir.0 00000080:00200000:0.0:1787826020.523981:0:155445:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1bb5:0x0](ffff9181c83fa1c8) 00000100:00100000:1.0:1787826020.559873:0:155458:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f0700 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155458:1874663042909440:192.168.203.147@tcp:101:lfs.0 00010000:00010000:1.0:1787826020.559897:0:155458:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6600/0xbef42c0221d6ec46 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155458 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.559910:0:155458:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6600/0xbef42c0221d6ec46 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155458 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.559917:0:155458:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6600/0xbef42c0221d6ec46 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155458 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.559923:0:155458:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6600/0xbef42c0221d6ec46 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155458 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.559931:0:155458:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6600/0xbef42c0221d6ec46 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155458 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.559937:0:155458:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e06f6600 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:1.0:1787826020.559942:0:155458:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6600/0xbef42c0221d6ec46 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155458 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.559949:0:155458:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6600/0xbef42c0221d6ec46 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155458 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.559955:0:155458:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6600/0xbef42c0221d6ec46 lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 155458 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.559961:0:155458:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787826020.559967:0:155458:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6600/0xbef42c0221d6ec46 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155458 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.559972:0:155458:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6600/0xbef42c0221d6ec46 lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155458 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826020.559988:0:155458:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c71f0700 x1874663042909440/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787826036 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787826020.560002:0:155458:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:1.0:1787826020.560079:0:155458:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:1.0:1787826020.560970:0:155458:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826020.560989:0:155458:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826020.561001:0:155458:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826020.561013:0:155458:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826020.561020:0:155458:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826020.561025:0:155458:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826020.561028:0:155458:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787826020.561046:0:155458:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501c43be8 00010000:00010000:1.0:1787826020.561050:0:155458:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826020.561058:0:155458:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826020.561071:0:155458:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826020.561079:0:155458:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=16408 00000080:00200000:1.0:1787826020.561836:0:155458:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826020.561853:0:155458:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=16392 00000080:00200000:1.0:1787826020.561858:0:155458:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:1.0:1787826020.561874:0:155458:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787826020.561921:0:155458:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4200/0xbef42c0221d6ec4d lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155458 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.561929:0:155458:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4200/0xbef42c0221d6ec4d lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155458 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.561939:0:155458:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4200/0xbef42c0221d6ec4d lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155458 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826020.561964:0:155458:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155458:1874663042909568:192.168.203.147@tcp:101:lfs.0 00000100:00100000:1.0:1787826020.562065:0:155458:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826020.689473:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826020.689520:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049149440 00000100:00100000:1.0:1787826020.689529:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049149440 00000100:00100000:1.0:1787826020.689578:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9217 00000100:00100000:1.0:1787826020.689587:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b3b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187904:x1874663049149440:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826020.689604:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481200/0xbef42c0221d6ec62 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x2a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceb678 expref: -99 pid: 155469 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826020.689673:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826020.689679:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b3b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187904:x1874663049149440:12345-192.168.203.147@tcp:104: Request processed in 94us (218us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826020.689687:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9217 00010000:00010000:1.0:1787826020.689709:0:15160:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826020.689715:0:15160:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481200/0xbef42c0221d6ec62 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x2a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceb678 expref: -99 pid: 155469 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826020.689730:0:15160:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3481200 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:1.0:1787826020.689735:0:15160:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481200/0xbef42c0221d6ec62 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x2a:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00ceb678 expref: -99 pid: 155469 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826020.689885:0:15160:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481200/0xbef42c0221d6ec62 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x2a:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00ceb678 expref: -99 pid: 155469 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826020.689896:0:15160:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481200/0xbef42c0221d6ec62 lrc: 3/0,0 mode: PR/PR res: [0x200026560:0x2a:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x408400000000 nid: local remote: 0x878a5eea00ceb678 expref: -99 pid: 155469 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826020.689950:0:15160:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481200/0xbef42c0221d6ec62 lrc: 2/0,0 mode: --/PR res: [0x200026560:0x2a:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ceb678 expref: -99 pid: 155469 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826020.689959:0:15160:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826020.689981:0:15160:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481200/0xbef42c0221d6ec62 lrc: 1/0,0 mode: --/PR res: [0x200026560:0x2a:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ceb678 expref: -99 pid: 155469 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826020.689989:0:15160:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481200/0xbef42c0221d6ec62 lrc: 0/0,0 mode: --/PR res: [0x200026560:0x2a:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ceb678 expref: -99 pid: 155469 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826020.690007:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826020.690009:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826020.690010:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826020.690027:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042910720:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787826020.696113:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826020.696173:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042910720:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1787826020.698549:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049149696 00000100:00100000:0.0:1787826020.698564:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049149696 00000100:00100000:0.0:1787826020.698619:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9218 00000100:00100000:0.0:1787826020.698631:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b0700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663049149696:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826020.698650:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481800/0xbef42c0221d6ec69 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x2a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceb686 expref: -99 pid: 155469 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826020.698737:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826020.698776:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b0700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663049149696:12345-192.168.203.147@tcp:104: Request processed in 147us (2672us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826020.698785:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9218 00000100:00100000:1.0:1787826020.757327:0:155469:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c494ea00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155469:1874663042910592:192.168.203.147@tcp:36:rmdir.0 00000080:00200000:1.0:1787826020.757372:0:155469:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1bb6:0x0](ffff9181c83ee3c8) 00000080:00200000:0.0:1787826020.781062:0:155492:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826020.781078:0:155492:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=16392 00000080:00200000:0.0:1787826020.781084:0:155492:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787826020.781102:0:155492:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787826020.781151:0:155492:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6ec77 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155492 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826020.781160:0:155492:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6ec77 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155492 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826020.781169:0:155492:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6ec77 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155492 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826020.781189:0:155492:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155492:1874663042911104:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787826020.781269:0:155492:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bbcc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826020.784188:0:155492:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155492:1874663042911104:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787826020.784206:0:155492:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6ec77 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155492 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826020.784216:0:155492:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6ec77 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155492 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826020.784221:0:155492:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6ec77 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155492 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826020.784226:0:155492:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6ec77 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155492 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826020.784232:0:155492:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6ec77 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155492 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826020.784236:0:155492:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0513800 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787826020.784240:0:155492:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6ec77 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155492 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826020.784245:0:155492:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6ec77 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155492 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826020.784250:0:155492:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6ec77 lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 155492 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826020.784254:0:155492:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787826020.784260:0:155492:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6ec77 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155492 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826020.784264:0:155492:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6ec77 lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155492 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826020.784274:0:155492:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c55a5c00 x1874663042911104/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787826036 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787826020.784285:0:155492:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787826020.784306:0:155492:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787826020.784358:0:155492:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826020.784368:0:155492:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826020.784374:0:155492:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826020.784381:0:155492:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826020.784385:0:155492:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826020.784388:0:155492:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826020.784391:0:155492:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826020.784394:0:155492:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501c83b58 00010000:00010000:0.0:1787826020.784397:0:155492:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826020.784402:0:155492:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826020.784409:0:155492:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826020.784414:0:155492:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=13 00000080:00200008:0.0:1787826020.784420:0:155492:0:(file.c:6296:ll_getattr_dentry()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1787826020.784424:0:155492:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08),name=d71a.replay-single 00000002:00010000:0.0:1787826020.784441:0:155492:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x20001a9e3:0x17e0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826020.784445:0:155492:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00010000:00010000:0.0:1787826020.784470:0:155492:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6ec7e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 155492 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826020.784478:0:155492:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6ec7e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 155492 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826020.784487:0:155492:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6ec7e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 155492 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826020.784504:0:155492:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155492:1874663042911232:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787826020.784574:0:155492:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bbcc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826020.788809:0:155492:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155492:1874663042911232:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787826020.788836:0:155492:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6ec7e lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb6e8 expref: -99 pid: 155492 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826020.788860:0:155492:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6ec7e lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb6e8 expref: -99 pid: 155492 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826020.788868:0:155492:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6ec7e lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb6e8 expref: -99 pid: 155492 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826020.788880:0:155492:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6ec7e lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb6e8 expref: -99 pid: 155492 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826020.788891:0:155492:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6ec7e lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb6e8 expref: -99 pid: 155492 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826020.788904:0:155492:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6ec7e lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb6e8 expref: -99 pid: 155492 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826020.788912:0:155492:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6ec7e lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb6e8 expref: -99 pid: 155492 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826020.788921:0:155492:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6ec7e lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb6e8 expref: -99 pid: 155492 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826020.788932:0:155492:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c55a5c00 x1874663042911232/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826036 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787826020.788946:0:155492:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6ec7e lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb6e8 expref: -99 pid: 155492 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826020.788961:0:155492:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787826020.788968:0:155492:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1787826020.788993:0:155492:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x17e0:0x0], updating i_size 4096 00000080:00010000:0.0:1787826020.789000:0:155492:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) for lock 0xbef42c0221d6ec7e 00000080:00010000:0.0:1787826020.789007:0:155492:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00000080:00010000:0.0:1787826020.789010:0:155492:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6ec7e from it ffffb97501c83cb8 00010000:00010000:0.0:1787826020.789013:0:155492:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6ec7e lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb6e8 expref: -99 pid: 155492 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826020.789022:0:155492:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6ec7e lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb6e8 expref: -99 pid: 155492 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:0.0:1787826020.789573:0:155492:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1787826020.789579:0:155492:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826020.789596:0:155492:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826020.789606:0:155492:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826020.789617:0:155492:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826020.789623:0:155492:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826020.789628:0:155492:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826020.789631:0:155492:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826020.789635:0:155492:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501c83cb8 00010000:00010000:0.0:1787826020.789638:0:155492:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826020.789646:0:155492:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826020.789769:0:155492:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826020.789778:0:155492:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826020.789785:0:155492:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826020.789794:0:155492:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826020.789801:0:155492:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826020.789806:0:155492:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826020.789808:0:155492:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826020.789812:0:155492:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501c83bf8 00010000:00010000:0.0:1787826020.789816:0:155492:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826020.789823:0:155492:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826020.789832:0:155492:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826020.789838:0:155492:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=267 00000080:00200000:0.0:1787826020.789843:0:155492:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 44 00000080:00200008:0.0:1787826020.789848:0:155492:0:(file.c:1090:ll_file_open()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00000080:00200080:0.0:1787826020.789868:0:155492:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) cmd=c03066f0 arg=7ffe29c40fe0 00000080:00200000:0.0:1787826020.789881:0:155492:0:(dir.c:451:ll_dir_setdirstripe()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) name=test stripe_offset=0 stripe_count=2, hash_type=0 00010000:00010000:0.0:1787826020.789905:0:155492:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6ec7e lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ceb6e8 expref: -99 pid: 155492 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826020.789919:0:155492:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6800/0xbef42c0221d64b3f lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9634 expref: -99 pid: 152202 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826020.789958:0:155492:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6ec7e lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ceb6e8 expref: -99 pid: 155492 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826020.789965:0:155492:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826020.789967:0:155492:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6ec7e lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ceb6e8 expref: -99 pid: 155492 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:0.0:1787826020.789985:0:155492:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787826020.789992:0:155492:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c55a5c00 x1874663042911488/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 552/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826020.790013:0:155492:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155492:1874663042911488:192.168.203.147@tcp:36:lfs.0 00000100:00100000:0.0:1787826020.790100:0:155492:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bbcc0 going to sleep for 11 seconds 00000080:00200000:1.0:1787826020.828911:0:155507:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826020.828943:0:155507:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826020.828962:0:155507:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826020.828980:0:155507:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826020.828989:0:155507:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826020.828997:0:155507:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826020.829000:0:155507:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787826020.829007:0:155507:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501c6bc20 00010000:00010000:1.0:1787826020.829011:0:155507:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826020.829021:0:155507:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826020.833608:0:155507:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826020.833625:0:155507:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=24 00000080:00200000:1.0:1787826020.833633:0:155507:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826020.833707:0:155507:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=0 00000080:00200000:1.0:1787826020.833712:0:155507:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:1.0:1787826020.833740:0:155507:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787826020.833819:0:155507:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6ec85 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155507 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.833830:0:155507:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6ec85 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155507 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.833841:0:155507:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6ec85 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155507 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826020.833863:0:155507:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155507:1874663042911616:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:1.0:1787826020.833947:0:155507:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c86c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826020.845319:0:155507:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155507:1874663042911616:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:1.0:1787826020.845346:0:155507:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6ec85 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb72e expref: -99 pid: 155507 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.845370:0:155507:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6ec85 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1bb7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb72e expref: -99 pid: 155507 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.845379:0:155507:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6ec85 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1bb7:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb72e expref: -99 pid: 155507 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.845390:0:155507:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6ec85 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bb7:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb72e expref: -99 pid: 155507 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.845401:0:155507:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6ec85 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bb7:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb72e expref: -99 pid: 155507 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.845413:0:155507:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6ec85 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1bb7:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb72e expref: -99 pid: 155507 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.845426:0:155507:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6ec85 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1bb7:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb72e expref: -99 pid: 155507 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.845432:0:155507:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6ec85 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bb7:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb72e expref: -99 pid: 155507 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826020.845442:0:155507:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c35b2a00 x1874663042911616/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/672 e 0 to 0 dl 1787826036 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826020.845456:0:155507:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6ec85 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1bb7:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb72e expref: -99 pid: 155507 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.845467:0:155507:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/7095 (0 0) 00000080:00200000:1.0:1787826020.845474:0:155507:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00000002:00010000:1.0:1787826020.845526:0:155507:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200026560:0x2b:0x0]) in obj [0x200026560:0x2b:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826020.845571:0:155507:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6ec8c lrc: 3/1,0 mode: --/CR res: [0x200026560:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 155507 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.845579:0:155507:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6ec8c lrc: 3/1,0 mode: --/CR res: [0x200026560:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155507 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.845588:0:155507:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6ec8c lrc: 3/1,0 mode: --/CR res: [0x200026560:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155507 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826020.845607:0:155507:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2300 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155507:1874663042911744:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:1.0:1787826020.845687:0:155507:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c86c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826020.879810:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826020.879848:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049151104 00000100:00100000:1.0:1787826020.879858:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049151104 00000100:00100000:1.0:1787826020.879907:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9219 00000100:00100000:1.0:1787826020.879918:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b0e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:188682:x1874663049151104:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826020.879935:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6ec8c lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x2b:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceb73c expref: -99 pid: 155507 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826020.880004:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826020.880011:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b0e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:188682:x1874663049151104:12345-192.168.203.147@tcp:104: Request processed in 96us (211us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826020.880018:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9219 00000100:00100000:0.0:1787826020.889206:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826020.889574:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049151360 00000100:00100000:1.0:1787826020.889584:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049151360 00000100:00100000:1.0:1787826020.889635:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9220 00000100:00100000:1.0:1787826020.889645:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663049151360:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826020.889673:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481800/0xbef42c0221d6ec93 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x2b:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceb74a expref: -99 pid: 155507 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826020.889763:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826020.889770:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663049151360:12345-192.168.203.147@tcp:104: Request processed in 127us (575us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826020.889777:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9220 00010000:00010000:1.0:1787826020.890046:0:33329:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826020.890053:0:33329:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481800/0xbef42c0221d6ec93 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x2b:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceb74a expref: -99 pid: 155507 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826020.890063:0:33329:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3481800 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:1.0:1787826020.890068:0:33329:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481800/0xbef42c0221d6ec93 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x2b:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00ceb74a expref: -99 pid: 155507 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826020.890077:0:33329:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481800/0xbef42c0221d6ec93 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x2b:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00ceb74a expref: -99 pid: 155507 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826020.890188:0:33329:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481800/0xbef42c0221d6ec93 lrc: 3/0,0 mode: PR/PR res: [0x240001f58:0x2b:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x408400000000 nid: local remote: 0x878a5eea00ceb74a expref: -99 pid: 155507 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826020.890235:0:33329:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481800/0xbef42c0221d6ec93 lrc: 2/0,0 mode: --/PR res: [0x240001f58:0x2b:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ceb74a expref: -99 pid: 155507 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826020.890242:0:33329:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826020.890263:0:33329:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481800/0xbef42c0221d6ec93 lrc: 1/0,0 mode: --/PR res: [0x240001f58:0x2b:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ceb74a expref: -99 pid: 155507 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826020.890271:0:33329:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481800/0xbef42c0221d6ec93 lrc: 0/0,0 mode: --/PR res: [0x240001f58:0x2b:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ceb74a expref: -99 pid: 155507 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826020.890289:0:33329:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826020.890291:0:33329:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826020.890293:0:33329:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826020.890308:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042912384:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787826020.896905:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042912384:192.168.203.147@tcp:103:ldlm_bl.0 00000080:00200000:1.0:1787826020.920450:0:155520:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826020.920486:0:155520:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826020.920508:0:155520:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826020.920530:0:155520:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826020.920539:0:155520:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826020.920546:0:155520:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826020.920550:0:155520:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787826020.920557:0:155520:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501d2bb58 00010000:00010000:1.0:1787826020.920561:0:155520:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826020.920571:0:155520:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826020.920585:0:155520:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826020.920593:0:155520:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=29 00000080:00200000:1.0:1787826020.920598:0:155520:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826020.920608:0:155520:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=13 00000080:00200000:1.0:1787826020.920612:0:155520:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:1.0:1787826020.920624:0:155520:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787826020.920665:0:155520:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6ec9a lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155520 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.920674:0:155520:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6ec9a lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155520 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.920683:0:155520:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6ec9a lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155520 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826020.920712:0:155520:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155520:1874663042912512:192.168.203.147@tcp:101:lfs.0 00000100:00100000:1.0:1787826020.920821:0:155520:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8900 going to sleep for 11 seconds 00000100:00100000:1.0:1787826020.929399:0:155520:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155520:1874663042912512:192.168.203.147@tcp:101:lfs.0 00010000:00010000:1.0:1787826020.929423:0:155520:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6ec9a lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155520 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.929436:0:155520:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6ec9a lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155520 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.929444:0:155520:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6ec9a lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155520 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.929452:0:155520:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6ec9a lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155520 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.929512:0:155520:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6ec9a lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155520 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.929521:0:155520:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b5200 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:1.0:1787826020.929525:0:155520:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6ec9a lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155520 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.929533:0:155520:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6ec9a lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155520 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.929540:0:155520:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6ec9a lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 155520 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.929547:0:155520:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787826020.929555:0:155520:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6ec9a lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155520 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.929562:0:155520:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6ec9a lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155520 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826020.929581:0:155520:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c35b3800 x1874663042912512/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787826036 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787826020.929597:0:155520:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:1.0:1787826020.929624:0:155520:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:1.0:1787826020.929692:0:155520:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826020.930063:0:155520:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826020.930151:0:155520:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826020.930165:0:155520:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826020.930173:0:155520:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826020.930179:0:155520:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826020.930183:0:155520:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787826020.930188:0:155520:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501d2bbe8 00010000:00010000:1.0:1787826020.930192:0:155520:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826020.930201:0:155520:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826020.930214:0:155520:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826020.930222:0:155520:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=16408 00000080:00200000:1.0:1787826020.930228:0:155520:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826020.930237:0:155520:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=16392 00000080:00200000:1.0:1787826020.930242:0:155520:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:1.0:1787826020.930253:0:155520:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787826020.930290:0:155520:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4e00/0xbef42c0221d6eca1 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155520 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.930298:0:155520:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4e00/0xbef42c0221d6eca1 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155520 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.930307:0:155520:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4e00/0xbef42c0221d6eca1 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155520 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826020.930327:0:155520:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155520:1874663042912640:192.168.203.147@tcp:101:lfs.0 00000100:00100000:1.0:1787826020.930514:0:155520:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8900 going to sleep for 11 seconds 00000100:00100000:1.0:1787826020.942538:0:155520:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155520:1874663042912640:192.168.203.147@tcp:101:lfs.0 00010000:00010000:1.0:1787826020.942561:0:155520:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4e00/0xbef42c0221d6eca1 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155520 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.942575:0:155520:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4e00/0xbef42c0221d6eca1 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155520 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.942583:0:155520:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4e00/0xbef42c0221d6eca1 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155520 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.942591:0:155520:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4e00/0xbef42c0221d6eca1 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155520 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.942600:0:155520:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4e00/0xbef42c0221d6eca1 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155520 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.942607:0:155520:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4e00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:1.0:1787826020.942612:0:155520:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4e00/0xbef42c0221d6eca1 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155520 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.942619:0:155520:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4e00/0xbef42c0221d6eca1 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155520 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.942627:0:155520:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4e00/0xbef42c0221d6eca1 lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 155520 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.942635:0:155520:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787826020.942641:0:155520:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4e00/0xbef42c0221d6eca1 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155520 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.942660:0:155520:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4e00/0xbef42c0221d6eca1 lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155520 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826020.942677:0:155520:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c35b3800 x1874663042912640/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787826036 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787826020.942692:0:155520:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:1.0:1787826020.942938:0:155520:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:1.0:1787826020.942994:0:155520:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826020.943009:0:155520:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826020.943017:0:155520:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826020.943040:0:155520:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826020.943053:0:155520:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826020.943059:0:155520:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826020.943062:0:155520:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787826020.943068:0:155520:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501d2bb58 00010000:00010000:1.0:1787826020.943072:0:155520:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826020.943149:0:155520:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826020.943163:0:155520:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826020.943171:0:155520:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=13 00000080:00200008:1.0:1787826020.943179:0:155520:0:(file.c:6296:ll_getattr_dentry()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), request_mask 0, flags 0, foreign 0 00000080:00200000:1.0:1787826020.943184:0:155520:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08),name=d71a.replay-single 00000002:00010000:1.0:1787826020.943196:0:155520:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x20001a9e3:0x17e0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826020.943203:0:155520:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00010000:00010000:1.0:1787826020.943239:0:155520:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6eca8 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 155520 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.943246:0:155520:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6eca8 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 155520 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.943256:0:155520:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6eca8 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 155520 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826020.943278:0:155520:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155520:1874663042912768:192.168.203.147@tcp:101:lfs.0 00000100:00100000:1.0:1787826020.943353:0:155520:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8900 going to sleep for 11 seconds 00000100:00100000:1.0:1787826020.955920:0:155520:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155520:1874663042912768:192.168.203.147@tcp:101:lfs.0 00010000:00010000:1.0:1787826020.955949:0:155520:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6eca8 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb7ac expref: -99 pid: 155520 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.955966:0:155520:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6eca8 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb7ac expref: -99 pid: 155520 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.955975:0:155520:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6eca8 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb7ac expref: -99 pid: 155520 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.955987:0:155520:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6eca8 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb7ac expref: -99 pid: 155520 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.955998:0:155520:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6eca8 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb7ac expref: -99 pid: 155520 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.956011:0:155520:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6eca8 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb7ac expref: -99 pid: 155520 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.956019:0:155520:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6eca8 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb7ac expref: -99 pid: 155520 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.956027:0:155520:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6eca8 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb7ac expref: -99 pid: 155520 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826020.956054:0:155520:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c35b3800 x1874663042912768/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826036 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826020.956068:0:155520:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6eca8 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb7ac expref: -99 pid: 155520 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.956088:0:155520:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/6112 (0 0) 00000080:00200000:1.0:1787826020.956095:0:155520:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787826020.956119:0:155520:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x17e0:0x0], updating i_size 4096 00000080:00010000:1.0:1787826020.956126:0:155520:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) for lock 0xbef42c0221d6eca8 00000080:00010000:1.0:1787826020.956132:0:155520:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00000080:00010000:1.0:1787826020.956135:0:155520:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6eca8 from it ffffb97501d2bcb8 00010000:00010000:1.0:1787826020.956138:0:155520:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6eca8 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb7ac expref: -99 pid: 155520 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826020.956148:0:155520:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6eca8 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb7ac expref: -99 pid: 155520 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:1.0:1787826020.956645:0:155520:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), request_mask 0, flags 0, foreign 0 00000080:00200000:1.0:1787826020.956651:0:155520:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826020.956667:0:155520:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826020.956678:0:155520:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826020.956689:0:155520:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826020.957169:0:155520:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826020.957175:0:155520:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826020.957178:0:155520:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787826020.957183:0:155520:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501d2bcb8 00010000:00010000:1.0:1787826020.957187:0:155520:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826020.957197:0:155520:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826020.957333:0:155520:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826020.957343:0:155520:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826020.957351:0:155520:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826020.957361:0:155520:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826020.957368:0:155520:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826020.957372:0:155520:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826020.957374:0:155520:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787826020.957378:0:155520:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501d2bbf8 00010000:00010000:1.0:1787826020.957381:0:155520:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826020.957389:0:155520:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826020.957398:0:155520:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826020.957404:0:155520:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=267 00000080:00200000:1.0:1787826020.957409:0:155520:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 44 00000080:00200008:1.0:1787826020.957415:0:155520:0:(file.c:1090:ll_file_open()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00000080:00200080:1.0:1787826020.957435:0:155520:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) cmd=c03066f0 arg=7fffce481ce0 00000080:00200000:1.0:1787826020.957448:0:155520:0:(dir.c:451:ll_dir_setdirstripe()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) name=test stripe_offset=0 stripe_count=2, hash_type=0 00010000:00010000:1.0:1787826020.957477:0:155520:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6eca8 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ceb7ac expref: -99 pid: 155520 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826020.957493:0:155520:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6800/0xbef42c0221d64b3f lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9634 expref: -99 pid: 152202 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826020.957531:0:155520:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6eca8 lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ceb7ac expref: -99 pid: 155520 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826020.957538:0:155520:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826020.957541:0:155520:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6eca8 lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ceb7ac expref: -99 pid: 155520 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:1.0:1787826020.957561:0:155520:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787826020.957571:0:155520:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c35b3800 x1874663042913024/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 552/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826020.957592:0:155520:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155520:1874663042913024:192.168.203.147@tcp:36:lfs.0 00000100:00100000:1.0:1787826020.957664:0:155520:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8900 going to sleep for 11 seconds 00000100:00100000:1.0:1787826021.059782:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826021.059820:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049152896 00000100:00100000:1.0:1787826021.059829:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049152896 00000100:00100000:1.0:1787826021.059875:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9221 00000100:00100000:1.0:1787826021.059885:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:188682:x1874663049152896:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826021.059904:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481800/0xbef42c0221d6ecb6 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x2c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceb800 expref: -99 pid: 155528 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826021.059977:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826021.059983:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:188682:x1874663049152896:12345-192.168.203.147@tcp:104: Request processed in 100us (210us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826021.059991:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9221 00010000:00010000:1.0:1787826021.060016:0:15160:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826021.060022:0:15160:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481800/0xbef42c0221d6ecb6 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x2c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceb800 expref: -99 pid: 155528 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826021.060049:0:15160:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3481800 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:1.0:1787826021.060053:0:15160:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481800/0xbef42c0221d6ecb6 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x2c:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00ceb800 expref: -99 pid: 155528 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826021.060063:0:15160:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481800/0xbef42c0221d6ecb6 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x2c:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00ceb800 expref: -99 pid: 155528 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826021.060071:0:15160:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481800/0xbef42c0221d6ecb6 lrc: 3/0,0 mode: PR/PR res: [0x200026560:0x2c:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x408400000000 nid: local remote: 0x878a5eea00ceb800 expref: -99 pid: 155528 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826021.060447:0:15160:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481800/0xbef42c0221d6ecb6 lrc: 2/0,0 mode: --/PR res: [0x200026560:0x2c:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ceb800 expref: -99 pid: 155528 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826021.060455:0:15160:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826021.060484:0:15160:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481800/0xbef42c0221d6ecb6 lrc: 1/0,0 mode: --/PR res: [0x200026560:0x2c:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ceb800 expref: -99 pid: 155528 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826021.060492:0:15160:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481800/0xbef42c0221d6ecb6 lrc: 0/0,0 mode: --/PR res: [0x200026560:0x2c:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ceb800 expref: -99 pid: 155528 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826021.060517:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826021.060518:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826021.060520:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826021.060527:0:2389:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1787826021.060540:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042913792:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1787826021.063811:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042913792:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1787826021.078183:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826021.078227:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049153152 00000100:00100000:1.0:1787826021.078237:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049153152 00000100:00100000:1.0:1787826021.078289:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9222 00000100:00100000:1.0:1787826021.078301:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a4000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663049153152:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826021.078318:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480c00/0xbef42c0221d6ecbd lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x2c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceb80e expref: -99 pid: 155528 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826021.080836:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826021.080855:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a4000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663049153152:12345-192.168.203.147@tcp:104: Request processed in 2551us (2675us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826021.080867:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9222 00010000:00010000:0.0:1787826021.089248:0:33350:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480c00/0xbef42c0221d6ecbd lrc: 2/0,0 mode: --/PR res: [0x240001f58:0x2c:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ceb80e expref: -99 pid: 155528 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826021.089265:0:33350:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826021.089309:0:33350:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480c00/0xbef42c0221d6ecbd lrc: 1/0,0 mode: --/PR res: [0x240001f58:0x2c:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ceb80e expref: -99 pid: 155528 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826021.089319:0:33350:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480c00/0xbef42c0221d6ecbd lrc: 0/0,0 mode: --/PR res: [0x240001f58:0x2c:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ceb80e expref: -99 pid: 155528 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000100:00100000:0.0:1787826021.089355:0:2390:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1787826021.089451:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042913920:192.168.203.147@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826021.089715:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826021.089717:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826021.089720:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826021.097165:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042913920:192.168.203.147@tcp:103:ldlm_bl.0 00000080:00200000:0.0:1787826021.210818:0:155537:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826021.210850:0:155537:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826021.210868:0:155537:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826021.210886:0:155537:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826021.210894:0:155537:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826021.210903:0:155537:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826021.210907:0:155537:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826021.210915:0:155537:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97500d53c20 00010000:00010000:0.0:1787826021.210919:0:155537:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826021.210928:0:155537:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826021.210942:0:155537:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826021.210949:0:155537:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=24 00000080:00200000:0.0:1787826021.210956:0:155537:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826021.210966:0:155537:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=0 00000080:00200000:0.0:1787826021.210970:0:155537:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787826021.210983:0:155537:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787826021.211458:0:155537:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6ecd9 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155537 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826021.211468:0:155537:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6ecd9 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155537 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826021.211480:0:155537:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6ecd9 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155537 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826021.211509:0:155537:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155537:1874663042914688:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787826021.211602:0:155537:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bbd80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826021.218391:0:155537:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155537:1874663042914688:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:0.0:1787826021.218419:0:155537:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6ecd9 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb8b6 expref: -99 pid: 155537 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826021.218440:0:155537:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6ecd9 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1bb9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb8b6 expref: -99 pid: 155537 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826021.218448:0:155537:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6ecd9 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1bb9:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb8b6 expref: -99 pid: 155537 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826021.218461:0:155537:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6ecd9 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bb9:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb8b6 expref: -99 pid: 155537 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826021.218471:0:155537:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6ecd9 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bb9:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb8b6 expref: -99 pid: 155537 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826021.218483:0:155537:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6ecd9 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1bb9:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb8b6 expref: -99 pid: 155537 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826021.218491:0:155537:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6ecd9 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1bb9:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb8b6 expref: -99 pid: 155537 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826021.218498:0:155537:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6ecd9 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bb9:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb8b6 expref: -99 pid: 155537 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826021.218509:0:155537:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c55a6680 x1874663042914688/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/672 e 0 to 0 dl 1787826037 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787826021.218524:0:155537:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6ecd9 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1bb9:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb8b6 expref: -99 pid: 155537 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826021.218534:0:155537:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/7097 (0 0) 00000080:00200000:0.0:1787826021.218541:0:155537:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00000002:00010000:0.0:1787826021.218596:0:155537:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200026560:0x2d:0x0]) in obj [0x200026560:0x2d:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826021.218638:0:155537:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6ece0 lrc: 3/1,0 mode: --/CR res: [0x200026560:0x2d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 155537 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826021.218650:0:155537:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6ece0 lrc: 3/1,0 mode: --/CR res: [0x200026560:0x2d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155537 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826021.218658:0:155537:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6ece0 lrc: 3/1,0 mode: --/CR res: [0x200026560:0x2d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155537 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826021.218676:0:155537:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155537:1874663042914816:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787826021.218792:0:155537:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bbd80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826021.226516:0:155537:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155537:1874663042914816:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:0.0:1787826021.226555:0:155537:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6ece0 lrc: 4/1,0 mode: --/CR res: [0x200026560:0x2d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb8c4 expref: -99 pid: 155537 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826021.226567:0:155537:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6ece0 lrc: 4/1,0 mode: --/PR res: [0x200026560:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb8c4 expref: -99 pid: 155537 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826021.226578:0:155537:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6ece0 lrc: 4/1,0 mode: PR/PR res: [0x200026560:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb8c4 expref: -99 pid: 155537 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826021.226588:0:155537:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6ece0 lrc: 4/1,0 mode: PR/PR res: [0x200026560:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb8c4 expref: -99 pid: 155537 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826021.226600:0:155537:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6ece0 lrc: 5/2,0 mode: PR/PR res: [0x200026560:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb8c4 expref: -99 pid: 155537 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826021.226608:0:155537:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6ece0 lrc: 5/2,0 mode: PR/PR res: [0x200026560:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb8c4 expref: -99 pid: 155537 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826021.226620:0:155537:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6ece0 lrc: 4/1,0 mode: PR/PR res: [0x200026560:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb8c4 expref: -99 pid: 155537 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826021.226631:0:155537:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c55a5f80 x1874663042914816/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787826037 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787826021.226646:0:155537:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6ece0 lrc: 3/1,0 mode: PR/PR res: [0x200026560:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb8c4 expref: -99 pid: 155537 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826021.226657:0:155537:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590091616/45 (0 0) 00010000:00010000:0.0:1787826021.226665:0:155537:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6ece0 lrc: 3/1,0 mode: PR/PR res: [0x200026560:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb8c4 expref: -99 pid: 155537 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826021.226673:0:155537:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6ece0 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb8c4 expref: -99 pid: 155537 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000002:00010000:0.0:1787826021.226699:0:155537:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001f58:0x2d:0x0]) in obj [0x240001f58:0x2d:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826021.226737:0:155537:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6ece7 lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x2d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 155537 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826021.226774:0:155537:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6ece7 lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x2d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155537 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826021.226781:0:155537:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6ece7 lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x2d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155537 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826021.226796:0:155537:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155537:1874663042914944:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787826021.226845:0:155537:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bbd80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826021.239582:0:155537:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155537:1874663042914944:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:0.0:1787826021.239609:0:155537:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6ece7 lrc: 4/1,0 mode: --/CR res: [0x240001f58:0x2d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb8d2 expref: -99 pid: 155537 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826021.239620:0:155537:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6ece7 lrc: 4/1,0 mode: --/PR res: [0x240001f58:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb8d2 expref: -99 pid: 155537 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826021.239631:0:155537:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6ece7 lrc: 4/1,0 mode: PR/PR res: [0x240001f58:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb8d2 expref: -99 pid: 155537 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826021.239641:0:155537:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6ece7 lrc: 4/1,0 mode: PR/PR res: [0x240001f58:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb8d2 expref: -99 pid: 155537 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826021.239651:0:155537:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6ece7 lrc: 5/2,0 mode: PR/PR res: [0x240001f58:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb8d2 expref: -99 pid: 155537 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826021.239658:0:155537:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6ece7 lrc: 5/2,0 mode: PR/PR res: [0x240001f58:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb8d2 expref: -99 pid: 155537 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826021.239665:0:155537:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6ece7 lrc: 4/1,0 mode: PR/PR res: [0x240001f58:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb8d2 expref: -99 pid: 155537 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826021.239674:0:155537:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c55a5f80 x1874663042914944/t0(0) o101->lustre-MDT0001-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787826037 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787826021.239687:0:155537:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6ece7 lrc: 3/1,0 mode: PR/PR res: [0x240001f58:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb8d2 expref: -99 pid: 155537 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826021.239698:0:155537:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d0f54800 type 13 mode 4 res 9663684440/45 (0 0) 00010000:00010000:0.0:1787826021.239706:0:155537:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6ece7 lrc: 3/1,0 mode: PR/PR res: [0x240001f58:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb8d2 expref: -99 pid: 155537 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826021.239713:0:155537:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6ece7 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb8d2 expref: -99 pid: 155537 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826021.239775:0:155537:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1bb9:0x0], updating i_size 8192 00000080:00200000:0.0:1787826021.239778:0:155537:0:(namei.c:147:ll_iget()) got inode: [0x20001a9e3:0x1bb9:0x0](ffff9181c83faa08): rc = 0 00000080:00010000:0.0:1787826021.239786:0:155537:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x1bb9:0x0](ffff9181c83faa08) for lock 0xbef42c0221d6ecd9 00000080:00010000:0.0:1787826021.239802:0:155537:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x1bb9:0x0](ffff9181c83faa08) 00000080:00010000:0.0:1787826021.239805:0:155537:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6ecd9 from it ffffb97500d53e08 00010000:00010000:0.0:1787826021.239808:0:155537:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6ecd9 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1bb9:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb8b6 expref: -99 pid: 155537 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826021.239815:0:155537:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6ecd9 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1bb9:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb8b6 expref: -99 pid: 155537 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826021.239831:0:155537:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 3 00000080:00200000:0.0:1787826021.239837:0:155537:0:(namei.c:2328:ll_rmdir()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00010000:00010000:0.0:1787826021.239872:0:155537:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6ecd9 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1bb9:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ceb8b6 expref: -99 pid: 155537 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826021.239914:0:155537:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6ecd9 lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x1bb9:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ceb8b6 expref: -99 pid: 155537 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826021.239919:0:155537:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826021.239922:0:155537:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6ecd9 lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x1bb9:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ceb8b6 expref: -99 pid: 155537 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:0.0:1787826021.239947:0:155537:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787826021.239953:0:155537:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c55a6680 x1874663042915200/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826021.239971:0:155537:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155537:1874663042915200:192.168.203.147@tcp:36:rmdir.0 00000100:00100000:0.0:1787826021.240054:0:155537:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bbd80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826021.247131:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826021.247173:0:15150:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:0.0:1787826021.247189:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049154560 00000100:00100000:0.0:1787826021.247198:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049154560 00000100:00100000:0.0:1787826021.247252:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9223 00000100:00100000:0.0:1787826021.247262:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c44ea850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:188682:x1874663049154560:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826021.247280:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6ece0 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceb8c4 expref: -99 pid: 155537 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826021.247356:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826021.247365:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c44ea850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:188682:x1874663049154560:12345-192.168.203.147@tcp:104: Request processed in 104us (241us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826021.247373:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9223 00000100:00100000:0.0:1787826021.271201:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826021.271260:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049154816 00000100:00100000:1.0:1787826021.271270:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049154816 00000100:00100000:1.0:1787826021.271315:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9224 00000100:00100000:1.0:1787826021.271324:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8414:x1874663049154816:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826021.271340:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6ece7 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceb8d2 expref: -99 pid: 155537 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826021.271411:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826021.271418:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8414:x1874663049154816:12345-192.168.203.147@tcp:104: Request processed in 95us (224us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826021.271426:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9224 00000080:00200000:0.0:1787826021.355387:0:155545:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x17e0:0x0], updating i_size 4096 00000080:00010000:0.0:1787826021.355462:0:155545:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) for lock 0xbef42c0221d6ecfc 00000080:00010000:0.0:1787826021.355472:0:155545:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00000080:00010000:0.0:1787826021.355476:0:155545:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6ecfc from it ffffb97501d6bcb8 00010000:00010000:0.0:1787826021.355479:0:155545:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6ecfc lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceb934 expref: -99 pid: 155545 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826021.355489:0:155545:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6ecfc lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceb934 expref: -99 pid: 155545 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:0.0:1787826021.356594:0:155545:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1787826021.356603:0:155545:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826021.356623:0:155545:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826021.356638:0:155545:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826021.356650:0:155545:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826021.356657:0:155545:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826021.356663:0:155545:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826021.356666:0:155545:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826021.356672:0:155545:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501d6bcb8 00010000:00010000:0.0:1787826021.356675:0:155545:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826021.356683:0:155545:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826021.356920:0:155545:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826021.356929:0:155545:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826021.356937:0:155545:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826021.356950:0:155545:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826021.356956:0:155545:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826021.356961:0:155545:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826021.356964:0:155545:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826021.356967:0:155545:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501d6bbf8 00010000:00010000:0.0:1787826021.356970:0:155545:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826021.356979:0:155545:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826021.356988:0:155545:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826021.356994:0:155545:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=267 00000080:00200000:0.0:1787826021.356999:0:155545:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 44 00000080:00200008:0.0:1787826021.357004:0:155545:0:(file.c:1090:ll_file_open()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00000080:00200080:0.0:1787826021.357037:0:155545:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) cmd=c03066f0 arg=7fffe5329910 00000080:00200000:0.0:1787826021.357050:0:155545:0:(dir.c:451:ll_dir_setdirstripe()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) name=test stripe_offset=0 stripe_count=2, hash_type=0 00010000:00010000:0.0:1787826021.357287:0:155545:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6ecfc lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ceb934 expref: -99 pid: 155545 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826021.357305:0:155545:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6800/0xbef42c0221d64b3f lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9634 expref: -99 pid: 152202 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826021.357348:0:155545:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6ecfc lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ceb934 expref: -99 pid: 155545 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826021.357357:0:155545:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826021.357359:0:155545:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6ecfc lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ceb934 expref: -99 pid: 155545 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:0.0:1787826021.357379:0:155545:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787826021.357391:0:155545:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c55a4000 x1874663042916096/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 552/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826021.357716:0:155545:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4000 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155545:1874663042916096:192.168.203.147@tcp:36:lfs.0 00000100:00100000:0.0:1787826021.358098:0:155545:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bbf00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826021.367644:0:155545:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4000 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155545:1874663042916096:192.168.203.147@tcp:36:lfs.0 00000080:00200000:1.0:1787826021.367702:0:155545:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x20001a9e3:0x1bba:0x0](ffff9181c83e7448) 00000080:00200000:1.0:1787826021.367710:0:155545:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1bba:0x0], updating i_size 4096 00000080:00200008:1.0:1787826021.367783:0:155545:0:(file.c:479:ll_file_release()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00010000:00010000:1.0:1787826021.367825:0:155545:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513000/0xbef42c0221d64bee lrc: 2/0,0 mode: CR/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce962d expref: -99 pid: 152210 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000100:00100000:1.0:1787826021.428674:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826021.429205:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049156224 00000100:00100000:0.0:1787826021.429217:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049156224 00000100:00100000:0.0:1787826021.429280:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9225 00000100:00100000:0.0:1787826021.429289:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b3b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187904:x1874663049156224:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826021.429306:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480000/0xbef42c0221d6ed0a lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x2e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceb988 expref: -99 pid: 155550 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826021.429382:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826021.429389:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b3b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187904:x1874663049156224:12345-192.168.203.147@tcp:104: Request processed in 101us (724us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826021.429397:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9225 00000100:00100000:1.0:1787826021.444879:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826021.444925:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049156480 00000100:00100000:0.0:1787826021.444935:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049156480 00000100:00100000:0.0:1787826021.444989:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9226 00000100:00100000:0.0:1787826021.445001:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b0700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663049156480:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826021.445018:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6ed11 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x2e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceb996 expref: -99 pid: 155550 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826021.445157:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826021.445174:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b0700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663049156480:12345-192.168.203.147@tcp:104: Request processed in 166us (294us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826021.445182:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9226 00000080:00200000:1.0:1787826021.528354:0:155563:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826021.528384:0:155563:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826021.528403:0:155563:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826021.528419:0:155563:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826021.528427:0:155563:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826021.528435:0:155563:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826021.528438:0:155563:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787826021.528446:0:155563:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501d2bb58 00010000:00010000:1.0:1787826021.528450:0:155563:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826021.528459:0:155563:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826021.528474:0:155563:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826021.528481:0:155563:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=29 00000080:00200000:1.0:1787826021.528487:0:155563:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826021.528499:0:155563:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=13 00000080:00200000:1.0:1787826021.528503:0:155563:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:1.0:1787826021.528515:0:155563:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787826021.528563:0:155563:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6ed18 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155563 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826021.528571:0:155563:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6ed18 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155563 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826021.528580:0:155563:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6ed18 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155563 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826021.528605:0:155563:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155563:1874663042917120:192.168.203.147@tcp:101:lfs.0 00000100:00100000:1.0:1787826021.528709:0:155563:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b596c0 going to sleep for 11 seconds 00000080:00200000:0.0:1787826021.592489:0:155570:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826021.592527:0:155570:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826021.592548:0:155570:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826021.592565:0:155570:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826021.592574:0:155570:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826021.592584:0:155570:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826021.592589:0:155570:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826021.592597:0:155570:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb9750161bc20 00010000:00010000:0.0:1787826021.592602:0:155570:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826021.592610:0:155570:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826021.592623:0:155570:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826021.592631:0:155570:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=24 00000080:00200000:0.0:1787826021.592637:0:155570:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826021.592651:0:155570:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=0 00000080:00200000:0.0:1787826021.592657:0:155570:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787826021.592670:0:155570:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787826021.592725:0:155570:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6ed2d lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155570 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826021.592733:0:155570:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6ed2d lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155570 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826021.592762:0:155570:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6ed2d lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155570 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826021.592793:0:155570:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155570:1874663042917760:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787826021.592907:0:155570:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bb240 going to sleep for 11 seconds 00000100:00100000:0.0:1787826021.606688:0:155570:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155570:1874663042917760:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:0.0:1787826021.606719:0:155570:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6ed2d lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceba3e expref: -99 pid: 155570 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826021.606767:0:155570:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6ed2d lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1bbb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceba3e expref: -99 pid: 155570 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826021.606778:0:155570:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6ed2d lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1bbb:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceba3e expref: -99 pid: 155570 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826021.606797:0:155570:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6ed2d lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bbb:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceba3e expref: -99 pid: 155570 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826021.606809:0:155570:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6ed2d lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bbb:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceba3e expref: -99 pid: 155570 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826021.606827:0:155570:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6ed2d lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1bbb:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceba3e expref: -99 pid: 155570 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826021.606835:0:155570:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6ed2d lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1bbb:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceba3e expref: -99 pid: 155570 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826021.606841:0:155570:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6ed2d lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bbb:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceba3e expref: -99 pid: 155570 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826021.606853:0:155570:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c55a5f80 x1874663042917760/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/672 e 0 to 0 dl 1787826037 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787826021.606868:0:155570:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512400/0xbef42c0221d6ed2d lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1bbb:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceba3e expref: -99 pid: 155570 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826021.606880:0:155570:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/7099 (0 0) 00000080:00200000:0.0:1787826021.606887:0:155570:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00000002:00010000:0.0:1787826021.606956:0:155570:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200026560:0x2f:0x0]) in obj [0x200026560:0x2f:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826021.607007:0:155570:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6ed34 lrc: 3/1,0 mode: --/CR res: [0x200026560:0x2f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 155570 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826021.607035:0:155570:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6ed34 lrc: 3/1,0 mode: --/CR res: [0x200026560:0x2f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155570 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826021.607045:0:155570:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6ed34 lrc: 3/1,0 mode: --/CR res: [0x200026560:0x2f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155570 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826021.607071:0:155570:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155570:1874663042917888:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787826021.607161:0:155570:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bb240 going to sleep for 11 seconds 00000100:00100000:0.0:1787826021.641503:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826021.641548:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049157888 00000100:00100000:0.0:1787826021.641558:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049157888 00000100:00100000:0.0:1787826021.641611:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9227 00000100:00100000:0.0:1787826021.641623:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:188682:x1874663049157888:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826021.641829:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6ed34 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x2f:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceba4c expref: -99 pid: 155570 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826021.641941:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826021.641949:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:188682:x1874663049157888:12345-192.168.203.147@tcp:104: Request processed in 328us (454us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826021.641957:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9227 00000100:00100000:0.0:1787826021.655357:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826021.655700:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049158144 00000100:00100000:1.0:1787826021.655711:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049158144 00000100:00100000:1.0:1787826021.655809:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9228 00000100:00100000:1.0:1787826021.655821:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663049158144:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826021.655840:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f7a00/0xbef42c0221d6ed3b lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x2f:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceba5a expref: -99 pid: 155570 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826021.655926:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826021.655933:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663049158144:12345-192.168.203.147@tcp:104: Request processed in 115us (588us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826021.655945:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9228 00000080:00200000:1.0:1787826021.775767:0:155581:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826021.775798:0:155581:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826021.775819:0:155581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826021.775836:0:155581:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826021.775844:0:155581:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826021.775852:0:155581:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826021.775855:0:155581:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787826021.775863:0:155581:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501603c20 00010000:00010000:1.0:1787826021.775867:0:155581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826021.775876:0:155581:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826021.775890:0:155581:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826021.775897:0:155581:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=24 00000080:00200000:1.0:1787826021.775903:0:155581:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826021.775915:0:155581:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=0 00000080:00200000:1.0:1787826021.775918:0:155581:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:1.0:1787826021.775930:0:155581:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787826021.775971:0:155581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6ed57 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155581 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826021.775983:0:155581:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6ed57 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155581 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826021.775993:0:155581:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5c00/0xbef42c0221d6ed57 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155581 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826021.776018:0:155581:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155581:1874663042919296:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:1.0:1787826021.777021:0:155581:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b599c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826021.834133:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826021.834226:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049159552 00000100:00100000:0.0:1787826021.834238:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049159552 00000100:00100000:0.0:1787826021.835548:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9229 00000100:00100000:0.0:1787826021.835561:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b3100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187905:x1874663049159552:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826021.835582:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6ed5e lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x30:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cebb10 expref: -99 pid: 155581 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826021.835680:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826021.835687:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b3100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187905:x1874663049159552:12345-192.168.203.147@tcp:104: Request processed in 129us (1565us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826021.835694:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9229 00000100:00100000:0.0:1787826021.844435:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826021.844640:0:15151:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:1.0:1787826021.844664:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049159808 00000100:00100000:1.0:1787826021.844673:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049159808 00000100:00100000:1.0:1787826021.844774:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9230 00000100:00100000:1.0:1787826021.844784:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c44eb450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663049159808:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826021.844801:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481400/0xbef42c0221d6ed65 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x30:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cebb1e expref: -99 pid: 155581 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826021.844866:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826021.844873:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c44eb450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663049159808:12345-192.168.203.147@tcp:104: Request processed in 90us (443us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826021.844879:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9230 00010000:00010000:1.0:1787826021.845055:0:15161:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826021.845061:0:15161:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481400/0xbef42c0221d6ed65 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x30:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cebb1e expref: -99 pid: 155581 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826021.845187:0:15161:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3481400 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:1.0:1787826021.845193:0:15161:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481400/0xbef42c0221d6ed65 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x30:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00cebb1e expref: -99 pid: 155581 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826021.845202:0:15161:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481400/0xbef42c0221d6ed65 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x30:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00cebb1e expref: -99 pid: 155581 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826021.845210:0:15161:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481400/0xbef42c0221d6ed65 lrc: 3/0,0 mode: PR/PR res: [0x240001f58:0x30:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x408400000000 nid: local remote: 0x878a5eea00cebb1e expref: -99 pid: 155581 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826021.845263:0:15161:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481400/0xbef42c0221d6ed65 lrc: 2/0,0 mode: --/PR res: [0x240001f58:0x30:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cebb1e expref: -99 pid: 155581 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826021.845270:0:15161:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:0.0:1787826021.845320:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042920064:192.168.203.147@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826021.845391:0:15161:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481400/0xbef42c0221d6ed65 lrc: 1/0,0 mode: --/PR res: [0x240001f58:0x30:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cebb1e expref: -99 pid: 155581 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826021.845400:0:15161:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481400/0xbef42c0221d6ed65 lrc: 0/0,0 mode: --/PR res: [0x240001f58:0x30:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cebb1e expref: -99 pid: 155581 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826021.845419:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826021.845421:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826021.845423:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826021.848857:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042920064:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787826021.881589:0:155581:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c494ca80 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155581:1874663042919808:192.168.203.147@tcp:36:rmdir.0 00000080:00200000:1.0:1787826021.881633:0:155581:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1bbc:0x0](ffff9181c83ea1c8) 00000100:00100000:1.0:1787826021.919300:0:155589:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f2a00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155589:1874663042920192:192.168.203.147@tcp:101:lfs.0 00010000:00010000:1.0:1787826021.919326:0:155589:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6ed6c lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155589 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826021.919346:0:155589:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6ed6c lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155589 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826021.919353:0:155589:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6ed6c lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155589 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826021.919361:0:155589:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6ed6c lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155589 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826021.919371:0:155589:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6ed6c lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155589 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826021.919378:0:155589:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e06f6e00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:1.0:1787826021.919383:0:155589:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6ed6c lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155589 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826021.919390:0:155589:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6ed6c lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155589 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826021.919397:0:155589:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6ed6c lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 155589 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826021.919404:0:155589:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787826021.919410:0:155589:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6ed6c lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155589 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826021.919416:0:155589:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6ed6c lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155589 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826021.919433:0:155589:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c71f2a00 x1874663042920192/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787826037 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787826021.919449:0:155589:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:1.0:1787826021.922336:0:155589:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:1.0:1787826021.922407:0:155589:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826021.922424:0:155589:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826021.922436:0:155589:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826021.922450:0:155589:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826021.922458:0:155589:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826021.922463:0:155589:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826021.922467:0:155589:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787826021.922485:0:155589:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501603be8 00010000:00010000:1.0:1787826021.922490:0:155589:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826021.922499:0:155589:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826021.922512:0:155589:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826021.922519:0:155589:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=16408 00000080:00200000:1.0:1787826021.922524:0:155589:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826021.922533:0:155589:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=16392 00000080:00200000:1.0:1787826021.922537:0:155589:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:1.0:1787826021.922548:0:155589:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787826021.922589:0:155589:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6ed73 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155589 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826021.922597:0:155589:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6ed73 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155589 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826021.922606:0:155589:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6ed73 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155589 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826021.922630:0:155589:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155589:1874663042920320:192.168.203.147@tcp:101:lfs.0 00000100:00100000:1.0:1787826021.922870:0:155589:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b59540 going to sleep for 11 seconds 00000100:00100000:1.0:1787826021.928393:0:155589:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155589:1874663042920320:192.168.203.147@tcp:101:lfs.0 00010000:00010000:1.0:1787826021.928419:0:155589:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6ed73 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155589 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826021.928433:0:155589:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6ed73 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155589 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826021.928440:0:155589:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6ed73 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155589 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826021.928449:0:155589:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6ed73 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155589 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826021.928458:0:155589:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6ed73 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155589 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826021.928466:0:155589:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4c00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:1.0:1787826021.928587:0:155589:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6ed73 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155589 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826021.928597:0:155589:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6ed73 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155589 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826021.928605:0:155589:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6ed73 lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 155589 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826021.928614:0:155589:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787826021.928620:0:155589:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6ed73 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155589 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826021.928627:0:155589:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6ed73 lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155589 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826021.928647:0:155589:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c35b3b80 x1874663042920320/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787826037 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787826021.928662:0:155589:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:1.0:1787826021.928686:0:155589:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:1.0:1787826021.928766:0:155589:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826021.928780:0:155589:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826021.928789:0:155589:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826021.928800:0:155589:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826021.928808:0:155589:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826021.928814:0:155589:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826021.928817:0:155589:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787826021.928822:0:155589:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501603b58 00010000:00010000:1.0:1787826021.928826:0:155589:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826021.928836:0:155589:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826021.928848:0:155589:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826021.928855:0:155589:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=13 00000080:00200008:1.0:1787826021.928864:0:155589:0:(file.c:6296:ll_getattr_dentry()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), request_mask 0, flags 0, foreign 0 00000080:00200000:1.0:1787826021.928870:0:155589:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08),name=d71a.replay-single 00000002:00010000:1.0:1787826021.928884:0:155589:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x20001a9e3:0x17e0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826021.928890:0:155589:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00010000:00010000:1.0:1787826021.928926:0:155589:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6ed7a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 155589 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826021.928934:0:155589:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6ed7a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 155589 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826021.928944:0:155589:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6ed7a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 155589 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826021.928965:0:155589:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155589:1874663042920448:192.168.203.147@tcp:101:lfs.0 00000100:00100000:1.0:1787826021.929054:0:155589:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b59540 going to sleep for 11 seconds 00000100:00100000:1.0:1787826021.933868:0:155589:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155589:1874663042920448:192.168.203.147@tcp:101:lfs.0 00010000:00010000:1.0:1787826021.933896:0:155589:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6ed7a lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cebb80 expref: -99 pid: 155589 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826021.933914:0:155589:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6ed7a lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cebb80 expref: -99 pid: 155589 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826021.933928:0:155589:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6ed7a lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cebb80 expref: -99 pid: 155589 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826021.933939:0:155589:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6ed7a lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cebb80 expref: -99 pid: 155589 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826021.933949:0:155589:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6ed7a lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cebb80 expref: -99 pid: 155589 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826021.933961:0:155589:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6ed7a lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cebb80 expref: -99 pid: 155589 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826021.933969:0:155589:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6ed7a lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cebb80 expref: -99 pid: 155589 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826021.933977:0:155589:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6ed7a lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cebb80 expref: -99 pid: 155589 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826021.933989:0:155589:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c35b3b80 x1874663042920448/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826037 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826021.934002:0:155589:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6ed7a lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cebb80 expref: -99 pid: 155589 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826021.934012:0:155589:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/6112 (0 0) 00000080:00200000:1.0:1787826021.934018:0:155589:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787826021.934056:0:155589:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x17e0:0x0], updating i_size 4096 00000080:00010000:1.0:1787826021.934063:0:155589:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) for lock 0xbef42c0221d6ed7a 00000080:00010000:1.0:1787826021.934069:0:155589:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00000080:00010000:1.0:1787826021.934184:0:155589:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6ed7a from it ffffb97501603cb8 00010000:00010000:1.0:1787826021.934190:0:155589:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6ed7a lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cebb80 expref: -99 pid: 155589 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826021.934199:0:155589:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6ed7a lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cebb80 expref: -99 pid: 155589 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:1.0:1787826021.936323:0:155589:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), request_mask 0, flags 0, foreign 0 00000080:00200000:1.0:1787826021.936333:0:155589:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826021.936353:0:155589:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826021.936368:0:155589:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826021.936380:0:155589:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826021.936387:0:155589:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826021.936392:0:155589:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826021.936395:0:155589:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787826021.936400:0:155589:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501603cb8 00010000:00010000:1.0:1787826021.936403:0:155589:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826021.936411:0:155589:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826021.936537:0:155589:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826021.936546:0:155589:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826021.936554:0:155589:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826021.936563:0:155589:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826021.936569:0:155589:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826021.936574:0:155589:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826021.936577:0:155589:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787826021.936581:0:155589:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501603bf8 00010000:00010000:1.0:1787826021.936584:0:155589:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826021.936591:0:155589:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826021.936600:0:155589:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826021.936607:0:155589:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=267 00000080:00200000:1.0:1787826021.936614:0:155589:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 44 00000080:00200008:1.0:1787826021.936620:0:155589:0:(file.c:1090:ll_file_open()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00000080:00200080:1.0:1787826021.936654:0:155589:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) cmd=c03066f0 arg=7ffcbc60a380 00000080:00200000:1.0:1787826021.936669:0:155589:0:(dir.c:451:ll_dir_setdirstripe()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) name=test stripe_offset=0 stripe_count=2, hash_type=0 00010000:00010000:1.0:1787826021.936698:0:155589:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6ed7a lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00cebb80 expref: -99 pid: 155589 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826021.936718:0:155589:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6800/0xbef42c0221d64b3f lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9634 expref: -99 pid: 152202 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826021.936781:0:155589:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6ed7a lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cebb80 expref: -99 pid: 155589 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826021.936789:0:155589:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826021.936792:0:155589:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6ed7a lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cebb80 expref: -99 pid: 155589 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:1.0:1787826021.936809:0:155589:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787826021.936823:0:155589:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c35b3b80 x1874663042920704/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 552/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826021.936849:0:155589:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155589:1874663042920704:192.168.203.147@tcp:36:lfs.0 00000100:00100000:1.0:1787826021.936931:0:155589:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b59540 going to sleep for 11 seconds 00000100:00100000:1.0:1787826021.959392:0:155589:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155589:1874663042920704:192.168.203.147@tcp:36:lfs.0 00000080:00200000:1.0:1787826021.959445:0:155589:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x20001a9e3:0x1bbd:0x0](ffff9181c83e21c8) 00000080:00200000:1.0:1787826021.959453:0:155589:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1bbd:0x0], updating i_size 4096 00000080:00200008:1.0:1787826021.959486:0:155589:0:(file.c:479:ll_file_release()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00010000:00010000:1.0:1787826021.959515:0:155589:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513000/0xbef42c0221d64bee lrc: 2/0,0 mode: CR/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce962d expref: -99 pid: 152210 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826021.968145:0:155598:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826021.968182:0:155598:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826021.968201:0:155598:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826021.968220:0:155598:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826021.968229:0:155598:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826021.968238:0:155598:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826021.968242:0:155598:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787826021.968251:0:155598:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb975019ebc20 00010000:00010000:1.0:1787826021.968255:0:155598:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826021.968265:0:155598:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826021.968281:0:155598:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826021.968292:0:155598:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=24 00000080:00200000:1.0:1787826021.968299:0:155598:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826021.968312:0:155598:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=0 00000080:00200000:1.0:1787826021.968316:0:155598:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:1.0:1787826021.968330:0:155598:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787826021.968381:0:155598:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6ed81 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155598 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826021.968389:0:155598:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6ed81 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155598 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826021.968398:0:155598:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6ed81 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155598 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826021.968430:0:155598:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155598:1874663042920832:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:1.0:1787826021.968613:0:155598:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b59540 going to sleep for 11 seconds 00000100:00100000:0.0:1787826022.034184:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826022.034220:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049161216 00000100:00100000:0.0:1787826022.034228:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049161216 00000100:00100000:0.0:1787826022.034271:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9231 00000100:00100000:0.0:1787826022.034279:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a5180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:188682:x1874663049161216:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826022.034296:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7000/0xbef42c0221d6ed88 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x31:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cebbd4 expref: -99 pid: 155598 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826022.034452:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826022.034459:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a5180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:188682:x1874663049161216:12345-192.168.203.147@tcp:104: Request processed in 181us (283us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826022.034466:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9231 00000100:00100000:0.0:1787826022.038656:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826022.038727:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049161472 00000100:00100000:1.0:1787826022.038735:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049161472 00000100:00100000:1.0:1787826022.039653:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9232 00000100:00100000:1.0:1787826022.039662:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663049161472:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826022.039678:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f7800/0xbef42c0221d6ed8f lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x31:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cebbe2 expref: -99 pid: 155598 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826022.039767:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826022.039774:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663049161472:12345-192.168.203.147@tcp:104: Request processed in 112us (1125us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826022.039781:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9232 00000080:00200000:1.0:1787826022.087389:0:155616:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826022.087422:0:155616:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826022.087441:0:155616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826022.087458:0:155616:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826022.087467:0:155616:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826022.087474:0:155616:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826022.087477:0:155616:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787826022.087484:0:155616:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501c73b58 00010000:00010000:1.0:1787826022.087488:0:155616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826022.087498:0:155616:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826022.087511:0:155616:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826022.087519:0:155616:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=29 00000080:00200000:1.0:1787826022.087525:0:155616:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826022.087536:0:155616:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=13 00000080:00200000:1.0:1787826022.087540:0:155616:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:1.0:1787826022.087553:0:155616:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787826022.087595:0:155616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5600/0xbef42c0221d6ed96 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155616 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826022.087603:0:155616:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5600/0xbef42c0221d6ed96 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155616 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826022.087612:0:155616:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5600/0xbef42c0221d6ed96 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155616 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826022.087647:0:155616:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155616:1874663042921728:192.168.203.147@tcp:101:lfs.0 00000100:00100000:1.0:1787826022.088122:0:155616:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b59000 going to sleep for 11 seconds 00000100:00100000:1.0:1787826022.269174:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826022.269223:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049162880 00000100:00100000:1.0:1787826022.269236:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049162880 00000100:00100000:1.0:1787826022.269295:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9233 00000100:00100000:1.0:1787826022.269305:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:188682:x1874663049162880:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826022.269327:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7000/0xbef42c0221d6edb2 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x32:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cebc98 expref: -99 pid: 155631 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826022.269428:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826022.269436:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:188682:x1874663049162880:12345-192.168.203.147@tcp:104: Request processed in 132us (274us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826022.269444:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9233 00000100:00100000:1.0:1787826022.287629:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826022.287785:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049163136 00000100:00100000:0.0:1787826022.287796:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049163136 00000100:00100000:0.0:1787826022.287846:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9234 00000100:00100000:0.0:1787826022.287856:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663049163136:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826022.287874:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f7c00/0xbef42c0221d6edb9 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x32:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cebca6 expref: -99 pid: 155631 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826022.287951:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826022.287958:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663049163136:12345-192.168.203.147@tcp:104: Request processed in 104us (350us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826022.287966:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9234 00010000:00010000:0.0:1787826022.287986:0:33329:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826022.287992:0:33329:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f7c00/0xbef42c0221d6edb9 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x32:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cebca6 expref: -99 pid: 155631 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826022.288001:0:33329:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e06f7c00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787826022.288006:0:33329:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f7c00/0xbef42c0221d6edb9 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x32:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00cebca6 expref: -99 pid: 155631 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826022.288015:0:33329:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f7c00/0xbef42c0221d6edb9 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x32:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00cebca6 expref: -99 pid: 155631 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826022.288034:0:33329:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f7c00/0xbef42c0221d6edb9 lrc: 3/0,0 mode: PR/PR res: [0x240001f58:0x32:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x408400000000 nid: local remote: 0x878a5eea00cebca6 expref: -99 pid: 155631 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826022.288094:0:33329:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f7c00/0xbef42c0221d6edb9 lrc: 2/0,0 mode: --/PR res: [0x240001f58:0x32:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cebca6 expref: -99 pid: 155631 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826022.288100:0:33329:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826022.288123:0:33329:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f7c00/0xbef42c0221d6edb9 lrc: 1/0,0 mode: --/PR res: [0x240001f58:0x32:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cebca6 expref: -99 pid: 155631 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826022.288131:0:33329:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f7c00/0xbef42c0221d6edb9 lrc: 0/0,0 mode: --/PR res: [0x240001f58:0x32:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cebca6 expref: -99 pid: 155631 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826022.288153:0:33329:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826022.288156:0:33329:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826022.288157:0:33329:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826022.288174:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042923136:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1787826022.299399:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042923136:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787826022.331780:0:155631:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c494d880 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155631:1874663042922880:192.168.203.147@tcp:36:rmdir.0 00000080:00200000:1.0:1787826022.331830:0:155631:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1bbe:0x0](ffff9181c83fc2c8) 00000080:00200000:0.0:1787826022.355820:0:155650:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826022.355854:0:155650:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826022.355875:0:155650:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826022.355896:0:155650:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826022.355903:0:155650:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826022.355914:0:155650:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826022.355919:0:155650:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826022.355927:0:155650:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501cfbb58 00010000:00010000:0.0:1787826022.355931:0:155650:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826022.355940:0:155650:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826022.355957:0:155650:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826022.355966:0:155650:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=29 00000080:00200000:0.0:1787826022.355973:0:155650:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826022.355985:0:155650:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=13 00000080:00200000:0.0:1787826022.355989:0:155650:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787826022.356003:0:155650:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787826022.361195:0:155650:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6edc0 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155650 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826022.361216:0:155650:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6edc0 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155650 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826022.361228:0:155650:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6edc0 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155650 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826022.361256:0:155650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6300 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155650:1874663042923264:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787826022.361449:0:155650:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bbe40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826022.365423:0:155650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6300 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155650:1874663042923264:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787826022.365441:0:155650:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6edc0 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155650 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826022.365451:0:155650:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6edc0 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155650 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826022.365457:0:155650:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6edc0 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155650 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826022.365461:0:155650:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6edc0 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155650 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826022.365467:0:155650:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6edc0 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155650 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826022.365471:0:155650:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0513400 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787826022.365474:0:155650:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6edc0 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155650 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826022.365479:0:155650:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6edc0 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155650 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826022.365483:0:155650:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6edc0 lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 155650 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826022.365488:0:155650:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787826022.365493:0:155650:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6edc0 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155650 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826022.365497:0:155650:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6edc0 lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155650 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826022.365507:0:155650:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c55a6300 x1874663042923264/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787826038 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787826022.365517:0:155650:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787826022.365538:0:155650:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787826022.365583:0:155650:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826022.365593:0:155650:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826022.365599:0:155650:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826022.365607:0:155650:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826022.365612:0:155650:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826022.365616:0:155650:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826022.365619:0:155650:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826022.365624:0:155650:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501cfbbe8 00010000:00010000:0.0:1787826022.365627:0:155650:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826022.365636:0:155650:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826022.365649:0:155650:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826022.365655:0:155650:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=16408 00000080:00200000:0.0:1787826022.365660:0:155650:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826022.365670:0:155650:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=16392 00000080:00200000:0.0:1787826022.365673:0:155650:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787826022.365684:0:155650:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787826022.365715:0:155650:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6edc7 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155650 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826022.365722:0:155650:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6edc7 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155650 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826022.365731:0:155650:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6edc7 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155650 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826022.365768:0:155650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6300 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155650:1874663042923392:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787826022.365832:0:155650:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bbe40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826022.375607:0:155650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6300 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155650:1874663042923392:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787826022.375629:0:155650:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6edc7 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155650 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826022.375642:0:155650:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6edc7 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155650 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826022.375650:0:155650:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6edc7 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155650 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826022.375657:0:155650:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6edc7 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155650 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826022.375667:0:155650:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6edc7 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155650 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826022.375673:0:155650:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0513a00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787826022.375677:0:155650:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6edc7 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155650 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826022.375685:0:155650:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6edc7 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155650 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826022.375692:0:155650:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6edc7 lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 155650 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826022.375699:0:155650:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787826022.375704:0:155650:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6edc7 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155650 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826022.375711:0:155650:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6edc7 lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155650 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826022.375726:0:155650:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c55a6300 x1874663042923392/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787826038 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787826022.377835:0:155650:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787826022.377868:0:155650:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787826022.377947:0:155650:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826022.377963:0:155650:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826022.377975:0:155650:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826022.377992:0:155650:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826022.377999:0:155650:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826022.378004:0:155650:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826022.378008:0:155650:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826022.378014:0:155650:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501cfbb58 00010000:00010000:0.0:1787826022.378028:0:155650:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826022.378037:0:155650:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826022.378052:0:155650:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826022.378060:0:155650:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=13 00000080:00200008:0.0:1787826022.378071:0:155650:0:(file.c:6296:ll_getattr_dentry()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1787826022.378075:0:155650:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08),name=d71a.replay-single 00000002:00010000:0.0:1787826022.378087:0:155650:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x20001a9e3:0x17e0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826022.378093:0:155650:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00010000:00010000:0.0:1787826022.378127:0:155650:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6edce lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 155650 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826022.378134:0:155650:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6edce lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 155650 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826022.378143:0:155650:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6edce lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 155650 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826022.378170:0:155650:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6300 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155650:1874663042923520:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787826022.378254:0:155650:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bbe40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826022.424612:0:155650:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c494d180 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155650:1874663042923776:192.168.203.147@tcp:36:lfs.0 00000080:00200000:1.0:1787826022.424728:0:155650:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x20001a9e3:0x1bbf:0x0](ffff9181c83e1988) 00000080:00200000:1.0:1787826022.424736:0:155650:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1bbf:0x0], updating i_size 4096 00000080:00200008:1.0:1787826022.424933:0:155650:0:(file.c:479:ll_file_release()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00010000:00010000:1.0:1787826022.424963:0:155650:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513000/0xbef42c0221d64bee lrc: 2/0,0 mode: CR/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce962d expref: -99 pid: 152210 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826022.438161:0:155667:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826022.438658:0:155667:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826022.438680:0:155667:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826022.438695:0:155667:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826022.438703:0:155667:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826022.438711:0:155667:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826022.438715:0:155667:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787826022.438722:0:155667:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501d8bc20 00010000:00010000:1.0:1787826022.438727:0:155667:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826022.438736:0:155667:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826022.438765:0:155667:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826022.438774:0:155667:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=24 00000080:00200000:1.0:1787826022.438781:0:155667:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826022.438793:0:155667:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=0 00000080:00200000:1.0:1787826022.438797:0:155667:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:1.0:1787826022.438810:0:155667:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787826022.438857:0:155667:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6edd5 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155667 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826022.438865:0:155667:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6edd5 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155667 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826022.438874:0:155667:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6edd5 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155667 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826022.438899:0:155667:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155667:1874663042923904:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:1.0:1787826022.441712:0:155667:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b590c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826022.460346:0:155667:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f3480 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155667:1874663042924032:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:1.0:1787826022.460371:0:155667:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7000/0xbef42c0221d6eddc lrc: 4/1,0 mode: --/CR res: [0x200026560:0x33:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cebd5c expref: -99 pid: 155667 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826022.460384:0:155667:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7000/0xbef42c0221d6eddc lrc: 4/1,0 mode: --/PR res: [0x200026560:0x33:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cebd5c expref: -99 pid: 155667 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826022.460417:0:155667:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7000/0xbef42c0221d6eddc lrc: 4/1,0 mode: PR/PR res: [0x200026560:0x33:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cebd5c expref: -99 pid: 155667 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826022.460428:0:155667:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7000/0xbef42c0221d6eddc lrc: 4/1,0 mode: PR/PR res: [0x200026560:0x33:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cebd5c expref: -99 pid: 155667 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826022.460439:0:155667:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7000/0xbef42c0221d6eddc lrc: 5/2,0 mode: PR/PR res: [0x200026560:0x33:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cebd5c expref: -99 pid: 155667 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826022.460446:0:155667:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7000/0xbef42c0221d6eddc lrc: 5/2,0 mode: PR/PR res: [0x200026560:0x33:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cebd5c expref: -99 pid: 155667 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826022.460483:0:155667:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7000/0xbef42c0221d6eddc lrc: 4/1,0 mode: PR/PR res: [0x200026560:0x33:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cebd5c expref: -99 pid: 155667 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826022.460495:0:155667:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c71f3480 x1874663042924032/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787826038 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826022.460507:0:155667:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7000/0xbef42c0221d6eddc lrc: 3/1,0 mode: PR/PR res: [0x200026560:0x33:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cebd5c expref: -99 pid: 155667 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826022.460519:0:155667:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590091616/51 (0 0) 00010000:00010000:1.0:1787826022.460526:0:155667:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7000/0xbef42c0221d6eddc lrc: 3/1,0 mode: PR/PR res: [0x200026560:0x33:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cebd5c expref: -99 pid: 155667 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826022.460535:0:155667:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7000/0xbef42c0221d6eddc lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x33:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cebd5c expref: -99 pid: 155667 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000002:00010000:1.0:1787826022.460566:0:155667:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001f58:0x33:0x0]) in obj [0x240001f58:0x33:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826022.460607:0:155667:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6ede3 lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x33:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 155667 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826022.460614:0:155667:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6ede3 lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x33:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155667 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826022.460623:0:155667:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6ede3 lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x33:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155667 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826022.460639:0:155667:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1880 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155667:1874663042924160:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:1.0:1787826022.460714:0:155667:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b59cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826022.486212:0:155667:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1880 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155667:1874663042924160:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:1.0:1787826022.486241:0:155667:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6ede3 lrc: 4/1,0 mode: --/CR res: [0x240001f58:0x33:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cebd6a expref: -99 pid: 155667 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826022.486253:0:155667:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6ede3 lrc: 4/1,0 mode: --/PR res: [0x240001f58:0x33:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cebd6a expref: -99 pid: 155667 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826022.486263:0:155667:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6ede3 lrc: 4/1,0 mode: PR/PR res: [0x240001f58:0x33:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cebd6a expref: -99 pid: 155667 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826022.486272:0:155667:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6ede3 lrc: 4/1,0 mode: PR/PR res: [0x240001f58:0x33:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cebd6a expref: -99 pid: 155667 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826022.486283:0:155667:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6ede3 lrc: 5/2,0 mode: PR/PR res: [0x240001f58:0x33:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cebd6a expref: -99 pid: 155667 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826022.486295:0:155667:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6ede3 lrc: 5/2,0 mode: PR/PR res: [0x240001f58:0x33:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cebd6a expref: -99 pid: 155667 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826022.486302:0:155667:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6ede3 lrc: 4/1,0 mode: PR/PR res: [0x240001f58:0x33:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cebd6a expref: -99 pid: 155667 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826022.486313:0:155667:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c35b1880 x1874663042924160/t0(0) o101->lustre-MDT0001-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787826038 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826022.486326:0:155667:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6ede3 lrc: 3/1,0 mode: PR/PR res: [0x240001f58:0x33:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cebd6a expref: -99 pid: 155667 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826022.486342:0:155667:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d0f54800 type 13 mode 4 res 9663684440/51 (0 0) 00010000:00010000:1.0:1787826022.486350:0:155667:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6ede3 lrc: 3/1,0 mode: PR/PR res: [0x240001f58:0x33:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cebd6a expref: -99 pid: 155667 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826022.486478:0:155667:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6ede3 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x33:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cebd6a expref: -99 pid: 155667 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826022.486515:0:155667:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1bbf:0x0], updating i_size 8192 00000080:00200000:1.0:1787826022.486519:0:155667:0:(namei.c:147:ll_iget()) got inode: [0x20001a9e3:0x1bbf:0x0](ffff9181c83e1988): rc = 0 00000080:00010000:1.0:1787826022.486528:0:155667:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x1bbf:0x0](ffff9181c83e1988) for lock 0xbef42c0221d6edd5 00000080:00010000:0.0:1787826022.486620:0:155667:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x1bbf:0x0](ffff9181c83e1988) 00000080:00010000:0.0:1787826022.486626:0:155667:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6edd5 from it ffffb97501d8be08 00010000:00010000:0.0:1787826022.486632:0:155667:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6edd5 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1bbf:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cebd4e expref: -99 pid: 155667 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826022.486643:0:155667:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6edd5 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1bbf:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cebd4e expref: -99 pid: 155667 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826022.486668:0:155667:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 3 00000080:00200000:0.0:1787826022.486677:0:155667:0:(namei.c:2328:ll_rmdir()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00010000:00010000:0.0:1787826022.486705:0:155667:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6edd5 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1bbf:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00cebd4e expref: -99 pid: 155667 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826022.486788:0:155667:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6edd5 lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x1bbf:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cebd4e expref: -99 pid: 155667 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826022.486796:0:155667:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826022.486799:0:155667:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6edd5 lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x1bbf:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cebd4e expref: -99 pid: 155667 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:0.0:1787826022.486830:0:155667:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787826022.486838:0:155667:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c55a5500 x1874663042924416/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826022.486866:0:155667:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155667:1874663042924416:192.168.203.147@tcp:36:rmdir.0 00000100:00100000:0.0:1787826022.486947:0:155667:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bb180 going to sleep for 11 seconds 00000100:00100000:1.0:1787826022.490537:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826022.490568:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049164544 00000100:00100000:1.0:1787826022.490576:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049164544 00000100:00100000:1.0:1787826022.490618:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9235 00000100:00100000:1.0:1787826022.490626:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b1880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187904:x1874663049164544:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826022.490643:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7000/0xbef42c0221d6eddc lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x33:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cebd5c expref: -99 pid: 155667 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826022.490710:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826022.490716:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b1880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187904:x1874663049164544:12345-192.168.203.147@tcp:104: Request processed in 91us (188us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826022.490723:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9235 00000100:00100000:0.0:1787826022.499994:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826022.501036:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049164800 00000100:00100000:1.0:1787826022.501049:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049164800 00000100:00100000:1.0:1787826022.501112:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9236 00000100:00100000:1.0:1787826022.501122:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a7b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663049164800:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826022.501138:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6ede3 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x33:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cebd6a expref: -99 pid: 155667 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826022.501558:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826022.501581:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a7b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663049164800:12345-192.168.203.147@tcp:104: Request processed in 446us (1575us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826022.501590:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9236 00000100:00100000:0.0:1787826022.522397:0:155667:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155667:1874663042924416:192.168.203.147@tcp:36:rmdir.0 00000080:00200000:0.0:1787826022.522440:0:155667:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1bbf:0x0](ffff9181c83e1988) 00000080:00200000:0.0:1787826022.538769:0:155686:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826022.538810:0:155686:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826022.538830:0:155686:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826022.538846:0:155686:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826022.538854:0:155686:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826022.538862:0:155686:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826022.538866:0:155686:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826022.538873:0:155686:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501dcbb58 00010000:00010000:0.0:1787826022.538877:0:155686:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826022.538886:0:155686:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826022.538899:0:155686:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826022.538906:0:155686:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=29 00000080:00200000:0.0:1787826022.538913:0:155686:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826022.538925:0:155686:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=13 00000080:00200000:0.0:1787826022.538928:0:155686:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787826022.538940:0:155686:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787826022.538982:0:155686:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6edea lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155686 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826022.538990:0:155686:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6edea lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155686 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826022.538999:0:155686:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6edea lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155686 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826022.539033:0:155686:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155686:1874663042924800:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787826022.539223:0:155686:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bbb40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826022.541963:0:155686:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155686:1874663042924800:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787826022.541985:0:155686:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6edea lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155686 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826022.541999:0:155686:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6edea lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155686 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826022.542007:0:155686:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6edea lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155686 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826022.542014:0:155686:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6edea lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155686 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826022.542034:0:155686:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6edea lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155686 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826022.542045:0:155686:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0512000 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787826022.542049:0:155686:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6edea lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155686 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826022.542056:0:155686:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6edea lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155686 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826022.542063:0:155686:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6edea lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 155686 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826022.542070:0:155686:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787826022.542076:0:155686:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6edea lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155686 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826022.542083:0:155686:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6edea lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155686 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826022.542101:0:155686:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c55a4380 x1874663042924800/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787826038 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787826022.542114:0:155686:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787826022.542139:0:155686:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787826022.542202:0:155686:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826022.542216:0:155686:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826022.542225:0:155686:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826022.542237:0:155686:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826022.542249:0:155686:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826022.542253:0:155686:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826022.542256:0:155686:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826022.542262:0:155686:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501dcbbe8 00010000:00010000:0.0:1787826022.542265:0:155686:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826022.542273:0:155686:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826022.542285:0:155686:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826022.542302:0:155686:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=16408 00000080:00200000:0.0:1787826022.542308:0:155686:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826022.544424:0:155686:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=16392 00000080:00200000:0.0:1787826022.544431:0:155686:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787826022.544451:0:155686:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787826022.544515:0:155686:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6edf1 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155686 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826022.544526:0:155686:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6edf1 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155686 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826022.544536:0:155686:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512c00/0xbef42c0221d6edf1 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155686 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826022.544560:0:155686:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155686:1874663042924928:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787826022.544802:0:155686:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bbb40 going to sleep for 11 seconds 00000002:00010000:0.0:1787826022.612276:0:155688:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001f58:0x34:0x0]) in obj [0x240001f58:0x34:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826022.612342:0:155688:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6ee0d lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x34:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 155688 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826022.612354:0:155688:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6ee0d lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x34:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155688 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826022.612364:0:155688:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6ee0d lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x34:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155688 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826022.612389:0:155688:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155688:1874663042925696:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787826022.612979:0:155688:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bb900 going to sleep for 11 seconds 00000100:00100000:0.0:1787826022.637491:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826022.637559:0:33333:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:0.0:1787826022.637582:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049166208 00000100:00100000:0.0:1787826022.637595:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049166208 00000100:00100000:0.0:1787826022.637668:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9237 00000100:00100000:0.0:1787826022.637688:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181e053c050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:188682:x1874663049166208:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826022.637710:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481000/0xbef42c0221d6ee06 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x34:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cebe20 expref: -99 pid: 155688 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826022.637967:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826022.637976:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181e053c050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:188682:x1874663049166208:12345-192.168.203.147@tcp:104: Request processed in 290us (495us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826022.638362:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9237 00000100:00100000:0.0:1787826022.642695:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826022.642740:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049166464 00000100:00100000:0.0:1787826022.642781:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049166464 00000100:00100000:0.0:1787826022.643145:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9238 00000100:00100000:0.0:1787826022.643159:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a5180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663049166464:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826022.643182:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6ee0d lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x34:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cebe2e expref: -99 pid: 155688 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826022.643276:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826022.643283:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a5180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663049166464:12345-192.168.203.147@tcp:104: Request processed in 127us (611us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826022.643291:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9238 00000080:00200000:1.0:1787826022.697883:0:155692:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826022.698201:0:155692:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826022.698227:0:155692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826022.698247:0:155692:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826022.698254:0:155692:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826022.698266:0:155692:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826022.698270:0:155692:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787826022.698279:0:155692:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501dfbb58 00010000:00010000:1.0:1787826022.698283:0:155692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826022.698292:0:155692:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826022.698308:0:155692:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826022.698317:0:155692:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=29 00000080:00200000:1.0:1787826022.698324:0:155692:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826022.698337:0:155692:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=13 00000080:00200000:1.0:1787826022.698342:0:155692:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:1.0:1787826022.698356:0:155692:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787826022.698413:0:155692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6ee14 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826022.698421:0:155692:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6ee14 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155692 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826022.698433:0:155692:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6ee14 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155692 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826022.698466:0:155692:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155692:1874663042926336:192.168.203.147@tcp:101:lfs.0 00000100:00100000:1.0:1787826022.699287:0:155692:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b59a80 going to sleep for 11 seconds 00000080:00200000:0.0:1787826022.796420:0:155696:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826022.796458:0:155696:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826022.796481:0:155696:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826022.796502:0:155696:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826022.796510:0:155696:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826022.796522:0:155696:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826022.796526:0:155696:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826022.796534:0:155696:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97500c87c20 00010000:00010000:0.0:1787826022.796539:0:155696:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826022.796550:0:155696:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826022.796566:0:155696:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826022.796574:0:155696:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=24 00000080:00200000:0.0:1787826022.796581:0:155696:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826022.796596:0:155696:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=0 00000080:00200000:0.0:1787826022.796601:0:155696:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787826022.796614:0:155696:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787826022.796660:0:155696:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6ee29 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155696 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826022.796668:0:155696:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6ee29 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155696 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826022.796678:0:155696:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6ee29 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155696 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826022.796705:0:155696:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155696:1874663042926976:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787826022.796845:0:155696:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d18bb240 going to sleep for 11 seconds 00000100:00100000:1.0:1787826022.808290:0:155696:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155696:1874663042926976:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:1.0:1787826022.808402:0:155696:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6ee29 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cebed6 expref: -99 pid: 155696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826022.808509:0:155696:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6ee29 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1bc1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cebed6 expref: -99 pid: 155696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826022.808520:0:155696:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6ee29 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1bc1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cebed6 expref: -99 pid: 155696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826022.808533:0:155696:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6ee29 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bc1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cebed6 expref: -99 pid: 155696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826022.808548:0:155696:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6ee29 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bc1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cebed6 expref: -99 pid: 155696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826022.808564:0:155696:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6ee29 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1bc1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cebed6 expref: -99 pid: 155696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826022.808574:0:155696:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6ee29 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1bc1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cebed6 expref: -99 pid: 155696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826022.808582:0:155696:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6ee29 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bc1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cebed6 expref: -99 pid: 155696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826022.808595:0:155696:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c55a7100 x1874663042926976/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/672 e 0 to 0 dl 1787826038 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826022.808611:0:155696:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6ee29 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1bc1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cebed6 expref: -99 pid: 155696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826022.808629:0:155696:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/7105 (0 0) 00000080:00200000:1.0:1787826022.808637:0:155696:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00000002:00010000:1.0:1787826022.810132:0:155696:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200026560:0x35:0x0]) in obj [0x200026560:0x35:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826022.810224:0:155696:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5600/0xbef42c0221d6ee30 lrc: 3/1,0 mode: --/CR res: [0x200026560:0x35:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 155696 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826022.810237:0:155696:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5600/0xbef42c0221d6ee30 lrc: 3/1,0 mode: --/CR res: [0x200026560:0x35:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155696 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826022.810248:0:155696:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5600/0xbef42c0221d6ee30 lrc: 3/1,0 mode: --/CR res: [0x200026560:0x35:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155696 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826022.810279:0:155696:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155696:1874663042927104:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:1.0:1787826022.810393:0:155696:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b59e40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826022.855386:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826022.855432:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049167872 00000100:00100000:1.0:1787826022.855446:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049167872 00000100:00100000:1.0:1787826022.855506:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9239 00000100:00100000:1.0:1787826022.855519:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b3480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187904:x1874663049167872:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826022.855541:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5600/0xbef42c0221d6ee30 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x35:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cebee4 expref: -99 pid: 155696 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826022.855641:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826022.855649:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b3480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187904:x1874663049167872:12345-192.168.203.147@tcp:104: Request processed in 133us (278us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826022.855658:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9239 00000100:00100000:1.0:1787826022.862928:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826022.863685:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049168128 00000100:00100000:0.0:1787826022.863696:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049168128 00000100:00100000:0.0:1787826022.863788:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9240 00000100:00100000:0.0:1787826022.863800:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8414:x1874663049168128:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826022.863831:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f7400/0xbef42c0221d6ee37 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x35:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cebef2 expref: -99 pid: 155696 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826022.863929:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826022.863936:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8414:x1874663049168128:12345-192.168.203.147@tcp:104: Request processed in 139us (1031us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826022.863944:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9240 00000080:00200000:1.0:1787826022.936672:0:155698:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826022.936716:0:155698:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826022.936741:0:155698:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826022.936788:0:155698:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826022.936797:0:155698:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826022.936808:0:155698:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826022.936812:0:155698:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787826022.936822:0:155698:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97500c97b58 00010000:00010000:1.0:1787826022.936827:0:155698:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826022.936837:0:155698:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826022.936852:0:155698:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826022.936860:0:155698:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=29 00000080:00200000:1.0:1787826022.936867:0:155698:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826022.936879:0:155698:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=13 00000080:00200000:1.0:1787826022.936883:0:155698:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:1.0:1787826022.936898:0:155698:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787826022.936960:0:155698:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6ee3e lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155698 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826022.936970:0:155698:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6ee3e lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155698 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826022.936982:0:155698:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6ee3e lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155698 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826022.939076:0:155698:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155698:1874663042927872:192.168.203.147@tcp:101:lfs.0 00000100:00100000:1.0:1787826022.940948:0:155698:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b59840 going to sleep for 11 seconds 00000080:00200000:1.0:1787826023.054395:0:155703:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826023.054440:0:155703:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826023.054466:0:155703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826023.054486:0:155703:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826023.054495:0:155703:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826023.054505:0:155703:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826023.054509:0:155703:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787826023.054520:0:155703:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501c5bc20 00010000:00010000:1.0:1787826023.054524:0:155703:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826023.054540:0:155703:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826023.054557:0:155703:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826023.054567:0:155703:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=24 00000080:00200000:1.0:1787826023.054575:0:155703:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826023.054591:0:155703:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=0 00000080:00200000:1.0:1787826023.054597:0:155703:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:1.0:1787826023.054614:0:155703:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787826023.055074:0:155703:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6ee53 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.055085:0:155703:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6ee53 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155703 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.055099:0:155703:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6ee53 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155703 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826023.055135:0:155703:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155703:1874663042928512:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:1.0:1787826023.055307:0:155703:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b59e40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826023.100624:0:155703:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5cb6cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826023.105653:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826023.105700:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049169536 00000100:00100000:1.0:1787826023.105709:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049169536 00000100:00100000:1.0:1787826023.105813:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9241 00000100:00100000:1.0:1787826023.105824:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187904:x1874663049169536:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826023.105842:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7000/0xbef42c0221d6ee5a lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x36:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cebfa8 expref: -99 pid: 155703 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826023.105919:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826023.105927:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187904:x1874663049169536:12345-192.168.203.147@tcp:104: Request processed in 105us (282us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826023.105937:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9241 00010000:00010000:0.0:1787826023.106933:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826023.106937:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826023.106940:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826023.116090:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826023.116134:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049169792 00000100:00100000:1.0:1787826023.116143:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049169792 00000100:00100000:1.0:1787826023.116436:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9242 00000100:00100000:1.0:1787826023.116447:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b1c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663049169792:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826023.116468:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6ee61 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x36:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cebfb6 expref: -99 pid: 155703 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826023.116546:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826023.116553:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b1c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663049169792:12345-192.168.203.147@tcp:104: Request processed in 108us (480us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826023.116566:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9242 00000100:00100000:1.0:1787826023.160976:0:155703:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155703:1874663042929024:192.168.203.147@tcp:36:rmdir.0 00000080:00200000:1.0:1787826023.161040:0:155703:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1bc2:0x0](ffff9181c83f9148) 00000100:00100000:3.0:1787826023.223604:0:155710:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5cb6480 going to sleep for 11 seconds 00000100:00100000:3.0:1787826023.258118:0:155710:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155710:1874663042929664:192.168.203.147@tcp:101:lfs.0 00010000:00010000:3.0:1787826023.258142:0:155710:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6ee76 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec018 expref: -99 pid: 155710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.258158:0:155710:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6ee76 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec018 expref: -99 pid: 155710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.258166:0:155710:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6ee76 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec018 expref: -99 pid: 155710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.258176:0:155710:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6ee76 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec018 expref: -99 pid: 155710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.258186:0:155710:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6ee76 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec018 expref: -99 pid: 155710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.258197:0:155710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6ee76 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec018 expref: -99 pid: 155710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.258205:0:155710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6ee76 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec018 expref: -99 pid: 155710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.258212:0:155710:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6ee76 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec018 expref: -99 pid: 155710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826023.258222:0:155710:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c5634380 x1874663042929664/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826039 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826023.258235:0:155710:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6ee76 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec018 expref: -99 pid: 155710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.258245:0:155710:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/6112 (0 0) 00000080:00200000:3.0:1787826023.258251:0:155710:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:3.0:1787826023.258276:0:155710:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x17e0:0x0], updating i_size 4096 00000080:00010000:3.0:1787826023.258281:0:155710:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) for lock 0xbef42c0221d6ee76 00000080:00010000:3.0:1787826023.258287:0:155710:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00000080:00010000:3.0:1787826023.258291:0:155710:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6ee76 from it ffffb97501e03cb8 00010000:00010000:3.0:1787826023.258294:0:155710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6ee76 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec018 expref: -99 pid: 155710 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.258302:0:155710:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6ee76 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec018 expref: -99 pid: 155710 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:3.0:1787826023.258900:0:155710:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), request_mask 0, flags 0, foreign 0 00000080:00200000:3.0:1787826023.258908:0:155710:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:3.0:1787826023.258924:0:155710:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826023.258936:0:155710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826023.258947:0:155710:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826023.258954:0:155710:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826023.258959:0:155710:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:3.0:1787826023.258962:0:155710:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:3.0:1787826023.258966:0:155710:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501e03cb8 00010000:00010000:3.0:1787826023.258969:0:155710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826023.258977:0:155710:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826023.259095:0:155710:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:3.0:1787826023.259104:0:155710:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826023.259112:0:155710:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826023.259120:0:155710:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826023.259126:0:155710:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826023.259130:0:155710:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:3.0:1787826023.259132:0:155710:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:3.0:1787826023.259136:0:155710:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501e03bf8 00010000:00010000:3.0:1787826023.259139:0:155710:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826023.259146:0:155710:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826023.259154:0:155710:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826023.259160:0:155710:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=267 00000080:00200000:3.0:1787826023.259165:0:155710:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 44 00000080:00200008:3.0:1787826023.259171:0:155710:0:(file.c:1090:ll_file_open()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00000080:00200080:3.0:1787826023.259190:0:155710:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) cmd=c03066f0 arg=7ffdd4dabcc0 00000080:00200000:3.0:1787826023.259203:0:155710:0:(dir.c:451:ll_dir_setdirstripe()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) name=test stripe_offset=0 stripe_count=2, hash_type=0 00010000:00010000:3.0:1787826023.259226:0:155710:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6ee76 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00cec018 expref: -99 pid: 155710 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826023.259240:0:155710:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6800/0xbef42c0221d64b3f lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9634 expref: -99 pid: 152202 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826023.259277:0:155710:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6ee76 lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cec018 expref: -99 pid: 155710 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826023.259284:0:155710:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826023.259287:0:155710:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6ee76 lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cec018 expref: -99 pid: 155710 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:3.0:1787826023.259305:0:155710:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:3.0:1787826023.259313:0:155710:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c5634380 x1874663042929920/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 552/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826023.259336:0:155710:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155710:1874663042929920:192.168.203.147@tcp:36:lfs.0 00000100:00100000:3.0:1787826023.259409:0:155710:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5cb6480 going to sleep for 11 seconds 00010000:00010000:3.0:1787826023.265877:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826023.292128:0:155710:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9181c4d2b000: committing for last_committed 335007449194 gen 11 00000100:00100000:3.0:1787826023.292253:0:155710:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155710:1874663042929920:192.168.203.147@tcp:36:lfs.0 00000080:00200000:3.0:1787826023.292302:0:155710:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x20001a9e3:0x1bc3:0x0](ffff9181c83fe3c8) 00000080:00200000:3.0:1787826023.292309:0:155710:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1bc3:0x0], updating i_size 4096 00000080:00200008:3.0:1787826023.292340:0:155710:0:(file.c:479:ll_file_release()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00010000:00010000:3.0:1787826023.292368:0:155710:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513000/0xbef42c0221d64bee lrc: 2/0,0 mode: CR/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce962d expref: -99 pid: 152210 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826023.311679:0:155721:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826023.311707:0:155721:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826023.311723:0:155721:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826023.311740:0:155721:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826023.311776:0:155721:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826023.311784:0:155721:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826023.311788:0:155721:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787826023.311795:0:155721:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97500e2bc20 00010000:00010000:1.0:1787826023.311800:0:155721:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826023.311817:0:155721:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826023.311831:0:155721:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826023.311838:0:155721:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=24 00000080:00200000:1.0:1787826023.311843:0:155721:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826023.311854:0:155721:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=0 00000080:00200000:1.0:1787826023.311858:0:155721:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:1.0:1787826023.311871:0:155721:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787826023.311913:0:155721:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6ee7d lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155721 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.311922:0:155721:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6ee7d lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155721 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.311931:0:155721:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6ee7d lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155721 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826023.311957:0:155721:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155721:1874663042930048:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:1.0:1787826023.316470:0:155721:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b59c00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826023.323177:0:155721:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155721:1874663042930048:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:1.0:1787826023.323205:0:155721:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6ee7d lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec05e expref: -99 pid: 155721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.323226:0:155721:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6ee7d lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1bc3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec05e expref: -99 pid: 155721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.323240:0:155721:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6ee7d lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1bc3:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec05e expref: -99 pid: 155721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.323250:0:155721:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6ee7d lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bc3:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec05e expref: -99 pid: 155721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.323260:0:155721:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6ee7d lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bc3:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec05e expref: -99 pid: 155721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.323273:0:155721:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6ee7d lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1bc3:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec05e expref: -99 pid: 155721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.323281:0:155721:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6ee7d lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1bc3:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec05e expref: -99 pid: 155721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.323289:0:155721:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6ee7d lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bc3:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec05e expref: -99 pid: 155721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826023.323300:0:155721:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c35b2680 x1874663042930048/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/672 e 0 to 0 dl 1787826039 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826023.323313:0:155721:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6ee7d lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1bc3:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec05e expref: -99 pid: 155721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.323322:0:155721:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/7107 (0 0) 00000080:00200000:1.0:1787826023.323329:0:155721:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00000002:00010000:1.0:1787826023.323378:0:155721:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200026560:0x37:0x0]) in obj [0x200026560:0x37:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826023.323418:0:155721:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6ee84 lrc: 3/1,0 mode: --/CR res: [0x200026560:0x37:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 155721 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.323425:0:155721:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6ee84 lrc: 3/1,0 mode: --/CR res: [0x200026560:0x37:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155721 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.323433:0:155721:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6ee84 lrc: 3/1,0 mode: --/CR res: [0x200026560:0x37:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155721 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826023.323450:0:155721:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155721:1874663042930176:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:1.0:1787826023.323523:0:155721:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b59c00 going to sleep for 11 seconds 00000100:00100000:0.0:1787826023.328852:0:155721:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155721:1874663042930176:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:0.0:1787826023.329011:0:155721:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6ee84 lrc: 4/1,0 mode: --/CR res: [0x200026560:0x37:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec06c expref: -99 pid: 155721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826023.329033:0:155721:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6ee84 lrc: 4/1,0 mode: --/PR res: [0x200026560:0x37:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec06c expref: -99 pid: 155721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826023.329045:0:155721:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6ee84 lrc: 4/1,0 mode: PR/PR res: [0x200026560:0x37:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec06c expref: -99 pid: 155721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826023.329057:0:155721:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6ee84 lrc: 4/1,0 mode: PR/PR res: [0x200026560:0x37:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec06c expref: -99 pid: 155721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826023.329069:0:155721:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6ee84 lrc: 5/2,0 mode: PR/PR res: [0x200026560:0x37:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec06c expref: -99 pid: 155721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826023.329076:0:155721:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6ee84 lrc: 5/2,0 mode: PR/PR res: [0x200026560:0x37:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec06c expref: -99 pid: 155721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826023.329084:0:155721:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6ee84 lrc: 4/1,0 mode: PR/PR res: [0x200026560:0x37:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec06c expref: -99 pid: 155721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826023.329096:0:155721:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c35b1f80 x1874663042930176/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787826039 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787826023.329110:0:155721:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6ee84 lrc: 3/1,0 mode: PR/PR res: [0x200026560:0x37:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec06c expref: -99 pid: 155721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826023.329129:0:155721:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590091616/55 (0 0) 00010000:00010000:0.0:1787826023.329137:0:155721:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6ee84 lrc: 3/1,0 mode: PR/PR res: [0x200026560:0x37:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec06c expref: -99 pid: 155721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826023.329145:0:155721:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6ee84 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x37:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec06c expref: -99 pid: 155721 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000002:00010000:0.0:1787826023.329174:0:155721:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001f58:0x37:0x0]) in obj [0x240001f58:0x37:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826023.329222:0:155721:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6ee8b lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x37:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 155721 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826023.329229:0:155721:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6ee8b lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x37:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155721 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826023.329237:0:155721:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6ee8b lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x37:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155721 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826023.329258:0:155721:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155721:1874663042930304:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787826023.329332:0:155721:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c55dab40 going to sleep for 11 seconds 00000001:02000400:3.0:1787826023.364167:0:155730:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec 00000100:00100000:1.0:1787826023.366836:0:155721:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155721:1874663042930304:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:1.0:1787826023.366864:0:155721:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6ee8b lrc: 4/1,0 mode: --/CR res: [0x240001f58:0x37:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec07a expref: -99 pid: 155721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.366965:0:155721:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6ee8b lrc: 4/1,0 mode: --/PR res: [0x240001f58:0x37:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec07a expref: -99 pid: 155721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.366979:0:155721:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6ee8b lrc: 4/1,0 mode: PR/PR res: [0x240001f58:0x37:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec07a expref: -99 pid: 155721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.366990:0:155721:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6ee8b lrc: 4/1,0 mode: PR/PR res: [0x240001f58:0x37:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec07a expref: -99 pid: 155721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.367004:0:155721:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6ee8b lrc: 5/2,0 mode: PR/PR res: [0x240001f58:0x37:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec07a expref: -99 pid: 155721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.367056:0:155721:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6ee8b lrc: 5/2,0 mode: PR/PR res: [0x240001f58:0x37:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec07a expref: -99 pid: 155721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.367141:0:155721:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6ee8b lrc: 4/1,0 mode: PR/PR res: [0x240001f58:0x37:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec07a expref: -99 pid: 155721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826023.367154:0:155721:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c55a7b80 x1874663042930304/t0(0) o101->lustre-MDT0001-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787826039 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826023.367329:0:155721:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6ee8b lrc: 3/1,0 mode: PR/PR res: [0x240001f58:0x37:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec07a expref: -99 pid: 155721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.367344:0:155721:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d0f54800 type 13 mode 4 res 9663684440/55 (0 0) 00010000:00010000:1.0:1787826023.367353:0:155721:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6ee8b lrc: 3/1,0 mode: PR/PR res: [0x240001f58:0x37:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec07a expref: -99 pid: 155721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.367362:0:155721:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6ee8b lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x37:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec07a expref: -99 pid: 155721 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826023.367396:0:155721:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1bc3:0x0], updating i_size 8192 00000080:00200000:1.0:1787826023.367401:0:155721:0:(namei.c:147:ll_iget()) got inode: [0x20001a9e3:0x1bc3:0x0](ffff9181c83fe3c8): rc = 0 00000080:00010000:1.0:1787826023.367410:0:155721:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x1bc3:0x0](ffff9181c83fe3c8) for lock 0xbef42c0221d6ee7d 00000080:00010000:1.0:1787826023.367428:0:155721:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x1bc3:0x0](ffff9181c83fe3c8) 00000080:00010000:1.0:1787826023.367432:0:155721:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6ee7d from it ffffb97500e2be08 00010000:00010000:1.0:1787826023.367436:0:155721:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6ee7d lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1bc3:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec05e expref: -99 pid: 155721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.367445:0:155721:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6ee7d lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1bc3:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec05e expref: -99 pid: 155721 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826023.367470:0:155721:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 3 00000080:00200000:1.0:1787826023.367478:0:155721:0:(namei.c:2328:ll_rmdir()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00010000:00010000:1.0:1787826023.367507:0:155721:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6ee7d lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1bc3:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00cec05e expref: -99 pid: 155721 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826023.367551:0:155721:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6ee7d lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x1bc3:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cec05e expref: -99 pid: 155721 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826023.367558:0:155721:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826023.367561:0:155721:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6ee7d lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x1bc3:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cec05e expref: -99 pid: 155721 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:1.0:1787826023.367586:0:155721:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787826023.368030:0:155721:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c35b2680 x1874663042930560/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826023.373002:0:155721:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155721:1874663042930560:192.168.203.147@tcp:36:rmdir.0 00000100:00100000:0.0:1787826023.373110:0:155721:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c55da0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826023.382042:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826023.382116:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049182464 00000100:00100000:1.0:1787826023.382127:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049182464 00000100:00100000:1.0:1787826023.382182:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9243 00000100:00100000:1.0:1787826023.382192:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a6300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187904:x1874663049182464:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826023.382208:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6ee84 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x37:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cec06c expref: -99 pid: 155721 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826023.382285:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826023.382292:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a6300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187904:x1874663049182464:12345-192.168.203.147@tcp:104: Request processed in 102us (264us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826023.382298:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9243 00010000:00010000:3.0:1787826023.382448:0:33350:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826023.382456:0:33350:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6ee84 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x37:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cec06c expref: -99 pid: 155721 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826023.382465:0:33350:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4400 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:3.0:1787826023.382469:0:33350:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6ee84 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x37:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00cec06c expref: -99 pid: 155721 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826023.382477:0:33350:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6ee84 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x37:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00cec06c expref: -99 pid: 155721 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826023.382485:0:33350:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6ee84 lrc: 3/0,0 mode: PR/PR res: [0x200026560:0x37:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x408400000000 nid: local remote: 0x878a5eea00cec06c expref: -99 pid: 155721 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826023.382592:0:33350:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6ee84 lrc: 2/0,0 mode: --/PR res: [0x200026560:0x37:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cec06c expref: -99 pid: 155721 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826023.382600:0:33350:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826023.382628:0:33350:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6ee84 lrc: 1/0,0 mode: --/PR res: [0x200026560:0x37:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cec06c expref: -99 pid: 155721 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826023.382635:0:33350:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6ee84 lrc: 0/0,0 mode: --/PR res: [0x200026560:0x37:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cec06c expref: -99 pid: 155721 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826023.382653:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826023.382655:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826023.382657:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826023.382671:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2392:1874663042930688:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1787826023.402059:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2392:1874663042930688:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787826023.408288:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826023.408331:0:15151:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:1.0:1787826023.408348:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049184512 00000100:00100000:1.0:1787826023.408357:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049184512 00000100:00100000:1.0:1787826023.408405:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9244 00000100:00100000:1.0:1787826023.408416:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c343e050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663049184512:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826023.408435:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6ee8b lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x37:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cec07a expref: -99 pid: 155721 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826023.408506:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826023.408512:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c343e050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663049184512:12345-192.168.203.147@tcp:104: Request processed in 99us (236us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826023.408520:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9244 00000100:00100000:3.0:1787826023.445080:0:155721:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155721:1874663042930560:192.168.203.147@tcp:36:rmdir.0 00000080:00200000:3.0:1787826023.445124:0:155721:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1bc3:0x0](ffff9181c83fe3c8) 00000080:00200000:1.0:1787826023.603236:0:155743:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826023.603265:0:155743:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826023.603283:0:155743:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826023.603298:0:155743:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826023.603306:0:155743:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826023.603313:0:155743:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826023.603317:0:155743:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787826023.603324:0:155743:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb975019cbc20 00010000:00010000:1.0:1787826023.603328:0:155743:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826023.603343:0:155743:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826023.603357:0:155743:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826023.603365:0:155743:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=24 00000080:00200000:1.0:1787826023.603370:0:155743:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826023.603382:0:155743:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=0 00000080:00200000:1.0:1787826023.603386:0:155743:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:1.0:1787826023.603399:0:155743:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787826023.603441:0:155743:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6eea7 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155743 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.603451:0:155743:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6eea7 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155743 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.603460:0:155743:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4000/0xbef42c0221d6eea7 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155743 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826023.603485:0:155743:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155743:1874663042931584:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:1.0:1787826023.603569:0:155743:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b59480 going to sleep for 11 seconds 00000100:00100000:1.0:1787826023.684393:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826023.684837:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049189504 00000100:00100000:1.0:1787826023.684846:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049189504 00000100:00100000:1.0:1787826023.684911:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9245 00000100:00100000:1.0:1787826023.684922:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:188992:x1874663049189504:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826023.684939:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7c00/0xbef42c0221d6eeae lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x38:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cec130 expref: -99 pid: 155743 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826023.685035:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826023.685043:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:188992:x1874663049189504:12345-192.168.203.147@tcp:104: Request processed in 122us (659us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826023.685050:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9245 00000100:00100000:3.0:1787826023.692595:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2392:1874663042932224:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787826023.707772:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:3.0:1787826023.707916:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2392:1874663042932224:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1787826023.713704:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049189760 00000100:00100000:0.0:1787826023.713732:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049189760 00000100:00100000:0.0:1787826023.713839:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9246 00000100:00100000:0.0:1787826023.713881:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663049189760:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826023.713917:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f7e00/0xbef42c0221d6eeb5 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x38:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cec13e expref: -99 pid: 155743 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826023.714106:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826023.714115:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663049189760:12345-192.168.203.147@tcp:104: Request processed in 252us (6352us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826023.714173:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9246 00010000:00010000:3.0:1787826023.714394:0:15160:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826023.714402:0:15160:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f7e00/0xbef42c0221d6eeb5 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x38:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cec13e expref: -99 pid: 155743 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826023.714412:0:15160:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e06f7e00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:3.0:1787826023.714416:0:15160:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f7e00/0xbef42c0221d6eeb5 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x38:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00cec13e expref: -99 pid: 155743 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826023.714424:0:15160:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f7e00/0xbef42c0221d6eeb5 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x38:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00cec13e expref: -99 pid: 155743 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826023.714433:0:15160:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f7e00/0xbef42c0221d6eeb5 lrc: 3/0,0 mode: PR/PR res: [0x240001f58:0x38:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x408400000000 nid: local remote: 0x878a5eea00cec13e expref: -99 pid: 155743 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826023.714489:0:15160:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f7e00/0xbef42c0221d6eeb5 lrc: 2/0,0 mode: --/PR res: [0x240001f58:0x38:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cec13e expref: -99 pid: 155743 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826023.714496:0:15160:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826023.714539:0:15160:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f7e00/0xbef42c0221d6eeb5 lrc: 1/0,0 mode: --/PR res: [0x240001f58:0x38:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cec13e expref: -99 pid: 155743 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826023.714547:0:15160:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f7e00/0xbef42c0221d6eeb5 lrc: 0/0,0 mode: --/PR res: [0x240001f58:0x38:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cec13e expref: -99 pid: 155743 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826023.714570:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826023.714572:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826023.714573:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826023.714582:0:2392:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787826023.714596:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2392:1874663042932352:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1787826023.731246:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2392:1874663042932352:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1787826023.786260:0:155746:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f1880 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155746:1874663042932480:192.168.203.147@tcp:101:lfs.0 00010000:00010000:3.0:1787826023.786292:0:155746:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7c00/0xbef42c0221d6eebc lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155746 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.786305:0:155746:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7c00/0xbef42c0221d6eebc lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155746 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.786312:0:155746:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7c00/0xbef42c0221d6eebc lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155746 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.786319:0:155746:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7c00/0xbef42c0221d6eebc lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155746 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.786328:0:155746:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7c00/0xbef42c0221d6eebc lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155746 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.786334:0:155746:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e06f7c00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:3.0:1787826023.786339:0:155746:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7c00/0xbef42c0221d6eebc lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155746 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.786347:0:155746:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7c00/0xbef42c0221d6eebc lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155746 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.786353:0:155746:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7c00/0xbef42c0221d6eebc lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 155746 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.786360:0:155746:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826023.786366:0:155746:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7c00/0xbef42c0221d6eebc lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155746 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.786372:0:155746:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7c00/0xbef42c0221d6eebc lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155746 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826023.786390:0:155746:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c71f1880 x1874663042932480/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787826039 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:3.0:1787826023.786404:0:155746:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:3.0:1787826023.786429:0:155746:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:3.0:1787826023.786497:0:155746:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:3.0:1787826023.786510:0:155746:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826023.786519:0:155746:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826023.786530:0:155746:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826023.786537:0:155746:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826023.786542:0:155746:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:3.0:1787826023.786547:0:155746:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:3.0:1787826023.786552:0:155746:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501cabbe8 00010000:00010000:3.0:1787826023.786555:0:155746:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826023.786563:0:155746:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826023.786575:0:155746:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826023.786582:0:155746:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=16408 00000080:00200000:3.0:1787826023.786586:0:155746:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826023.786597:0:155746:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=16392 00000080:00200000:3.0:1787826023.786601:0:155746:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:3.0:1787826023.786612:0:155746:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:3.0:1787826023.786659:0:155746:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6eec3 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155746 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.786666:0:155746:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6eec3 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155746 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.786675:0:155746:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6eec3 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155746 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826023.786693:0:155746:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155746:1874663042932608:192.168.203.147@tcp:101:lfs.0 00000100:00100000:3.0:1787826023.786810:0:155746:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5cb6480 going to sleep for 11 seconds 00000100:00100000:3.0:1787826023.790224:0:155746:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155746:1874663042932608:192.168.203.147@tcp:101:lfs.0 00010000:00010000:3.0:1787826023.790245:0:155746:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6eec3 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155746 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.790263:0:155746:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6eec3 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155746 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.790270:0:155746:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6eec3 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155746 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.790278:0:155746:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6eec3 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155746 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.790286:0:155746:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6eec3 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155746 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.790292:0:155746:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3481a00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:3.0:1787826023.790296:0:155746:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6eec3 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155746 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.790302:0:155746:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6eec3 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155746 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.790310:0:155746:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6eec3 lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 155746 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.790317:0:155746:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826023.790323:0:155746:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6eec3 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155746 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.790329:0:155746:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6eec3 lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155746 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826023.790349:0:155746:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c5634380 x1874663042932608/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787826039 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:3.0:1787826023.790365:0:155746:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:3.0:1787826023.790391:0:155746:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:3.0:1787826023.790461:0:155746:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:3.0:1787826023.790476:0:155746:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826023.790485:0:155746:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826023.790495:0:155746:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826023.790502:0:155746:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826023.790506:0:155746:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:3.0:1787826023.790509:0:155746:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:3.0:1787826023.790514:0:155746:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501cabb58 00010000:00010000:3.0:1787826023.790517:0:155746:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826023.790524:0:155746:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826023.790536:0:155746:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826023.790542:0:155746:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=13 00000080:00200008:3.0:1787826023.790550:0:155746:0:(file.c:6296:ll_getattr_dentry()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), request_mask 0, flags 0, foreign 0 00000080:00200000:3.0:1787826023.790555:0:155746:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08),name=d71a.replay-single 00000002:00010000:3.0:1787826023.790566:0:155746:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x20001a9e3:0x17e0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826023.790570:0:155746:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00010000:00010000:3.0:1787826023.790603:0:155746:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6eeca lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 155746 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.790610:0:155746:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6eeca lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 155746 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.790619:0:155746:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6eeca lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 155746 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826023.790647:0:155746:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155746:1874663042932736:192.168.203.147@tcp:101:lfs.0 00000100:00100000:3.0:1787826023.790734:0:155746:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5cb6480 going to sleep for 11 seconds 00000100:00100000:3.0:1787826023.795439:0:155746:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155746:1874663042932736:192.168.203.147@tcp:101:lfs.0 00010000:00010000:3.0:1787826023.795467:0:155746:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6eeca lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec1a0 expref: -99 pid: 155746 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.795483:0:155746:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6eeca lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec1a0 expref: -99 pid: 155746 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.795490:0:155746:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6eeca lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec1a0 expref: -99 pid: 155746 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.795500:0:155746:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6eeca lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec1a0 expref: -99 pid: 155746 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.795511:0:155746:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6eeca lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec1a0 expref: -99 pid: 155746 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.795522:0:155746:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6eeca lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec1a0 expref: -99 pid: 155746 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.795535:0:155746:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6eeca lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec1a0 expref: -99 pid: 155746 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.795542:0:155746:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6eeca lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec1a0 expref: -99 pid: 155746 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826023.795553:0:155746:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c5634380 x1874663042932736/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826039 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826023.795566:0:155746:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6eeca lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec1a0 expref: -99 pid: 155746 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.795576:0:155746:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/6112 (0 0) 00000080:00200000:3.0:1787826023.795583:0:155746:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:3.0:1787826023.795609:0:155746:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x17e0:0x0], updating i_size 4096 00000080:00010000:3.0:1787826023.795615:0:155746:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) for lock 0xbef42c0221d6eeca 00000080:00010000:3.0:1787826023.795623:0:155746:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00000080:00010000:3.0:1787826023.795626:0:155746:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6eeca from it ffffb97501cabcb8 00010000:00010000:3.0:1787826023.795629:0:155746:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6eeca lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec1a0 expref: -99 pid: 155746 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.795920:0:155746:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6eeca lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec1a0 expref: -99 pid: 155746 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:3.0:1787826023.799964:0:155746:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), request_mask 0, flags 0, foreign 0 00000080:00200000:3.0:1787826023.799975:0:155746:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:3.0:1787826023.799998:0:155746:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826023.800025:0:155746:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826023.800041:0:155746:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826023.800050:0:155746:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826023.800057:0:155746:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:3.0:1787826023.800061:0:155746:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:3.0:1787826023.800069:0:155746:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501cabcb8 00010000:00010000:3.0:1787826023.800073:0:155746:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826023.800082:0:155746:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826023.800940:0:155746:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:3.0:1787826023.800955:0:155746:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826023.800966:0:155746:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826023.800978:0:155746:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826023.800985:0:155746:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826023.800989:0:155746:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:3.0:1787826023.800992:0:155746:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:3.0:1787826023.800996:0:155746:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501cabbf8 00010000:00010000:3.0:1787826023.800999:0:155746:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826023.801006:0:155746:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826023.801026:0:155746:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826023.801033:0:155746:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=267 00000080:00200000:3.0:1787826023.801040:0:155746:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 44 00000080:00200008:3.0:1787826023.801047:0:155746:0:(file.c:1090:ll_file_open()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00000080:00200080:3.0:1787826023.801070:0:155746:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) cmd=c03066f0 arg=7ffc2fcd9680 00000080:00200000:3.0:1787826023.801084:0:155746:0:(dir.c:451:ll_dir_setdirstripe()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) name=test stripe_offset=0 stripe_count=2, hash_type=0 00010000:00010000:3.0:1787826023.801718:0:155746:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6eeca lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00cec1a0 expref: -99 pid: 155746 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826023.801736:0:155746:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6800/0xbef42c0221d64b3f lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9634 expref: -99 pid: 152202 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826023.801888:0:155746:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6eeca lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cec1a0 expref: -99 pid: 155746 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826023.801895:0:155746:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826023.801898:0:155746:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6eeca lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cec1a0 expref: -99 pid: 155746 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:3.0:1787826023.801918:0:155746:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:3.0:1787826023.801928:0:155746:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c5634380 x1874663042932992/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 552/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826023.801954:0:155746:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155746:1874663042932992:192.168.203.147@tcp:36:lfs.0 00000100:00100000:3.0:1787826023.802045:0:155746:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5cb6480 going to sleep for 11 seconds 00000100:00100000:3.0:1787826023.810962:0:155746:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155746:1874663042932992:192.168.203.147@tcp:36:lfs.0 00000080:00200000:0.0:1787826023.841580:0:155747:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826023.842546:0:155747:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826023.842572:0:155747:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826023.842589:0:155747:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826023.842597:0:155747:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826023.842609:0:155747:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826023.842614:0:155747:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826023.843162:0:155747:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb975019c3c20 00010000:00010000:0.0:1787826023.843168:0:155747:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826023.843179:0:155747:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826023.843197:0:155747:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826023.843209:0:155747:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=24 00000080:00200000:0.0:1787826023.843219:0:155747:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826023.843247:0:155747:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=0 00000080:00200000:0.0:1787826023.843252:0:155747:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787826023.843281:0:155747:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787826023.843548:0:155747:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6eed1 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155747 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826023.843557:0:155747:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6eed1 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155747 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826023.843567:0:155747:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6eed1 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155747 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826023.843593:0:155747:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155747:1874663042933120:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787826023.844342:0:155747:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c55daa80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826023.861962:0:155747:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155747:1874663042933120:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:0.0:1787826023.861988:0:155747:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6eed1 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec1e6 expref: -99 pid: 155747 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826023.862030:0:155747:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6eed1 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1bc5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec1e6 expref: -99 pid: 155747 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826023.862039:0:155747:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6eed1 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1bc5:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec1e6 expref: -99 pid: 155747 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826023.862049:0:155747:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6eed1 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bc5:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec1e6 expref: -99 pid: 155747 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826023.862068:0:155747:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6eed1 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bc5:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec1e6 expref: -99 pid: 155747 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826023.862082:0:155747:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6eed1 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1bc5:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec1e6 expref: -99 pid: 155747 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826023.862090:0:155747:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6eed1 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1bc5:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec1e6 expref: -99 pid: 155747 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826023.862098:0:155747:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6eed1 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bc5:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec1e6 expref: -99 pid: 155747 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826023.862111:0:155747:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c55a6d80 x1874663042933120/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/672 e 0 to 0 dl 1787826039 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787826023.862124:0:155747:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6eed1 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1bc5:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec1e6 expref: -99 pid: 155747 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826023.862133:0:155747:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/7109 (0 0) 00000080:00200000:0.0:1787826023.862141:0:155747:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00000002:00010000:0.0:1787826023.862198:0:155747:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200026560:0x39:0x0]) in obj [0x200026560:0x39:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826023.862239:0:155747:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6eed8 lrc: 3/1,0 mode: --/CR res: [0x200026560:0x39:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 155747 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826023.862247:0:155747:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6eed8 lrc: 3/1,0 mode: --/CR res: [0x200026560:0x39:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155747 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826023.862256:0:155747:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6eed8 lrc: 3/1,0 mode: --/CR res: [0x200026560:0x39:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155747 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826023.862275:0:155747:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155747:1874663042933248:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787826023.862525:0:155747:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c55daa80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826023.868646:0:155747:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155747:1874663042933248:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:0.0:1787826023.868674:0:155747:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6eed8 lrc: 4/1,0 mode: --/CR res: [0x200026560:0x39:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec1f4 expref: -99 pid: 155747 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826023.868687:0:155747:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6eed8 lrc: 4/1,0 mode: --/PR res: [0x200026560:0x39:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec1f4 expref: -99 pid: 155747 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826023.868699:0:155747:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6eed8 lrc: 4/1,0 mode: PR/PR res: [0x200026560:0x39:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec1f4 expref: -99 pid: 155747 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826023.868710:0:155747:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6eed8 lrc: 4/1,0 mode: PR/PR res: [0x200026560:0x39:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec1f4 expref: -99 pid: 155747 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826023.868723:0:155747:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6eed8 lrc: 5/2,0 mode: PR/PR res: [0x200026560:0x39:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec1f4 expref: -99 pid: 155747 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826023.868732:0:155747:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6eed8 lrc: 5/2,0 mode: PR/PR res: [0x200026560:0x39:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec1f4 expref: -99 pid: 155747 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826023.868739:0:155747:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6eed8 lrc: 4/1,0 mode: PR/PR res: [0x200026560:0x39:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec1f4 expref: -99 pid: 155747 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826023.868932:0:155747:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c55a7800 x1874663042933248/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787826039 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787826023.868946:0:155747:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6eed8 lrc: 3/1,0 mode: PR/PR res: [0x200026560:0x39:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec1f4 expref: -99 pid: 155747 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826023.868965:0:155747:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590091616/57 (0 0) 00010000:00010000:0.0:1787826023.868972:0:155747:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6eed8 lrc: 3/1,0 mode: PR/PR res: [0x200026560:0x39:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec1f4 expref: -99 pid: 155747 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826023.868982:0:155747:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6eed8 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x39:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec1f4 expref: -99 pid: 155747 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000002:00010000:1.0:1787826023.869156:0:155747:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001f58:0x39:0x0]) in obj [0x240001f58:0x39:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826023.869257:0:155747:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4200/0xbef42c0221d6eedf lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x39:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 155747 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.869263:0:155747:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4200/0xbef42c0221d6eedf lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x39:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155747 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.869269:0:155747:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4200/0xbef42c0221d6eedf lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x39:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155747 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826023.869283:0:155747:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155747:1874663042933376:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:1.0:1787826023.869359:0:155747:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b593c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826023.883566:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826023.883671:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049191168 00000100:00100000:0.0:1787826023.883681:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049191168 00000100:00100000:1.0:1787826023.883908:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9247 00000100:00100000:1.0:1787826023.883921:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187905:x1874663049191168:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826023.883939:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6eed8 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x39:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cec1f4 expref: -99 pid: 155747 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826023.884026:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826023.884034:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187905:x1874663049191168:12345-192.168.203.147@tcp:104: Request processed in 116us (478us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826023.884045:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9247 00010000:00010000:3.0:1787826023.884675:0:33350:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826023.884684:0:33350:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6eed8 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x39:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cec1f4 expref: -99 pid: 155747 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826023.884708:0:33350:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0513a00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:3.0:1787826023.884713:0:33350:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6eed8 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x39:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00cec1f4 expref: -99 pid: 155747 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826023.884724:0:33350:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6eed8 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x39:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00cec1f4 expref: -99 pid: 155747 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826023.884734:0:33350:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6eed8 lrc: 3/0,0 mode: PR/PR res: [0x200026560:0x39:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x408400000000 nid: local remote: 0x878a5eea00cec1f4 expref: -99 pid: 155747 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826023.884946:0:33350:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6eed8 lrc: 2/0,0 mode: --/PR res: [0x200026560:0x39:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cec1f4 expref: -99 pid: 155747 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826023.884954:0:33350:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826023.884985:0:33350:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6eed8 lrc: 1/0,0 mode: --/PR res: [0x200026560:0x39:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cec1f4 expref: -99 pid: 155747 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826023.884993:0:33350:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6eed8 lrc: 0/0,0 mode: --/PR res: [0x200026560:0x39:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cec1f4 expref: -99 pid: 155747 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826023.885026:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826023.885029:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826023.885030:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826023.885039:0:2391:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787826023.885058:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2391:1874663042933760:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1787826023.889557:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2391:1874663042933760:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787826023.892222:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826023.892390:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049191424 00000100:00100000:0.0:1787826023.892401:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049191424 00000100:00100000:0.0:1787826023.892459:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9248 00000100:00100000:0.0:1787826023.892470:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663049191424:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826023.892487:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4200/0xbef42c0221d6eedf lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x39:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cec202 expref: -99 pid: 155747 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826023.892572:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826023.892579:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663049191424:12345-192.168.203.147@tcp:104: Request processed in 112us (368us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826023.892587:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9248 00000080:00200000:1.0:1787826023.939032:0:155748:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826023.939079:0:155748:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826023.939098:0:155748:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826023.939114:0:155748:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826023.939122:0:155748:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826023.939129:0:155748:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826023.939132:0:155748:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787826023.939140:0:155748:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501d8bb58 00010000:00010000:1.0:1787826023.939144:0:155748:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826023.939153:0:155748:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826023.939167:0:155748:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826023.939175:0:155748:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=29 00000080:00200000:1.0:1787826023.939181:0:155748:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826023.939192:0:155748:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=13 00000080:00200000:1.0:1787826023.939196:0:155748:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:1.0:1787826023.939208:0:155748:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787826023.939252:0:155748:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6eee6 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155748 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.939261:0:155748:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6eee6 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155748 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.939270:0:155748:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6eee6 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155748 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826023.939295:0:155748:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1880 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155748:1874663042934016:192.168.203.147@tcp:101:lfs.0 00000100:00100000:1.0:1787826023.939517:0:155748:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b59cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826023.944151:0:155748:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1880 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155748:1874663042934016:192.168.203.147@tcp:101:lfs.0 00010000:00010000:1.0:1787826023.944174:0:155748:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6eee6 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155748 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.944188:0:155748:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6eee6 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155748 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.944195:0:155748:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6eee6 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155748 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.944201:0:155748:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6eee6 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155748 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.944211:0:155748:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6eee6 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155748 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.944217:0:155748:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b5200 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:1.0:1787826023.944221:0:155748:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6eee6 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155748 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.944227:0:155748:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6eee6 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155748 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.944234:0:155748:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6eee6 lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 155748 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.944241:0:155748:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787826023.944246:0:155748:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6eee6 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155748 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.944252:0:155748:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5200/0xbef42c0221d6eee6 lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155748 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826023.944271:0:155748:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c35b1880 x1874663042934016/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787826039 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787826023.944285:0:155748:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:1.0:1787826023.944311:0:155748:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:1.0:1787826023.944383:0:155748:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826023.944398:0:155748:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826023.944407:0:155748:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826023.944419:0:155748:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826023.944425:0:155748:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826023.944430:0:155748:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826023.944434:0:155748:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787826023.944439:0:155748:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501d8bbe8 00010000:00010000:1.0:1787826023.944442:0:155748:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826023.944450:0:155748:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826023.944462:0:155748:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826023.944469:0:155748:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=16408 00000080:00200000:1.0:1787826023.944474:0:155748:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826023.944498:0:155748:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=16392 00000080:00200000:1.0:1787826023.944503:0:155748:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:1.0:1787826023.944516:0:155748:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787826023.944553:0:155748:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6eeed lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155748 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.944560:0:155748:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6eeed lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155748 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.944569:0:155748:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6eeed lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155748 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826023.944588:0:155748:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1880 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155748:1874663042934144:192.168.203.147@tcp:101:lfs.0 00000100:00100000:1.0:1787826023.944660:0:155748:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b59cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826023.949175:0:155748:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1880 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155748:1874663042934144:192.168.203.147@tcp:101:lfs.0 00010000:00010000:1.0:1787826023.949197:0:155748:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6eeed lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155748 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.949209:0:155748:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6eeed lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155748 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.949221:0:155748:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6eeed lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155748 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.949227:0:155748:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6eeed lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155748 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.949236:0:155748:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6eeed lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155748 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.949242:0:155748:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4a00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:1.0:1787826023.949247:0:155748:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6eeed lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155748 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.949254:0:155748:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6eeed lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155748 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.949261:0:155748:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6eeed lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 155748 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.949267:0:155748:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787826023.949273:0:155748:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6eeed lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155748 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.949279:0:155748:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4a00/0xbef42c0221d6eeed lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155748 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826023.949296:0:155748:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c35b1880 x1874663042934144/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787826039 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787826023.949310:0:155748:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:1.0:1787826023.949337:0:155748:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:1.0:1787826023.949409:0:155748:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826023.949424:0:155748:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826023.949433:0:155748:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826023.949443:0:155748:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826023.949449:0:155748:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826023.949454:0:155748:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826023.949458:0:155748:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787826023.949463:0:155748:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501d8bb58 00010000:00010000:1.0:1787826023.949466:0:155748:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826023.949475:0:155748:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826023.949498:0:155748:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826023.949506:0:155748:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=13 00000080:00200008:1.0:1787826023.949515:0:155748:0:(file.c:6296:ll_getattr_dentry()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), request_mask 0, flags 0, foreign 0 00000080:00200000:1.0:1787826023.949519:0:155748:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08),name=d71a.replay-single 00000002:00010000:1.0:1787826023.949531:0:155748:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x20001a9e3:0x17e0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826023.949536:0:155748:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00010000:00010000:1.0:1787826023.949567:0:155748:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6eef4 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 155748 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.949578:0:155748:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6eef4 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 155748 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.949587:0:155748:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6eef4 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 155748 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826023.949606:0:155748:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1880 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155748:1874663042934272:192.168.203.147@tcp:101:lfs.0 00000100:00100000:1.0:1787826023.949676:0:155748:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b59cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826023.957355:0:155748:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1880 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155748:1874663042934272:192.168.203.147@tcp:101:lfs.0 00010000:00010000:1.0:1787826023.957383:0:155748:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6eef4 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec264 expref: -99 pid: 155748 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.957400:0:155748:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6eef4 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec264 expref: -99 pid: 155748 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.957409:0:155748:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6eef4 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec264 expref: -99 pid: 155748 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.957420:0:155748:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6eef4 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec264 expref: -99 pid: 155748 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.957430:0:155748:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6eef4 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec264 expref: -99 pid: 155748 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.957443:0:155748:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6eef4 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec264 expref: -99 pid: 155748 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.957457:0:155748:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6eef4 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec264 expref: -99 pid: 155748 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.957465:0:155748:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6eef4 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec264 expref: -99 pid: 155748 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826023.957477:0:155748:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c35b1880 x1874663042934272/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826039 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826023.957502:0:155748:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6eef4 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec264 expref: -99 pid: 155748 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.957512:0:155748:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/6112 (0 0) 00000080:00200000:1.0:1787826023.957518:0:155748:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787826023.957544:0:155748:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x17e0:0x0], updating i_size 4096 00000080:00010000:1.0:1787826023.957551:0:155748:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) for lock 0xbef42c0221d6eef4 00000080:00010000:1.0:1787826023.957558:0:155748:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00000080:00010000:1.0:1787826023.957561:0:155748:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6eef4 from it ffffb97501d8bcb8 00010000:00010000:1.0:1787826023.957564:0:155748:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6eef4 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec264 expref: -99 pid: 155748 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826023.957573:0:155748:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6eef4 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec264 expref: -99 pid: 155748 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:1.0:1787826023.958193:0:155748:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), request_mask 0, flags 0, foreign 0 00000080:00200000:1.0:1787826023.958200:0:155748:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826023.958216:0:155748:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826023.958234:0:155748:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826023.958245:0:155748:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826023.958252:0:155748:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826023.958256:0:155748:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826023.958259:0:155748:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787826023.958264:0:155748:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501d8bcb8 00010000:00010000:1.0:1787826023.958267:0:155748:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826023.958276:0:155748:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826023.958407:0:155748:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826023.958417:0:155748:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826023.958424:0:155748:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826023.958434:0:155748:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826023.958441:0:155748:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826023.958445:0:155748:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826023.958448:0:155748:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787826023.958453:0:155748:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501d8bbf8 00010000:00010000:1.0:1787826023.958457:0:155748:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826023.958465:0:155748:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826023.958475:0:155748:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826023.958481:0:155748:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=267 00000080:00200000:1.0:1787826023.958486:0:155748:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 44 00000080:00200008:1.0:1787826023.958492:0:155748:0:(file.c:1090:ll_file_open()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00000080:00200080:1.0:1787826023.958512:0:155748:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) cmd=c03066f0 arg=7fff3c0183b0 00000080:00200000:1.0:1787826023.958525:0:155748:0:(dir.c:451:ll_dir_setdirstripe()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) name=test stripe_offset=0 stripe_count=2, hash_type=0 00010000:00010000:1.0:1787826023.958567:0:155748:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6eef4 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00cec264 expref: -99 pid: 155748 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826023.958583:0:155748:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6800/0xbef42c0221d64b3f lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9634 expref: -99 pid: 152202 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826023.958627:0:155748:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6eef4 lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cec264 expref: -99 pid: 155748 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826023.958635:0:155748:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826023.958638:0:155748:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6eef4 lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cec264 expref: -99 pid: 155748 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:1.0:1787826023.958657:0:155748:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787826023.958667:0:155748:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c35b1880 x1874663042934528/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 552/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826023.958694:0:155748:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1880 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155748:1874663042934528:192.168.203.147@tcp:36:lfs.0 00000100:00100000:1.0:1787826023.958817:0:155748:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b59cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826023.975219:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042934784:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826023.975308:0:2389:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1787826023.975314:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7528148b-9bee-4d31-b597-8aae3b0b4982:2389:1874663042934656:192.168.203.147@tcp:400:kworker.0 00000080:00200000:3.0:1787826023.975316:0:155749:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:3.0:1787826023.975336:0:155749:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00000100:00100000:1.0:1787826023.975344:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042934912:192.168.203.147@tcp:400:kworker.0 00010000:00010000:3.0:1787826023.975357:0:155749:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826023.975370:0:155749:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826023.975378:0:155749:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826023.975385:0:155749:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:3.0:1787826023.975388:0:155749:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:3.0:1787826023.975395:0:155749:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97500c87c20 00010000:00010000:3.0:1787826023.975399:0:155749:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826023.975409:0:155749:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826023.975422:0:155749:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826023.975430:0:155749:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=24 00000080:00200000:3.0:1787826023.975437:0:155749:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826023.975447:0:155749:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=0 00000080:00200000:3.0:1787826023.975450:0:155749:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:3.0:1787826023.975463:0:155749:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:3.0:1787826023.975497:0:155749:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6eefb lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155749 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.975505:0:155749:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6eefb lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155749 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.975514:0:155749:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6eefb lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155749 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826023.975529:0:155749:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155749:1874663042935040:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:3.0:1787826023.975578:0:155749:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5cb6480 going to sleep for 11 seconds 00000100:00100000:1.0:1787826023.982200:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7528148b-9bee-4d31-b597-8aae3b0b4982:2389:1874663042934656:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826023.983231:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042934912:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826023.983247:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042934784:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826023.983454:0:155749:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155749:1874663042935040:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:3.0:1787826023.983474:0:155749:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6eefb lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec2aa expref: -99 pid: 155749 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.983495:0:155749:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6eefb lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1bc6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec2aa expref: -99 pid: 155749 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.983504:0:155749:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6eefb lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1bc6:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec2aa expref: -99 pid: 155749 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.983516:0:155749:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6eefb lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bc6:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec2aa expref: -99 pid: 155749 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.983533:0:155749:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6eefb lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bc6:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec2aa expref: -99 pid: 155749 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.983546:0:155749:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6eefb lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1bc6:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec2aa expref: -99 pid: 155749 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.983555:0:155749:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6eefb lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1bc6:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec2aa expref: -99 pid: 155749 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.983563:0:155749:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6eefb lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bc6:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec2aa expref: -99 pid: 155749 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826023.983574:0:155749:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c5635c00 x1874663042935040/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/672 e 0 to 0 dl 1787826039 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826023.983588:0:155749:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6eefb lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1bc6:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec2aa expref: -99 pid: 155749 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.983598:0:155749:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/7110 (0 0) 00000080:00200000:3.0:1787826023.983605:0:155749:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00000002:00010000:3.0:1787826023.983673:0:155749:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200026560:0x3a:0x0]) in obj [0x200026560:0x3a:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826023.983716:0:155749:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6ef02 lrc: 3/1,0 mode: --/CR res: [0x200026560:0x3a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 155749 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.983724:0:155749:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6ef02 lrc: 3/1,0 mode: --/CR res: [0x200026560:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155749 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.983732:0:155749:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6ef02 lrc: 3/1,0 mode: --/CR res: [0x200026560:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155749 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826023.983878:0:155749:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155749:1874663042935168:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:3.0:1787826023.984075:0:155749:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5cb6480 going to sleep for 11 seconds 00000100:00100000:3.0:1787826023.989188:0:155749:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155749:1874663042935168:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:3.0:1787826023.989213:0:155749:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6ef02 lrc: 4/1,0 mode: --/CR res: [0x200026560:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec2b8 expref: -99 pid: 155749 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.989225:0:155749:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6ef02 lrc: 4/1,0 mode: --/PR res: [0x200026560:0x3a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec2b8 expref: -99 pid: 155749 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.989235:0:155749:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6ef02 lrc: 4/1,0 mode: PR/PR res: [0x200026560:0x3a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec2b8 expref: -99 pid: 155749 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.989245:0:155749:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6ef02 lrc: 4/1,0 mode: PR/PR res: [0x200026560:0x3a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec2b8 expref: -99 pid: 155749 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.989256:0:155749:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6ef02 lrc: 5/2,0 mode: PR/PR res: [0x200026560:0x3a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec2b8 expref: -99 pid: 155749 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.989264:0:155749:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6ef02 lrc: 5/2,0 mode: PR/PR res: [0x200026560:0x3a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec2b8 expref: -99 pid: 155749 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.989272:0:155749:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6ef02 lrc: 4/1,0 mode: PR/PR res: [0x200026560:0x3a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec2b8 expref: -99 pid: 155749 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826023.989282:0:155749:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c5634e00 x1874663042935168/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787826039 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826023.989301:0:155749:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6ef02 lrc: 3/1,0 mode: PR/PR res: [0x200026560:0x3a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec2b8 expref: -99 pid: 155749 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.989312:0:155749:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590091616/58 (0 0) 00010000:00010000:3.0:1787826023.989319:0:155749:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6ef02 lrc: 3/1,0 mode: PR/PR res: [0x200026560:0x3a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec2b8 expref: -99 pid: 155749 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.989327:0:155749:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6ef02 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x3a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec2b8 expref: -99 pid: 155749 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000002:00010000:3.0:1787826023.989354:0:155749:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001f58:0x3a:0x0]) in obj [0x240001f58:0x3a:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826023.989400:0:155749:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6ef09 lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x3a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 155749 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.989408:0:155749:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6ef09 lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155749 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826023.989418:0:155749:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6ef09 lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155749 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826023.989436:0:155749:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155749:1874663042935296:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:3.0:1787826023.989541:0:155749:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5cb6480 going to sleep for 11 seconds 00000100:00100000:3.0:1787826024.018364:0:155749:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155749:1874663042935296:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:3.0:1787826024.018390:0:155749:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6ef09 lrc: 4/1,0 mode: --/CR res: [0x240001f58:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec2c6 expref: -99 pid: 155749 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.018402:0:155749:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6ef09 lrc: 4/1,0 mode: --/PR res: [0x240001f58:0x3a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec2c6 expref: -99 pid: 155749 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.018414:0:155749:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6ef09 lrc: 4/1,0 mode: PR/PR res: [0x240001f58:0x3a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec2c6 expref: -99 pid: 155749 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.018423:0:155749:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6ef09 lrc: 4/1,0 mode: PR/PR res: [0x240001f58:0x3a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec2c6 expref: -99 pid: 155749 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.018435:0:155749:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6ef09 lrc: 5/2,0 mode: PR/PR res: [0x240001f58:0x3a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec2c6 expref: -99 pid: 155749 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.018443:0:155749:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6ef09 lrc: 5/2,0 mode: PR/PR res: [0x240001f58:0x3a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec2c6 expref: -99 pid: 155749 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.018450:0:155749:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6ef09 lrc: 4/1,0 mode: PR/PR res: [0x240001f58:0x3a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec2c6 expref: -99 pid: 155749 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826024.018461:0:155749:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c5634e00 x1874663042935296/t0(0) o101->lustre-MDT0001-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787826039 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826024.018475:0:155749:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6ef09 lrc: 3/1,0 mode: PR/PR res: [0x240001f58:0x3a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec2c6 expref: -99 pid: 155749 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.018487:0:155749:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d0f54800 type 13 mode 4 res 9663684440/58 (0 0) 00010000:00010000:3.0:1787826024.018494:0:155749:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6ef09 lrc: 3/1,0 mode: PR/PR res: [0x240001f58:0x3a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec2c6 expref: -99 pid: 155749 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.018502:0:155749:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6ef09 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x3a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec2c6 expref: -99 pid: 155749 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826024.018537:0:155749:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1bc6:0x0], updating i_size 8192 00000080:00200000:3.0:1787826024.018541:0:155749:0:(namei.c:147:ll_iget()) got inode: [0x20001a9e3:0x1bc6:0x0](ffff9181c83ea1c8): rc = 0 00000080:00010000:3.0:1787826024.018549:0:155749:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x1bc6:0x0](ffff9181c83ea1c8) for lock 0xbef42c0221d6eefb 00000080:00010000:3.0:1787826024.018564:0:155749:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x1bc6:0x0](ffff9181c83ea1c8) 00000080:00010000:3.0:1787826024.018568:0:155749:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6eefb from it ffffb97500c87e08 00010000:00010000:3.0:1787826024.018572:0:155749:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6eefb lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1bc6:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec2aa expref: -99 pid: 155749 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.018580:0:155749:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6eefb lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1bc6:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec2aa expref: -99 pid: 155749 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826024.018597:0:155749:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 3 00000080:00200000:3.0:1787826024.018604:0:155749:0:(namei.c:2328:ll_rmdir()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00010000:00010000:3.0:1787826024.018629:0:155749:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6eefb lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1bc6:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00cec2aa expref: -99 pid: 155749 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826024.018670:0:155749:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6eefb lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x1bc6:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cec2aa expref: -99 pid: 155749 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826024.018676:0:155749:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826024.018679:0:155749:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6eefb lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x1bc6:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cec2aa expref: -99 pid: 155749 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:3.0:1787826024.018703:0:155749:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:3.0:1787826024.018709:0:155749:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c5635c00 x1874663042935552/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826024.018728:0:155749:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155749:1874663042935552:192.168.203.147@tcp:36:rmdir.0 00000100:00100000:3.0:1787826024.018842:0:155749:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5cb6480 going to sleep for 11 seconds 00000100:00100000:0.0:1787826024.028722:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826024.028893:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049192832 00000100:00100000:1.0:1787826024.028903:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049192832 00000100:00100000:1.0:1787826024.028962:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9249 00000100:00100000:1.0:1787826024.028973:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187904:x1874663049192832:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826024.028990:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481a00/0xbef42c0221d6ef02 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x3a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cec2b8 expref: -99 pid: 155749 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826024.029092:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826024.029099:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187904:x1874663049192832:12345-192.168.203.147@tcp:104: Request processed in 128us (387us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826024.029107:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9249 00000100:00100000:3.0:1787826024.034615:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2391:1874663042935680:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787826024.035291:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826024.035377:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049193088 00000100:00100000:0.0:1787826024.035386:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049193088 00000100:00100000:0.0:1787826024.035440:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9250 00000100:00100000:0.0:1787826024.035451:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b1180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663049193088:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826024.035468:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6ef09 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x3a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cec2c6 expref: -99 pid: 155749 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826024.035541:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826024.035547:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b1180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663049193088:12345-192.168.203.147@tcp:104: Request processed in 98us (260us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826024.035554:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9250 00010000:00010000:3.0:1787826024.035617:0:15160:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826024.035624:0:15160:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6ef09 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x3a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cec2c6 expref: -99 pid: 155749 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826024.035633:0:15160:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3480e00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:3.0:1787826024.035638:0:15160:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6ef09 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x3a:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00cec2c6 expref: -99 pid: 155749 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826024.035647:0:15160:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6ef09 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x3a:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00cec2c6 expref: -99 pid: 155749 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826024.035656:0:15160:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6ef09 lrc: 3/0,0 mode: PR/PR res: [0x240001f58:0x3a:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x408400000000 nid: local remote: 0x878a5eea00cec2c6 expref: -99 pid: 155749 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826024.035715:0:15160:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6ef09 lrc: 2/0,0 mode: --/PR res: [0x240001f58:0x3a:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cec2c6 expref: -99 pid: 155749 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826024.035723:0:15160:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826024.035794:0:15160:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6ef09 lrc: 1/0,0 mode: --/PR res: [0x240001f58:0x3a:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cec2c6 expref: -99 pid: 155749 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826024.035803:0:15160:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6ef09 lrc: 0/0,0 mode: --/PR res: [0x240001f58:0x3a:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cec2c6 expref: -99 pid: 155749 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826024.035825:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826024.035827:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826024.035829:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826024.035837:0:2392:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787826024.035851:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2392:1874663042935808:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1787826024.042039:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2392:1874663042935808:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1787826024.061987:0:155749:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155749:1874663042935552:192.168.203.147@tcp:36:rmdir.0 00000080:00200000:3.0:1787826024.062101:0:155749:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1bc6:0x0](ffff9181c83ea1c8) 00000080:00200000:3.0:1787826024.096999:0:155750:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:3.0:1787826024.097025:0:155750:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826024.097037:0:155750:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826024.097050:0:155750:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826024.097056:0:155750:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826024.097070:0:155750:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:3.0:1787826024.097073:0:155750:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:3.0:1787826024.097079:0:155750:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb9750149bbe8 00010000:00010000:3.0:1787826024.097082:0:155750:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826024.097091:0:155750:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826024.097102:0:155750:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826024.097117:0:155750:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=16408 00000080:00200000:3.0:1787826024.097123:0:155750:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826024.097137:0:155750:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=16392 00000080:00200000:3.0:1787826024.097141:0:155750:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:3.0:1787826024.097152:0:155750:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:3.0:1787826024.097186:0:155750:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6ef17 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155750 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.097193:0:155750:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6ef17 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155750 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.097201:0:155750:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6ef17 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155750 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826024.097220:0:155750:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155750:1874663042936064:192.168.203.147@tcp:101:lfs.0 00000100:00100000:3.0:1787826024.097295:0:155750:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5cb6480 going to sleep for 11 seconds 00000100:00100000:3.0:1787826024.110651:0:155750:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155750:1874663042936064:192.168.203.147@tcp:101:lfs.0 00010000:00010000:3.0:1787826024.110674:0:155750:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6ef17 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155750 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.110687:0:155750:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6ef17 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155750 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.110694:0:155750:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6ef17 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155750 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.110702:0:155750:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6ef17 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155750 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.110710:0:155750:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6ef17 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155750 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.110720:0:155750:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3480e00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:3.0:1787826024.110724:0:155750:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6ef17 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155750 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.110731:0:155750:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6ef17 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155750 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.110737:0:155750:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6ef17 lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 155750 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.110782:0:155750:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826024.110789:0:155750:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6ef17 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155750 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.110796:0:155750:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6ef17 lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155750 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826024.110816:0:155750:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c5634e00 x1874663042936064/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787826040 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:3.0:1787826024.110831:0:155750:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:3.0:1787826024.110857:0:155750:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:3.0:1787826024.110929:0:155750:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:3.0:1787826024.110945:0:155750:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826024.110954:0:155750:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826024.110965:0:155750:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826024.110977:0:155750:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826024.110982:0:155750:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:3.0:1787826024.110985:0:155750:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:3.0:1787826024.110990:0:155750:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb9750149bb58 00010000:00010000:3.0:1787826024.110994:0:155750:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826024.111003:0:155750:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826024.111024:0:155750:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826024.111032:0:155750:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=13 00000080:00200008:3.0:1787826024.111041:0:155750:0:(file.c:6296:ll_getattr_dentry()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), request_mask 0, flags 0, foreign 0 00000080:00200000:3.0:1787826024.111046:0:155750:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08),name=d71a.replay-single 00000002:00010000:3.0:1787826024.111068:0:155750:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x20001a9e3:0x17e0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826024.111073:0:155750:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00010000:00010000:3.0:1787826024.111118:0:155750:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6ef1e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 155750 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.111126:0:155750:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6ef1e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 155750 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.111135:0:155750:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6ef1e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 155750 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826024.111155:0:155750:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155750:1874663042936192:192.168.203.147@tcp:101:lfs.0 00000100:00100000:3.0:1787826024.111228:0:155750:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5cb6480 going to sleep for 11 seconds 00000100:00100000:3.0:1787826024.118969:0:155750:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155750:1874663042936192:192.168.203.147@tcp:101:lfs.0 00010000:00010000:3.0:1787826024.119005:0:155750:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6ef1e lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec328 expref: -99 pid: 155750 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.119029:0:155750:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6ef1e lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec328 expref: -99 pid: 155750 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.119037:0:155750:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6ef1e lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec328 expref: -99 pid: 155750 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.119047:0:155750:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6ef1e lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec328 expref: -99 pid: 155750 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.119068:0:155750:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6ef1e lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec328 expref: -99 pid: 155750 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.119081:0:155750:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6ef1e lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec328 expref: -99 pid: 155750 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.119088:0:155750:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6ef1e lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec328 expref: -99 pid: 155750 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.119094:0:155750:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6ef1e lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec328 expref: -99 pid: 155750 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826024.119105:0:155750:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c5634e00 x1874663042936192/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826040 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826024.119118:0:155750:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6ef1e lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec328 expref: -99 pid: 155750 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.119129:0:155750:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/6112 (0 0) 00000080:00200000:3.0:1787826024.119135:0:155750:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:3.0:1787826024.119160:0:155750:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x17e0:0x0], updating i_size 4096 00000080:00010000:3.0:1787826024.119166:0:155750:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) for lock 0xbef42c0221d6ef1e 00000080:00010000:3.0:1787826024.119173:0:155750:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00000080:00010000:3.0:1787826024.119175:0:155750:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6ef1e from it ffffb9750149bcb8 00010000:00010000:3.0:1787826024.119178:0:155750:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6ef1e lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec328 expref: -99 pid: 155750 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.119187:0:155750:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6ef1e lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec328 expref: -99 pid: 155750 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:3.0:1787826024.119795:0:155750:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), request_mask 0, flags 0, foreign 0 00000080:00200000:3.0:1787826024.119802:0:155750:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:3.0:1787826024.119820:0:155750:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826024.119832:0:155750:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826024.119843:0:155750:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826024.119849:0:155750:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826024.119853:0:155750:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:3.0:1787826024.119856:0:155750:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:3.0:1787826024.119860:0:155750:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb9750149bcb8 00010000:00010000:3.0:1787826024.119863:0:155750:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826024.119871:0:155750:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826024.119991:0:155750:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:3.0:1787826024.120000:0:155750:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826024.120017:0:155750:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826024.120026:0:155750:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826024.120033:0:155750:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826024.120038:0:155750:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:3.0:1787826024.120040:0:155750:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:3.0:1787826024.120044:0:155750:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb9750149bbf8 00010000:00010000:3.0:1787826024.120047:0:155750:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826024.120054:0:155750:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826024.120062:0:155750:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826024.120068:0:155750:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=267 00000080:00200000:3.0:1787826024.120073:0:155750:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 44 00000080:00200008:3.0:1787826024.120078:0:155750:0:(file.c:1090:ll_file_open()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00000080:00200080:3.0:1787826024.120098:0:155750:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) cmd=c03066f0 arg=7ffc82b6a950 00000080:00200000:3.0:1787826024.120112:0:155750:0:(dir.c:451:ll_dir_setdirstripe()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) name=test stripe_offset=0 stripe_count=2, hash_type=0 00010000:00010000:3.0:1787826024.120137:0:155750:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6ef1e lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00cec328 expref: -99 pid: 155750 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826024.120151:0:155750:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6800/0xbef42c0221d64b3f lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9634 expref: -99 pid: 152202 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826024.120190:0:155750:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6ef1e lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cec328 expref: -99 pid: 155750 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826024.120196:0:155750:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826024.120199:0:155750:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6ef1e lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cec328 expref: -99 pid: 155750 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:3.0:1787826024.120215:0:155750:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:3.0:1787826024.120224:0:155750:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c5634e00 x1874663042936448/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 552/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826024.120246:0:155750:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155750:1874663042936448:192.168.203.147@tcp:36:lfs.0 00000100:00100000:3.0:1787826024.120325:0:155750:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5cb6480 going to sleep for 11 seconds 00000100:00100000:3.0:1787826024.140547:0:155750:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155750:1874663042936448:192.168.203.147@tcp:36:lfs.0 00000080:00200000:3.0:1787826024.140598:0:155750:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x20001a9e3:0x1bc7:0x0](ffff9181c83f9148) 00000080:00200000:3.0:1787826024.140605:0:155750:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1bc7:0x0], updating i_size 4096 00000080:00200008:3.0:1787826024.140649:0:155750:0:(file.c:479:ll_file_release()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00010000:00010000:3.0:1787826024.140679:0:155750:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513000/0xbef42c0221d64bee lrc: 2/0,0 mode: CR/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce962d expref: -99 pid: 152210 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000100:00100000:0.0:1787826024.165875:0:155751:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f2300 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155751:1874663042936704:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:0.0:1787826024.165902:0:155751:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6ef2c lrc: 4/1,0 mode: --/CR res: [0x200026560:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec37c expref: -99 pid: 155751 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.165914:0:155751:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6ef2c lrc: 4/1,0 mode: --/PR res: [0x200026560:0x3b:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec37c expref: -99 pid: 155751 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.165925:0:155751:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6ef2c lrc: 4/1,0 mode: PR/PR res: [0x200026560:0x3b:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec37c expref: -99 pid: 155751 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.165936:0:155751:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6ef2c lrc: 4/1,0 mode: PR/PR res: [0x200026560:0x3b:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec37c expref: -99 pid: 155751 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.165948:0:155751:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6ef2c lrc: 5/2,0 mode: PR/PR res: [0x200026560:0x3b:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec37c expref: -99 pid: 155751 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.165956:0:155751:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6ef2c lrc: 5/2,0 mode: PR/PR res: [0x200026560:0x3b:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec37c expref: -99 pid: 155751 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.165970:0:155751:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6ef2c lrc: 4/1,0 mode: PR/PR res: [0x200026560:0x3b:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec37c expref: -99 pid: 155751 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826024.165982:0:155751:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c71f2300 x1874663042936704/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787826040 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787826024.165996:0:155751:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6ef2c lrc: 3/1,0 mode: PR/PR res: [0x200026560:0x3b:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec37c expref: -99 pid: 155751 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.166021:0:155751:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590091616/59 (0 0) 00010000:00010000:0.0:1787826024.166029:0:155751:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6ef2c lrc: 3/1,0 mode: PR/PR res: [0x200026560:0x3b:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec37c expref: -99 pid: 155751 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.166037:0:155751:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6ef2c lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x3b:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec37c expref: -99 pid: 155751 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000002:00010000:0.0:1787826024.166065:0:155751:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001f58:0x3b:0x0]) in obj [0x240001f58:0x3b:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826024.166115:0:155751:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6ef33 lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x3b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 155751 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.166122:0:155751:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6ef33 lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155751 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.166131:0:155751:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6ef33 lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155751 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826024.166150:0:155751:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155751:1874663042936832:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:0.0:1787826024.166236:0:155751:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c55dad80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826024.186200:0:155751:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155751:1874663042936832:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:0.0:1787826024.186226:0:155751:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6ef33 lrc: 4/1,0 mode: --/CR res: [0x240001f58:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec38a expref: -99 pid: 155751 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.186238:0:155751:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6ef33 lrc: 4/1,0 mode: --/PR res: [0x240001f58:0x3b:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec38a expref: -99 pid: 155751 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.186249:0:155751:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6ef33 lrc: 4/1,0 mode: PR/PR res: [0x240001f58:0x3b:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec38a expref: -99 pid: 155751 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.186260:0:155751:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6ef33 lrc: 4/1,0 mode: PR/PR res: [0x240001f58:0x3b:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec38a expref: -99 pid: 155751 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.186272:0:155751:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6ef33 lrc: 5/2,0 mode: PR/PR res: [0x240001f58:0x3b:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec38a expref: -99 pid: 155751 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.186278:0:155751:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6ef33 lrc: 5/2,0 mode: PR/PR res: [0x240001f58:0x3b:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec38a expref: -99 pid: 155751 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.186285:0:155751:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6ef33 lrc: 4/1,0 mode: PR/PR res: [0x240001f58:0x3b:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec38a expref: -99 pid: 155751 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826024.186294:0:155751:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c55a5f80 x1874663042936832/t0(0) o101->lustre-MDT0001-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787826040 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787826024.186306:0:155751:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6ef33 lrc: 3/1,0 mode: PR/PR res: [0x240001f58:0x3b:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec38a expref: -99 pid: 155751 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.186317:0:155751:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d0f54800 type 13 mode 4 res 9663684440/59 (0 0) 00010000:00010000:0.0:1787826024.186325:0:155751:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6ef33 lrc: 3/1,0 mode: PR/PR res: [0x240001f58:0x3b:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec38a expref: -99 pid: 155751 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.186333:0:155751:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6ef33 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x3b:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec38a expref: -99 pid: 155751 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826024.186363:0:155751:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1bc7:0x0], updating i_size 8192 00000080:00200000:0.0:1787826024.186367:0:155751:0:(namei.c:147:ll_iget()) got inode: [0x20001a9e3:0x1bc7:0x0](ffff9181c83f9148): rc = 0 00000080:00010000:0.0:1787826024.186375:0:155751:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x1bc7:0x0](ffff9181c83f9148) for lock 0xbef42c0221d6ef25 00000080:00010000:0.0:1787826024.186395:0:155751:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x1bc7:0x0](ffff9181c83f9148) 00000080:00010000:0.0:1787826024.186399:0:155751:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6ef25 from it ffffb975014fbe08 00010000:00010000:0.0:1787826024.186403:0:155751:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7800/0xbef42c0221d6ef25 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1bc7:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec36e expref: -99 pid: 155751 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.186411:0:155751:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7800/0xbef42c0221d6ef25 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1bc7:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec36e expref: -99 pid: 155751 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826024.186430:0:155751:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 3 00000080:00200000:0.0:1787826024.186438:0:155751:0:(namei.c:2328:ll_rmdir()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00010000:00010000:0.0:1787826024.186467:0:155751:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7800/0xbef42c0221d6ef25 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1bc7:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00cec36e expref: -99 pid: 155751 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826024.186507:0:155751:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7800/0xbef42c0221d6ef25 lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x1bc7:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cec36e expref: -99 pid: 155751 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826024.186514:0:155751:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826024.186517:0:155751:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7800/0xbef42c0221d6ef25 lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x1bc7:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cec36e expref: -99 pid: 155751 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:0.0:1787826024.186546:0:155751:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787826024.186553:0:155751:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c55a5f80 x1874663042937088/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826024.186574:0:155751:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155751:1874663042937088:192.168.203.147@tcp:36:rmdir.0 00000100:00100000:0.0:1787826024.187244:0:155751:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c55dad80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826024.192464:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826024.192554:0:15150:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:0.0:1787826024.192574:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049194496 00000100:00100000:0.0:1787826024.192584:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049194496 00000100:00100000:0.0:1787826024.192651:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9251 00000100:00100000:0.0:1787826024.192661:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181e053d850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187904:x1874663049194496:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826024.192678:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6ef2c lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x3b:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cec37c expref: -99 pid: 155751 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826024.192789:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826024.192796:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181e053d850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187904:x1874663049194496:12345-192.168.203.147@tcp:104: Request processed in 136us (343us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826024.192804:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9251 00010000:00010000:3.0:1787826024.193544:0:33350:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826024.193555:0:33350:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6ef2c lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x3b:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cec37c expref: -99 pid: 155751 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826024.193565:0:33350:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e06f6e00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:3.0:1787826024.193571:0:33350:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6ef2c lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x3b:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00cec37c expref: -99 pid: 155751 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826024.193580:0:33350:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6ef2c lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x3b:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00cec37c expref: -99 pid: 155751 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826024.193589:0:33350:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6ef2c lrc: 3/0,0 mode: PR/PR res: [0x200026560:0x3b:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x408400000000 nid: local remote: 0x878a5eea00cec37c expref: -99 pid: 155751 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826024.193663:0:33350:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6ef2c lrc: 2/0,0 mode: --/PR res: [0x200026560:0x3b:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cec37c expref: -99 pid: 155751 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826024.193670:0:33350:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826024.193708:0:33350:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6ef2c lrc: 1/0,0 mode: --/PR res: [0x200026560:0x3b:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cec37c expref: -99 pid: 155751 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826024.193717:0:33350:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6ef2c lrc: 0/0,0 mode: --/PR res: [0x200026560:0x3b:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cec37c expref: -99 pid: 155751 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826024.193735:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826024.193737:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826024.193739:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826024.198564:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826024.198714:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049194752 00000100:00100000:0.0:1787826024.198724:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049194752 00000100:00100000:0.0:1787826024.198799:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9252 00000100:00100000:0.0:1787826024.198812:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663049194752:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826024.198829:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6ef33 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x3b:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cec38a expref: -99 pid: 155751 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826024.198918:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826024.198926:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663049194752:12345-192.168.203.147@tcp:104: Request processed in 117us (372us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826024.198933:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9252 00000100:00100000:0.0:1787826024.220227:0:155751:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155751:1874663042937088:192.168.203.147@tcp:36:rmdir.0 00000080:00200000:0.0:1787826024.220274:0:155751:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1bc7:0x0](ffff9181c83f9148) 00000080:00200000:0.0:1787826024.236452:0:155752:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826024.236480:0:155752:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826024.236498:0:155752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826024.236514:0:155752:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826024.236521:0:155752:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826024.236529:0:155752:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826024.236532:0:155752:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826024.236539:0:155752:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97500ddbb58 00010000:00010000:0.0:1787826024.236543:0:155752:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826024.236552:0:155752:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826024.236565:0:155752:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826024.236572:0:155752:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=29 00000080:00200000:0.0:1787826024.236577:0:155752:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826024.236588:0:155752:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=13 00000080:00200000:0.0:1787826024.236592:0:155752:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787826024.236603:0:155752:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787826024.236643:0:155752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6ef3a lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.236650:0:155752:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6ef3a lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.236659:0:155752:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6ef3a lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155752 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826024.236683:0:155752:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4a80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155752:1874663042937472:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787826024.236789:0:155752:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c55dad80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826024.248094:0:155752:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4a80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155752:1874663042937472:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787826024.248116:0:155752:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6ef3a lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.248130:0:155752:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6ef3a lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.248138:0:155752:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6ef3a lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.248144:0:155752:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6ef3a lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.248152:0:155752:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6ef3a lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.248163:0:155752:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0513c00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787826024.248167:0:155752:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6ef3a lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.248173:0:155752:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6ef3a lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.248180:0:155752:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6ef3a lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 155752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.248185:0:155752:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787826024.248191:0:155752:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6ef3a lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.248196:0:155752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6ef3a lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155752 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826024.248212:0:155752:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c55a4a80 x1874663042937472/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787826040 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787826024.248225:0:155752:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787826024.248251:0:155752:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787826024.248315:0:155752:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826024.248329:0:155752:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826024.248338:0:155752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826024.248349:0:155752:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826024.248360:0:155752:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826024.248365:0:155752:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826024.248368:0:155752:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826024.248373:0:155752:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97500ddbbe8 00010000:00010000:0.0:1787826024.248377:0:155752:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826024.248385:0:155752:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826024.248397:0:155752:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826024.248404:0:155752:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=16408 00000080:00200000:0.0:1787826024.248409:0:155752:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826024.248422:0:155752:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=16392 00000080:00200000:0.0:1787826024.248426:0:155752:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787826024.248438:0:155752:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787826024.248477:0:155752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6ef41 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.248485:0:155752:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6ef41 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.248493:0:155752:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6ef41 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155752 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826024.248511:0:155752:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4a80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155752:1874663042937600:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787826024.248576:0:155752:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c55dad80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826024.274376:0:155752:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4a80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155752:1874663042937600:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787826024.274402:0:155752:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6ef41 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.274417:0:155752:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6ef41 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.274424:0:155752:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6ef41 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.274432:0:155752:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6ef41 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.274442:0:155752:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6ef41 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.274447:0:155752:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0512e00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787826024.274452:0:155752:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6ef41 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.274459:0:155752:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6ef41 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.274466:0:155752:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6ef41 lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 155752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.274473:0:155752:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787826024.274480:0:155752:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6ef41 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.274486:0:155752:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6ef41 lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155752 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826024.274506:0:155752:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c55a4a80 x1874663042937600/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787826040 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787826024.274520:0:155752:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787826024.274547:0:155752:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787826024.274628:0:155752:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826024.274642:0:155752:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826024.274651:0:155752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826024.274662:0:155752:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826024.274669:0:155752:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826024.274675:0:155752:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826024.274679:0:155752:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826024.274684:0:155752:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97500ddbb58 00010000:00010000:0.0:1787826024.274689:0:155752:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826024.274697:0:155752:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826024.274710:0:155752:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826024.274717:0:155752:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=13 00000080:00200008:0.0:1787826024.274726:0:155752:0:(file.c:6296:ll_getattr_dentry()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1787826024.274731:0:155752:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08),name=d71a.replay-single 00000002:00010000:0.0:1787826024.274817:0:155752:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x20001a9e3:0x17e0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826024.274823:0:155752:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00010000:00010000:0.0:1787826024.274871:0:155752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6ef48 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 155752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.274879:0:155752:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6ef48 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 155752 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.274888:0:155752:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6ef48 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 155752 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826024.274910:0:155752:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4a80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155752:1874663042937728:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787826024.274984:0:155752:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c55dad80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826024.290962:0:155752:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4a80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155752:1874663042937728:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787826024.290986:0:155752:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6ef48 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec3ec expref: -99 pid: 155752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.291003:0:155752:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6ef48 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec3ec expref: -99 pid: 155752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.291022:0:155752:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6ef48 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec3ec expref: -99 pid: 155752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.291032:0:155752:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6ef48 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec3ec expref: -99 pid: 155752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.291044:0:155752:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6ef48 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec3ec expref: -99 pid: 155752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.291131:0:155752:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6ef48 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec3ec expref: -99 pid: 155752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.291140:0:155752:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6ef48 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec3ec expref: -99 pid: 155752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.291147:0:155752:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6ef48 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec3ec expref: -99 pid: 155752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826024.291158:0:155752:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c55a4a80 x1874663042937728/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826040 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787826024.291171:0:155752:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513c00/0xbef42c0221d6ef48 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec3ec expref: -99 pid: 155752 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.291180:0:155752:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787826024.291186:0:155752:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:3.0:1787826024.325171:0:155753:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:3.0:1787826024.325203:0:155753:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826024.325223:0:155753:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826024.325239:0:155753:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826024.325247:0:155753:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826024.325255:0:155753:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:3.0:1787826024.325258:0:155753:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:3.0:1787826024.325265:0:155753:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb9750150bc20 00010000:00010000:3.0:1787826024.325269:0:155753:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826024.325278:0:155753:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826024.325290:0:155753:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826024.325298:0:155753:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=24 00000080:00200000:3.0:1787826024.325304:0:155753:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826024.325317:0:155753:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=0 00000080:00200000:3.0:1787826024.325321:0:155753:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:3.0:1787826024.325332:0:155753:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:3.0:1787826024.325373:0:155753:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6ef4f lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155753 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.325380:0:155753:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6ef4f lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155753 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.325389:0:155753:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6ef4f lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155753 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826024.325417:0:155753:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155753:1874663042938112:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:3.0:1787826024.326466:0:155753:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5cb6480 going to sleep for 11 seconds 00000100:00100000:3.0:1787826024.340977:0:155753:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155753:1874663042938112:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:3.0:1787826024.341015:0:155753:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6ef4f lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec432 expref: -99 pid: 155753 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826024.360037:0:155753:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c67bf480 going to sleep for 11 seconds 00000100:00100000:0.0:1787826024.369604:0:155753:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f2680 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155753:1874663042938368:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:0.0:1787826024.369644:0:155753:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6ef5d lrc: 4/1,0 mode: --/CR res: [0x240001f58:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec44e expref: -99 pid: 155753 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.369656:0:155753:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6ef5d lrc: 4/1,0 mode: --/PR res: [0x240001f58:0x3c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec44e expref: -99 pid: 155753 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.369667:0:155753:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6ef5d lrc: 4/1,0 mode: PR/PR res: [0x240001f58:0x3c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec44e expref: -99 pid: 155753 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.369679:0:155753:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6ef5d lrc: 4/1,0 mode: PR/PR res: [0x240001f58:0x3c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec44e expref: -99 pid: 155753 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.369692:0:155753:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6ef5d lrc: 5/2,0 mode: PR/PR res: [0x240001f58:0x3c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec44e expref: -99 pid: 155753 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.369711:0:155753:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6ef5d lrc: 5/2,0 mode: PR/PR res: [0x240001f58:0x3c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec44e expref: -99 pid: 155753 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.369718:0:155753:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6ef5d lrc: 4/1,0 mode: PR/PR res: [0x240001f58:0x3c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec44e expref: -99 pid: 155753 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826024.369729:0:155753:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c71f2680 x1874663042938368/t0(0) o101->lustre-MDT0001-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787826040 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787826024.369780:0:155753:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6ef5d lrc: 3/1,0 mode: PR/PR res: [0x240001f58:0x3c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec44e expref: -99 pid: 155753 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.369795:0:155753:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d0f54800 type 13 mode 4 res 9663684440/60 (0 0) 00010000:00010000:0.0:1787826024.369804:0:155753:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6ef5d lrc: 3/1,0 mode: PR/PR res: [0x240001f58:0x3c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec44e expref: -99 pid: 155753 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.369813:0:155753:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6ef5d lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x3c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec44e expref: -99 pid: 155753 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826024.369856:0:155753:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1bc8:0x0], updating i_size 8192 00000080:00200000:0.0:1787826024.369860:0:155753:0:(namei.c:147:ll_iget()) got inode: [0x20001a9e3:0x1bc8:0x0](ffff9181d08042c8): rc = 0 00000080:00010000:0.0:1787826024.369869:0:155753:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x1bc8:0x0](ffff9181d08042c8) for lock 0xbef42c0221d6ef4f 00000080:00010000:0.0:1787826024.369887:0:155753:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x1bc8:0x0](ffff9181d08042c8) 00000080:00010000:0.0:1787826024.369891:0:155753:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6ef4f from it ffffb9750150be08 00010000:00010000:0.0:1787826024.369895:0:155753:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6ef4f lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1bc8:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec432 expref: -99 pid: 155753 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.369903:0:155753:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6ef4f lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1bc8:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec432 expref: -99 pid: 155753 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826024.369924:0:155753:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 3 00000080:00200000:0.0:1787826024.369933:0:155753:0:(namei.c:2328:ll_rmdir()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00010000:00010000:0.0:1787826024.369961:0:155753:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6ef4f lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1bc8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00cec432 expref: -99 pid: 155753 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826024.370017:0:155753:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6ef4f lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x1bc8:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cec432 expref: -99 pid: 155753 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826024.370024:0:155753:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826024.370027:0:155753:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480a00/0xbef42c0221d6ef4f lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x1bc8:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cec432 expref: -99 pid: 155753 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:0.0:1787826024.370052:0:155753:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787826024.370069:0:155753:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c55a4e00 x1874663042938624/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826024.370091:0:155753:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155753:1874663042938624:192.168.203.147@tcp:36:rmdir.0 00000100:00100000:0.0:1787826024.370155:0:155753:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c55dad80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826024.374313:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826024.374478:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049196160 00000100:00100000:0.0:1787826024.374488:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049196160 00000100:00100000:0.0:1787826024.374537:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9253 00000100:00100000:0.0:1787826024.374547:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b0e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:188992:x1874663049196160:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826024.374565:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7a00/0xbef42c0221d6ef56 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x3c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cec440 expref: -99 pid: 155753 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826024.374647:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826024.374655:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b0e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:188992:x1874663049196160:12345-192.168.203.147@tcp:104: Request processed in 110us (351us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826024.374663:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9253 00010000:00010000:3.0:1787826024.374728:0:15160:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826024.374736:0:15160:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7a00/0xbef42c0221d6ef56 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x3c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cec440 expref: -99 pid: 155753 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826024.374777:0:15160:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e06f7a00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:3.0:1787826024.374782:0:15160:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7a00/0xbef42c0221d6ef56 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x3c:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00cec440 expref: -99 pid: 155753 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826024.374791:0:15160:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7a00/0xbef42c0221d6ef56 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x3c:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00cec440 expref: -99 pid: 155753 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826024.374799:0:15160:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7a00/0xbef42c0221d6ef56 lrc: 3/0,0 mode: PR/PR res: [0x200026560:0x3c:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x408400000000 nid: local remote: 0x878a5eea00cec440 expref: -99 pid: 155753 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826024.374863:0:15160:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7a00/0xbef42c0221d6ef56 lrc: 2/0,0 mode: --/PR res: [0x200026560:0x3c:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cec440 expref: -99 pid: 155753 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826024.374871:0:15160:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826024.374897:0:15160:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7a00/0xbef42c0221d6ef56 lrc: 1/0,0 mode: --/PR res: [0x200026560:0x3c:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cec440 expref: -99 pid: 155753 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826024.374907:0:15160:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7a00/0xbef42c0221d6ef56 lrc: 0/0,0 mode: --/PR res: [0x200026560:0x3c:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cec440 expref: -99 pid: 155753 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826024.374927:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826024.374929:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826024.374931:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826024.374939:0:2391:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787826024.374953:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2391:1874663042938752:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1787826024.379198:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2391:1874663042938752:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787826024.380502:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826024.381050:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049196416 00000100:00100000:0.0:1787826024.381128:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049196416 00000100:00100000:0.0:1787826024.381193:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9254 00000100:00100000:0.0:1787826024.381204:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663049196416:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826024.381221:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6ef5d lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x3c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cec44e expref: -99 pid: 155753 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826024.381300:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826024.381308:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663049196416:12345-192.168.203.147@tcp:104: Request processed in 106us (812us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826024.381316:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9254 00010000:00010000:3.0:1787826024.382114:0:33350:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826024.382123:0:33350:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6ef5d lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x3c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cec44e expref: -99 pid: 155753 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826024.382135:0:33350:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e06f6a00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:3.0:1787826024.382140:0:33350:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6ef5d lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x3c:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00cec44e expref: -99 pid: 155753 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826024.382150:0:33350:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6ef5d lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x3c:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00cec44e expref: -99 pid: 155753 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826024.382159:0:33350:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6ef5d lrc: 3/0,0 mode: PR/PR res: [0x240001f58:0x3c:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x408400000000 nid: local remote: 0x878a5eea00cec44e expref: -99 pid: 155753 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826024.382219:0:33350:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6ef5d lrc: 2/0,0 mode: --/PR res: [0x240001f58:0x3c:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cec44e expref: -99 pid: 155753 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826024.382232:0:33350:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826024.382867:0:33350:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6ef5d lrc: 1/0,0 mode: --/PR res: [0x240001f58:0x3c:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cec44e expref: -99 pid: 155753 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826024.382878:0:33350:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6a00/0xbef42c0221d6ef5d lrc: 0/0,0 mode: --/PR res: [0x240001f58:0x3c:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cec44e expref: -99 pid: 155753 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826024.382899:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826024.382901:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826024.382903:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826024.401659:0:155753:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155753:1874663042938624:192.168.203.147@tcp:36:rmdir.0 00000080:00200000:0.0:1787826024.401718:0:155753:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1bc8:0x0](ffff9181d08042c8) 00000080:00200000:0.0:1787826024.414626:0:155754:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826024.414963:0:155754:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826024.414984:0:155754:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826024.415017:0:155754:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826024.415025:0:155754:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826024.415034:0:155754:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826024.415038:0:155754:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826024.415046:0:155754:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb975015ebb58 00010000:00010000:0.0:1787826024.415050:0:155754:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826024.415138:0:155754:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826024.415152:0:155754:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826024.415160:0:155754:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=29 00000080:00200000:0.0:1787826024.415167:0:155754:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826024.415178:0:155754:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=13 00000080:00200000:0.0:1787826024.415182:0:155754:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787826024.415197:0:155754:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787826024.415248:0:155754:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6ef64 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155754 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.415256:0:155754:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6ef64 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155754 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.415265:0:155754:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6ef64 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155754 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826024.415296:0:155754:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4a80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155754:1874663042939008:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787826024.415449:0:155754:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c55dad80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826024.419402:0:155754:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4a80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155754:1874663042939008:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787826024.419424:0:155754:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6ef64 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155754 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.419439:0:155754:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6ef64 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155754 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.419447:0:155754:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6ef64 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155754 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.419454:0:155754:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6ef64 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155754 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.419464:0:155754:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6ef64 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155754 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.419470:0:155754:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0512e00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787826024.419475:0:155754:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6ef64 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155754 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.419481:0:155754:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6ef64 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155754 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.419488:0:155754:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6ef64 lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 155754 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.419495:0:155754:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787826024.419503:0:155754:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6ef64 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155754 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.419509:0:155754:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6ef64 lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155754 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826024.419527:0:155754:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c55a4a80 x1874663042939008/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787826040 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787826024.419542:0:155754:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787826024.419570:0:155754:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787826024.420129:0:155754:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826024.420148:0:155754:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826024.420158:0:155754:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826024.420170:0:155754:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826024.420176:0:155754:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826024.420182:0:155754:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826024.420185:0:155754:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826024.420191:0:155754:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb975015ebbe8 00010000:00010000:0.0:1787826024.420194:0:155754:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826024.420202:0:155754:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826024.420214:0:155754:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826024.420222:0:155754:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=16408 00000080:00200000:0.0:1787826024.420227:0:155754:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826024.420238:0:155754:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=16392 00000080:00200000:0.0:1787826024.420243:0:155754:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787826024.420254:0:155754:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787826024.420289:0:155754:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6ef6b lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155754 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.420297:0:155754:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6ef6b lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155754 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.420307:0:155754:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6ef6b lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155754 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826024.420328:0:155754:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4a80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155754:1874663042939136:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787826024.420458:0:155754:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c55dad80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826024.424805:0:155754:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4a80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155754:1874663042939136:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787826024.424829:0:155754:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6ef6b lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155754 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.424843:0:155754:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6ef6b lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155754 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.424929:0:155754:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6ef6b lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155754 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.424936:0:155754:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6ef6b lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155754 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.424944:0:155754:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6ef6b lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155754 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.424951:0:155754:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0513e00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787826024.424956:0:155754:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6ef6b lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155754 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.424962:0:155754:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6ef6b lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155754 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.424970:0:155754:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6ef6b lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 155754 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.424976:0:155754:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787826024.424983:0:155754:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6ef6b lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155754 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.424990:0:155754:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6ef6b lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155754 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826024.425019:0:155754:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c55a4a80 x1874663042939136/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787826040 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787826024.425035:0:155754:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787826024.428160:0:155754:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787826024.428257:0:155754:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826024.428279:0:155754:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826024.428292:0:155754:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826024.428306:0:155754:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826024.428320:0:155754:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826024.428325:0:155754:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826024.428328:0:155754:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826024.428335:0:155754:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb975015ebb58 00010000:00010000:0.0:1787826024.428339:0:155754:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826024.428349:0:155754:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826024.428361:0:155754:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826024.428368:0:155754:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=13 00000080:00200008:0.0:1787826024.428377:0:155754:0:(file.c:6296:ll_getattr_dentry()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1787826024.428382:0:155754:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08),name=d71a.replay-single 00000002:00010000:0.0:1787826024.428392:0:155754:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x20001a9e3:0x17e0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826024.428397:0:155754:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00010000:00010000:0.0:1787826024.428437:0:155754:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6ef72 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 155754 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.428445:0:155754:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6ef72 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 155754 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.428455:0:155754:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6ef72 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 155754 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826024.428479:0:155754:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4a80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155754:1874663042939264:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787826024.428560:0:155754:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c55dad80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826024.432523:0:155754:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4a80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155754:1874663042939264:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787826024.432551:0:155754:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6ef72 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec4b0 expref: -99 pid: 155754 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.432568:0:155754:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6ef72 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec4b0 expref: -99 pid: 155754 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.432576:0:155754:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6ef72 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec4b0 expref: -99 pid: 155754 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.432586:0:155754:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6ef72 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec4b0 expref: -99 pid: 155754 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.432597:0:155754:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6ef72 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec4b0 expref: -99 pid: 155754 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.432610:0:155754:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6ef72 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec4b0 expref: -99 pid: 155754 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.432618:0:155754:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6ef72 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec4b0 expref: -99 pid: 155754 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.432692:0:155754:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6ef72 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec4b0 expref: -99 pid: 155754 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826024.432971:0:155754:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c55a4a80 x1874663042939264/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826040 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787826024.432986:0:155754:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6ef72 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec4b0 expref: -99 pid: 155754 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.432998:0:155754:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787826024.433014:0:155754:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1787826024.433044:0:155754:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x17e0:0x0], updating i_size 4096 00000080:00010000:0.0:1787826024.433050:0:155754:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) for lock 0xbef42c0221d6ef72 00000080:00010000:0.0:1787826024.433068:0:155754:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00000080:00010000:0.0:1787826024.433071:0:155754:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6ef72 from it ffffb975015ebcb8 00010000:00010000:0.0:1787826024.433075:0:155754:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6ef72 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec4b0 expref: -99 pid: 155754 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.433083:0:155754:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6ef72 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec4b0 expref: -99 pid: 155754 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:0.0:1787826024.433649:0:155754:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1787826024.433656:0:155754:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826024.433672:0:155754:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826024.433683:0:155754:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826024.433694:0:155754:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826024.433712:0:155754:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826024.433717:0:155754:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826024.433720:0:155754:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826024.433724:0:155754:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb975015ebcb8 00010000:00010000:0.0:1787826024.433727:0:155754:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826024.433735:0:155754:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826024.437191:0:155754:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826024.437220:0:155754:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826024.437237:0:155754:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826024.437252:0:155754:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826024.437259:0:155754:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826024.437267:0:155754:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826024.437271:0:155754:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826024.437277:0:155754:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb975015ebbf8 00010000:00010000:0.0:1787826024.437281:0:155754:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826024.437290:0:155754:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826024.437304:0:155754:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826024.437311:0:155754:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=267 00000080:00200000:0.0:1787826024.437318:0:155754:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 44 00000080:00200008:0.0:1787826024.437325:0:155754:0:(file.c:1090:ll_file_open()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00000080:00200080:0.0:1787826024.437348:0:155754:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) cmd=c03066f0 arg=7ffe9e9eb690 00000080:00200000:0.0:1787826024.437361:0:155754:0:(dir.c:451:ll_dir_setdirstripe()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) name=test stripe_offset=0 stripe_count=2, hash_type=0 00010000:00010000:0.0:1787826024.437385:0:155754:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6ef72 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00cec4b0 expref: -99 pid: 155754 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826024.437400:0:155754:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6800/0xbef42c0221d64b3f lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9634 expref: -99 pid: 152202 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826024.437443:0:155754:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6ef72 lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cec4b0 expref: -99 pid: 155754 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826024.437451:0:155754:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826024.437453:0:155754:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6ef72 lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cec4b0 expref: -99 pid: 155754 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:0.0:1787826024.437470:0:155754:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787826024.437481:0:155754:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c55a4a80 x1874663042939520/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 552/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826024.437504:0:155754:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4a80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155754:1874663042939520:192.168.203.147@tcp:36:lfs.0 00000100:00100000:0.0:1787826024.437589:0:155754:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c55dad80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826024.446532:0:155754:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4a80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155754:1874663042939520:192.168.203.147@tcp:36:lfs.0 00000080:00200000:0.0:1787826024.446590:0:155754:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x20001a9e3:0x1bc9:0x0](ffff9181c83f3a88) 00000080:00200000:0.0:1787826024.446597:0:155754:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1bc9:0x0], updating i_size 4096 00000080:00200008:0.0:1787826024.446800:0:155754:0:(file.c:479:ll_file_release()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00010000:00010000:0.0:1787826024.446831:0:155754:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513000/0xbef42c0221d64bee lrc: 2/0,0 mode: CR/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce962d expref: -99 pid: 152210 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000100:00100000:1.0:1787826024.479548:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826024.479586:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049197824 00000100:00100000:1.0:1787826024.479595:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049197824 00000100:00100000:1.0:1787826024.479660:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9255 00000100:00100000:1.0:1787826024.479671:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:188992:x1874663049197824:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826024.479687:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6600/0xbef42c0221d6ef80 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x3d:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cec504 expref: -99 pid: 155755 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826024.479808:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826024.479815:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:188992:x1874663049197824:12345-192.168.203.147@tcp:104: Request processed in 146us (278us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826024.479823:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9255 00000100:00100000:1.0:1787826024.486262:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826024.486451:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049198080 00000100:00100000:0.0:1787826024.486461:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049198080 00000100:00100000:0.0:1787826024.486518:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9256 00000100:00100000:0.0:1787826024.486530:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663049198080:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826024.486548:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6c00/0xbef42c0221d6ef87 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x3d:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cec512 expref: -99 pid: 155755 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826024.486988:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826024.486998:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663049198080:12345-192.168.203.147@tcp:104: Request processed in 470us (752us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826024.487014:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9256 00000100:00100000:3.0:1787826024.488616:0:2392:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787826024.488636:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2392:1874663042940416:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1787826024.492248:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2392:1874663042940416:192.168.203.147@tcp:103:ldlm_bl.0 00000002:00010000:3.0:1787826024.535137:0:155756:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x20001a9e3:0x17e0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826024.535144:0:155756:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00010000:00010000:3.0:1787826024.535180:0:155756:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6ef9c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 155756 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.535187:0:155756:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6ef9c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 155756 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.535196:0:155756:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6ef9c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 155756 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826024.535215:0:155756:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155756:1874663042940800:192.168.203.147@tcp:101:lfs.0 00000100:00100000:3.0:1787826024.535286:0:155756:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5cb6480 going to sleep for 11 seconds 00000100:00100000:3.0:1787826024.539740:0:155756:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155756:1874663042940800:192.168.203.147@tcp:101:lfs.0 00010000:00010000:3.0:1787826024.539987:0:155756:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6ef9c lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec574 expref: -99 pid: 155756 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.540014:0:155756:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6ef9c lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec574 expref: -99 pid: 155756 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.540022:0:155756:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6ef9c lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec574 expref: -99 pid: 155756 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.540036:0:155756:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6ef9c lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec574 expref: -99 pid: 155756 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.540046:0:155756:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6ef9c lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec574 expref: -99 pid: 155756 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.540117:0:155756:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6ef9c lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec574 expref: -99 pid: 155756 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.540128:0:155756:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6ef9c lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec574 expref: -99 pid: 155756 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.540137:0:155756:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6ef9c lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec574 expref: -99 pid: 155756 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826024.540149:0:155756:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c5634000 x1874663042940800/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826040 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826024.540162:0:155756:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6ef9c lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec574 expref: -99 pid: 155756 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.540174:0:155756:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/6112 (0 0) 00000080:00200000:3.0:1787826024.540180:0:155756:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:3.0:1787826024.540208:0:155756:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x17e0:0x0], updating i_size 4096 00000080:00010000:3.0:1787826024.540213:0:155756:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) for lock 0xbef42c0221d6ef9c 00000080:00010000:3.0:1787826024.540219:0:155756:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00000080:00010000:3.0:1787826024.540222:0:155756:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6ef9c from it ffffb975014fbcb8 00010000:00010000:3.0:1787826024.540226:0:155756:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6ef9c lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec574 expref: -99 pid: 155756 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.540236:0:155756:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6ef9c lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec574 expref: -99 pid: 155756 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:3.0:1787826024.541040:0:155756:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), request_mask 0, flags 0, foreign 0 00000080:00200000:3.0:1787826024.541047:0:155756:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:3.0:1787826024.541134:0:155756:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826024.541147:0:155756:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826024.541160:0:155756:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826024.541168:0:155756:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826024.541173:0:155756:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:3.0:1787826024.541176:0:155756:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:3.0:1787826024.541181:0:155756:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb975014fbcb8 00010000:00010000:3.0:1787826024.541184:0:155756:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826024.541193:0:155756:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826024.541312:0:155756:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:3.0:1787826024.541320:0:155756:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826024.541327:0:155756:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826024.541335:0:155756:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826024.541342:0:155756:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826024.541346:0:155756:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:3.0:1787826024.541348:0:155756:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:3.0:1787826024.541352:0:155756:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb975014fbbf8 00010000:00010000:3.0:1787826024.541355:0:155756:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826024.541363:0:155756:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826024.541370:0:155756:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826024.541376:0:155756:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=267 00000080:00200000:3.0:1787826024.541381:0:155756:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 44 00000080:00200008:3.0:1787826024.541386:0:155756:0:(file.c:1090:ll_file_open()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00000080:00200080:3.0:1787826024.541403:0:155756:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) cmd=c03066f0 arg=7ffd230e06c0 00000080:00200000:3.0:1787826024.541413:0:155756:0:(dir.c:451:ll_dir_setdirstripe()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) name=test stripe_offset=0 stripe_count=2, hash_type=0 00010000:00010000:3.0:1787826024.541435:0:155756:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6ef9c lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00cec574 expref: -99 pid: 155756 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826024.541448:0:155756:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6800/0xbef42c0221d64b3f lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9634 expref: -99 pid: 152202 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826024.541484:0:155756:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6ef9c lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cec574 expref: -99 pid: 155756 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826024.541491:0:155756:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826024.541494:0:155756:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6ef9c lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cec574 expref: -99 pid: 155756 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:3.0:1787826024.541510:0:155756:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:3.0:1787826024.541518:0:155756:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c5634000 x1874663042941056/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 552/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826024.541539:0:155756:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155756:1874663042941056:192.168.203.147@tcp:36:lfs.0 00000100:00100000:3.0:1787826024.541739:0:155756:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5cb6480 going to sleep for 11 seconds 00000100:00100000:3.0:1787826024.563433:0:155756:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155756:1874663042941056:192.168.203.147@tcp:36:lfs.0 00000080:00200000:3.0:1787826024.563487:0:155756:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x20001a9e3:0x1bca:0x0](ffff9181c83f80c8) 00000080:00200000:3.0:1787826024.563494:0:155756:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1bca:0x0], updating i_size 4096 00000080:00200008:3.0:1787826024.563543:0:155756:0:(file.c:479:ll_file_release()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00010000:00010000:3.0:1787826024.563573:0:155756:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513000/0xbef42c0221d64bee lrc: 2/0,0 mode: CR/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce962d expref: -99 pid: 152210 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826024.586376:0:155761:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826024.586405:0:155761:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826024.586424:0:155761:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826024.586440:0:155761:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826024.586449:0:155761:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826024.586457:0:155761:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826024.586461:0:155761:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787826024.586469:0:155761:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb9750150bc20 00010000:00010000:1.0:1787826024.586472:0:155761:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826024.586485:0:155761:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826024.586498:0:155761:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826024.586505:0:155761:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=24 00000080:00200000:1.0:1787826024.586510:0:155761:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826024.586523:0:155761:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=0 00000080:00200000:1.0:1787826024.586527:0:155761:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:1.0:1787826024.586540:0:155761:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787826024.586587:0:155761:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6efa3 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155761 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826024.586595:0:155761:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6efa3 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155761 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826024.586605:0:155761:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6efa3 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155761 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826024.586630:0:155761:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2300 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155761:1874663042941184:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:1.0:1787826024.586725:0:155761:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b59b40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826024.607201:0:155761:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2300 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155761:1874663042941184:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:1.0:1787826024.607230:0:155761:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6efa3 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec5ba expref: -99 pid: 155761 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826024.607798:0:155761:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6efa3 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1bca:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec5ba expref: -99 pid: 155761 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826024.607815:0:155761:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6efa3 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1bca:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec5ba expref: -99 pid: 155761 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826024.607827:0:155761:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6efa3 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bca:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec5ba expref: -99 pid: 155761 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826024.607837:0:155761:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6efa3 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bca:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec5ba expref: -99 pid: 155761 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826024.607852:0:155761:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6efa3 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1bca:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec5ba expref: -99 pid: 155761 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826024.607931:0:155761:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6efa3 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1bca:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec5ba expref: -99 pid: 155761 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826024.607941:0:155761:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6efa3 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bca:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec5ba expref: -99 pid: 155761 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826024.607956:0:155761:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c35b2300 x1874663042941184/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/672 e 0 to 0 dl 1787826040 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826024.607971:0:155761:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6efa3 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1bca:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec5ba expref: -99 pid: 155761 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826024.607981:0:155761:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/7114 (0 0) 00000080:00200000:1.0:1787826024.607988:0:155761:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00000002:00010000:1.0:1787826024.608339:0:155761:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200026560:0x3e:0x0]) in obj [0x200026560:0x3e:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826024.608390:0:155761:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6efaa lrc: 3/1,0 mode: --/CR res: [0x200026560:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 155761 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826024.608398:0:155761:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6efaa lrc: 3/1,0 mode: --/CR res: [0x200026560:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155761 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826024.608406:0:155761:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6efaa lrc: 3/1,0 mode: --/CR res: [0x200026560:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155761 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826024.608425:0:155761:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155761:1874663042941312:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:1.0:1787826024.608504:0:155761:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b59b40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826024.613179:0:155761:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155761:1874663042941312:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:1.0:1787826024.613205:0:155761:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6efaa lrc: 4/1,0 mode: --/CR res: [0x200026560:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec5c8 expref: -99 pid: 155761 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826024.613218:0:155761:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6efaa lrc: 4/1,0 mode: --/PR res: [0x200026560:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec5c8 expref: -99 pid: 155761 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826024.613230:0:155761:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6efaa lrc: 4/1,0 mode: PR/PR res: [0x200026560:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec5c8 expref: -99 pid: 155761 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826024.613241:0:155761:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6efaa lrc: 4/1,0 mode: PR/PR res: [0x200026560:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec5c8 expref: -99 pid: 155761 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826024.613253:0:155761:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6efaa lrc: 5/2,0 mode: PR/PR res: [0x200026560:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec5c8 expref: -99 pid: 155761 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826024.613341:0:155761:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6efaa lrc: 5/2,0 mode: PR/PR res: [0x200026560:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec5c8 expref: -99 pid: 155761 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826024.613354:0:155761:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6efaa lrc: 4/1,0 mode: PR/PR res: [0x200026560:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec5c8 expref: -99 pid: 155761 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826024.613365:0:155761:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c35b1180 x1874663042941312/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787826040 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826024.613379:0:155761:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6efaa lrc: 3/1,0 mode: PR/PR res: [0x200026560:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec5c8 expref: -99 pid: 155761 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826024.613391:0:155761:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590091616/62 (0 0) 00010000:00010000:1.0:1787826024.613397:0:155761:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6efaa lrc: 3/1,0 mode: PR/PR res: [0x200026560:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec5c8 expref: -99 pid: 155761 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826024.613407:0:155761:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6efaa lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec5c8 expref: -99 pid: 155761 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000002:00010000:1.0:1787826024.613434:0:155761:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001f58:0x3e:0x0]) in obj [0x240001f58:0x3e:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826024.613480:0:155761:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6efb1 lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 155761 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826024.613487:0:155761:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6efb1 lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155761 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826024.613495:0:155761:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6efb1 lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155761 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826024.613516:0:155761:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155761:1874663042941440:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:1.0:1787826024.613596:0:155761:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b59b40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826024.649818:0:155761:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155761:1874663042941440:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:1.0:1787826024.649847:0:155761:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6efb1 lrc: 4/1,0 mode: --/CR res: [0x240001f58:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec5d6 expref: -99 pid: 155761 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826024.649860:0:155761:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6efb1 lrc: 4/1,0 mode: --/PR res: [0x240001f58:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec5d6 expref: -99 pid: 155761 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826024.649871:0:155761:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6efb1 lrc: 4/1,0 mode: PR/PR res: [0x240001f58:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec5d6 expref: -99 pid: 155761 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826024.649882:0:155761:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6efb1 lrc: 4/1,0 mode: PR/PR res: [0x240001f58:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec5d6 expref: -99 pid: 155761 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826024.649894:0:155761:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6efb1 lrc: 5/2,0 mode: PR/PR res: [0x240001f58:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec5d6 expref: -99 pid: 155761 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826024.649902:0:155761:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6efb1 lrc: 5/2,0 mode: PR/PR res: [0x240001f58:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec5d6 expref: -99 pid: 155761 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826024.649909:0:155761:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6efb1 lrc: 4/1,0 mode: PR/PR res: [0x240001f58:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec5d6 expref: -99 pid: 155761 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826024.649921:0:155761:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c35b1180 x1874663042941440/t0(0) o101->lustre-MDT0001-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787826040 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826024.649933:0:155761:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6efb1 lrc: 3/1,0 mode: PR/PR res: [0x240001f58:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec5d6 expref: -99 pid: 155761 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826024.649946:0:155761:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d0f54800 type 13 mode 4 res 9663684440/62 (0 0) 00010000:00010000:1.0:1787826024.649954:0:155761:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6efb1 lrc: 3/1,0 mode: PR/PR res: [0x240001f58:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec5d6 expref: -99 pid: 155761 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826024.649963:0:155761:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6efb1 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec5d6 expref: -99 pid: 155761 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826024.649996:0:155761:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1bca:0x0], updating i_size 8192 00000080:00200000:1.0:1787826024.650000:0:155761:0:(namei.c:147:ll_iget()) got inode: [0x20001a9e3:0x1bca:0x0](ffff9181c83f80c8): rc = 0 00000080:00010000:1.0:1787826024.650027:0:155761:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x1bca:0x0](ffff9181c83f80c8) for lock 0xbef42c0221d6efa3 00000080:00010000:1.0:1787826024.650046:0:155761:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x1bca:0x0](ffff9181c83f80c8) 00000080:00010000:1.0:1787826024.650049:0:155761:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6efa3 from it ffffb9750150be08 00010000:00010000:1.0:1787826024.650053:0:155761:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6efa3 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1bca:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec5ba expref: -99 pid: 155761 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826024.650063:0:155761:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6efa3 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1bca:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec5ba expref: -99 pid: 155761 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826024.650086:0:155761:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 3 00000080:00200000:1.0:1787826024.650096:0:155761:0:(namei.c:2328:ll_rmdir()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00010000:00010000:1.0:1787826024.650130:0:155761:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6efa3 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1bca:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00cec5ba expref: -99 pid: 155761 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826024.650176:0:155761:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6efa3 lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x1bca:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cec5ba expref: -99 pid: 155761 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826024.650184:0:155761:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826024.650187:0:155761:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4400/0xbef42c0221d6efa3 lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x1bca:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cec5ba expref: -99 pid: 155761 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:1.0:1787826024.650216:0:155761:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787826024.650223:0:155761:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c35b2300 x1874663042941696/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826024.650243:0:155761:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2300 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155761:1874663042941696:192.168.203.147@tcp:36:rmdir.0 00000100:00100000:1.0:1787826024.650325:0:155761:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b59b40 going to sleep for 11 seconds 00000100:00100000:0.0:1787826024.656810:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826024.656900:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049199488 00000100:00100000:1.0:1787826024.656909:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049199488 00000100:00100000:1.0:1787826024.656965:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9257 00000100:00100000:1.0:1787826024.656976:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187905:x1874663049199488:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826024.656992:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6efaa lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cec5c8 expref: -99 pid: 155761 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826024.657086:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826024.657093:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187905:x1874663049199488:12345-192.168.203.147@tcp:104: Request processed in 119us (292us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826024.657100:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9257 00010000:00010000:3.0:1787826024.657200:0:15160:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826024.657208:0:15160:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6efaa lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cec5c8 expref: -99 pid: 155761 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826024.657217:0:15160:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b5e00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:3.0:1787826024.657221:0:15160:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6efaa lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x3e:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00cec5c8 expref: -99 pid: 155761 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826024.657231:0:15160:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6efaa lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x3e:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00cec5c8 expref: -99 pid: 155761 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826024.657239:0:15160:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6efaa lrc: 3/0,0 mode: PR/PR res: [0x200026560:0x3e:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x408400000000 nid: local remote: 0x878a5eea00cec5c8 expref: -99 pid: 155761 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826024.657291:0:15160:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6efaa lrc: 2/0,0 mode: --/PR res: [0x200026560:0x3e:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cec5c8 expref: -99 pid: 155761 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826024.657297:0:15160:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826024.657317:0:15160:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6efaa lrc: 1/0,0 mode: --/PR res: [0x200026560:0x3e:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cec5c8 expref: -99 pid: 155761 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826024.657324:0:15160:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6efaa lrc: 0/0,0 mode: --/PR res: [0x200026560:0x3e:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cec5c8 expref: -99 pid: 155761 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826024.657342:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826024.657344:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826024.657345:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826024.657358:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2392:1874663042941824:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1787826024.695226:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826024.696683:0:15150:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:1.0:1787826024.696706:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049199744 00000100:00100000:1.0:1787826024.696715:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049199744 00000100:00100000:1.0:1787826024.697206:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9258 00000100:00100000:1.0:1787826024.697220:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c343fc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663049199744:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826024.697237:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6efb1 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cec5d6 expref: -99 pid: 155761 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826024.697392:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826024.697399:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c343fc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663049199744:12345-192.168.203.147@tcp:104: Request processed in 184us (2184us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826024.697407:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9258 00010000:00010000:3.0:1787826024.697935:0:33350:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826024.697944:0:33350:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6efb1 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x3e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cec5d6 expref: -99 pid: 155761 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826024.697953:0:33350:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4600 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:3.0:1787826024.697958:0:33350:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6efb1 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x3e:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00cec5d6 expref: -99 pid: 155761 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826024.697968:0:33350:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6efb1 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x3e:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00cec5d6 expref: -99 pid: 155761 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826024.697977:0:33350:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6efb1 lrc: 3/0,0 mode: PR/PR res: [0x240001f58:0x3e:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x408400000000 nid: local remote: 0x878a5eea00cec5d6 expref: -99 pid: 155761 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826024.698455:0:33350:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6efb1 lrc: 2/0,0 mode: --/PR res: [0x240001f58:0x3e:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cec5d6 expref: -99 pid: 155761 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826024.698464:0:33350:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826024.698491:0:33350:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6efb1 lrc: 1/0,0 mode: --/PR res: [0x240001f58:0x3e:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cec5d6 expref: -99 pid: 155761 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826024.698499:0:33350:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6efb1 lrc: 0/0,0 mode: --/PR res: [0x240001f58:0x3e:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cec5d6 expref: -99 pid: 155761 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826024.698584:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826024.698586:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826024.698588:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826024.698609:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2391:1874663042941952:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1787826024.709837:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2391:1874663042941952:192.168.203.147@tcp:103:ldlm_bl.0 00000080:00200000:3.0:1787826024.740213:0:155765:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:3.0:1787826024.740243:0:155765:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826024.740262:0:155765:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826024.740278:0:155765:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826024.740286:0:155765:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826024.740295:0:155765:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:3.0:1787826024.740299:0:155765:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:3.0:1787826024.740306:0:155765:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb9750181bb58 00010000:00010000:3.0:1787826024.740311:0:155765:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826024.740321:0:155765:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826024.740336:0:155765:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826024.740344:0:155765:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=29 00000080:00200000:3.0:1787826024.740349:0:155765:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826024.740360:0:155765:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=13 00000080:00200000:3.0:1787826024.740364:0:155765:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:3.0:1787826024.740376:0:155765:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:3.0:1787826024.740421:0:155765:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6efb8 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155765 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.740428:0:155765:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6efb8 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155765 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.740438:0:155765:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6efb8 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155765 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826024.740461:0:155765:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155765:1874663042942080:192.168.203.147@tcp:101:lfs.0 00000100:00100000:3.0:1787826024.740542:0:155765:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5cb6480 going to sleep for 11 seconds 00000100:00100000:3.0:1787826024.745223:0:155765:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155765:1874663042942080:192.168.203.147@tcp:101:lfs.0 00010000:00010000:3.0:1787826024.745245:0:155765:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6efb8 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155765 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.745259:0:155765:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6efb8 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155765 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.745267:0:155765:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6efb8 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155765 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.745274:0:155765:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6efb8 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155765 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.745283:0:155765:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6efb8 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155765 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.745289:0:155765:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3480e00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:3.0:1787826024.745293:0:155765:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6efb8 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155765 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.745300:0:155765:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6efb8 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155765 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.745308:0:155765:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6efb8 lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 155765 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.745317:0:155765:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826024.745324:0:155765:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6efb8 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155765 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.745332:0:155765:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6efb8 lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155765 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826024.745350:0:155765:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c5637b80 x1874663042942080/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787826040 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:3.0:1787826024.745365:0:155765:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:3.0:1787826024.745390:0:155765:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:3.0:1787826024.745455:0:155765:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:3.0:1787826024.745471:0:155765:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826024.745480:0:155765:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826024.745491:0:155765:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826024.745499:0:155765:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826024.745504:0:155765:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:3.0:1787826024.745508:0:155765:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:3.0:1787826024.745513:0:155765:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb9750181bbe8 00010000:00010000:3.0:1787826024.745517:0:155765:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826024.745525:0:155765:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826024.745537:0:155765:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826024.745544:0:155765:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=16408 00000080:00200000:3.0:1787826024.745549:0:155765:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826024.745563:0:155765:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=16392 00000080:00200000:3.0:1787826024.745567:0:155765:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:3.0:1787826024.745579:0:155765:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:3.0:1787826024.745614:0:155765:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480600/0xbef42c0221d6efbf lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155765 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.745622:0:155765:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480600/0xbef42c0221d6efbf lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155765 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.745632:0:155765:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480600/0xbef42c0221d6efbf lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155765 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826024.745652:0:155765:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155765:1874663042942208:192.168.203.147@tcp:101:lfs.0 00000100:00100000:3.0:1787826024.745938:0:155765:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5cb6480 going to sleep for 11 seconds 00000100:00100000:3.0:1787826024.749735:0:155765:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155765:1874663042942208:192.168.203.147@tcp:101:lfs.0 00010000:00010000:3.0:1787826024.749793:0:155765:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480600/0xbef42c0221d6efbf lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155765 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.749806:0:155765:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480600/0xbef42c0221d6efbf lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155765 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.749812:0:155765:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480600/0xbef42c0221d6efbf lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155765 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.749819:0:155765:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480600/0xbef42c0221d6efbf lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155765 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.749827:0:155765:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480600/0xbef42c0221d6efbf lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155765 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.749833:0:155765:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3480600 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:3.0:1787826024.749837:0:155765:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480600/0xbef42c0221d6efbf lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155765 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.749844:0:155765:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480600/0xbef42c0221d6efbf lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155765 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.749850:0:155765:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480600/0xbef42c0221d6efbf lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 155765 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.749857:0:155765:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826024.749863:0:155765:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480600/0xbef42c0221d6efbf lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155765 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.749868:0:155765:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480600/0xbef42c0221d6efbf lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155765 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826024.749884:0:155765:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c5637b80 x1874663042942208/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787826040 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:3.0:1787826024.749898:0:155765:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:3.0:1787826024.749925:0:155765:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:3.0:1787826024.750014:0:155765:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:3.0:1787826024.750029:0:155765:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826024.750037:0:155765:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826024.750053:0:155765:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826024.750213:0:155765:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826024.750219:0:155765:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:3.0:1787826024.750223:0:155765:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:3.0:1787826024.750229:0:155765:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb9750181bb58 00010000:00010000:3.0:1787826024.750233:0:155765:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826024.750241:0:155765:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826024.750254:0:155765:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826024.750262:0:155765:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=13 00000100:00100000:3.0:1787826024.760138:0:155765:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f3100 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155765:1874663042942336:192.168.203.147@tcp:101:lfs.0 00010000:00010000:3.0:1787826024.760168:0:155765:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6c00/0xbef42c0221d6efc6 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec638 expref: -99 pid: 155765 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.760185:0:155765:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6c00/0xbef42c0221d6efc6 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec638 expref: -99 pid: 155765 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.760193:0:155765:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6c00/0xbef42c0221d6efc6 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec638 expref: -99 pid: 155765 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.760205:0:155765:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6c00/0xbef42c0221d6efc6 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec638 expref: -99 pid: 155765 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.760217:0:155765:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6c00/0xbef42c0221d6efc6 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec638 expref: -99 pid: 155765 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.760230:0:155765:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6c00/0xbef42c0221d6efc6 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec638 expref: -99 pid: 155765 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.760238:0:155765:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6c00/0xbef42c0221d6efc6 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec638 expref: -99 pid: 155765 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.760245:0:155765:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6c00/0xbef42c0221d6efc6 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec638 expref: -99 pid: 155765 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826024.760256:0:155765:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c71f3100 x1874663042942336/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826040 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826024.760269:0:155765:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6c00/0xbef42c0221d6efc6 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec638 expref: -99 pid: 155765 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.760278:0:155765:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/6112 (0 0) 00000080:00200000:3.0:1787826024.760284:0:155765:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:3.0:1787826024.760311:0:155765:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x17e0:0x0], updating i_size 4096 00000080:00010000:3.0:1787826024.760316:0:155765:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) for lock 0xbef42c0221d6efc6 00000080:00010000:3.0:1787826024.760323:0:155765:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00000080:00010000:3.0:1787826024.760326:0:155765:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6efc6 from it ffffb9750181bcb8 00010000:00010000:3.0:1787826024.760330:0:155765:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6c00/0xbef42c0221d6efc6 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec638 expref: -99 pid: 155765 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.760343:0:155765:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6c00/0xbef42c0221d6efc6 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec638 expref: -99 pid: 155765 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000100:00100000:3.0:1787826024.786014:0:155765:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f0e00 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155765:1874663042942592:192.168.203.147@tcp:36:lfs.0 00000080:00200000:3.0:1787826024.786072:0:155765:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x20001a9e3:0x1bcb:0x0](ffff9181c83fc2c8) 00000080:00200000:3.0:1787826024.786079:0:155765:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1bcb:0x0], updating i_size 4096 00000080:00200008:3.0:1787826024.786113:0:155765:0:(file.c:479:ll_file_release()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00010000:00010000:3.0:1787826024.786144:0:155765:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513000/0xbef42c0221d64bee lrc: 2/0,0 mode: CR/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce962d expref: -99 pid: 152210 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826024.798681:0:155769:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:3.0:1787826024.798714:0:155769:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826024.798734:0:155769:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826024.801788:0:155769:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826024.801801:0:155769:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826024.801813:0:155769:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:3.0:1787826024.801817:0:155769:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:3.0:1787826024.801827:0:155769:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb975019d3c20 00010000:00010000:3.0:1787826024.801832:0:155769:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826024.801841:0:155769:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826024.801859:0:155769:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826024.801869:0:155769:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=24 00000080:00200000:3.0:1787826024.801876:0:155769:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826024.801890:0:155769:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=0 00000080:00200000:3.0:1787826024.801894:0:155769:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:3.0:1787826024.801913:0:155769:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:3.0:1787826024.801959:0:155769:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480600/0xbef42c0221d6efcd lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155769 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.801968:0:155769:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480600/0xbef42c0221d6efcd lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155769 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.801978:0:155769:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480600/0xbef42c0221d6efcd lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155769 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826024.802012:0:155769:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155769:1874663042942720:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:3.0:1787826024.802092:0:155769:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5cb6480 going to sleep for 11 seconds 00000100:00100000:3.0:1787826024.809472:0:155769:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155769:1874663042942720:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:3.0:1787826024.809500:0:155769:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480600/0xbef42c0221d6efcd lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec67e expref: -99 pid: 155769 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.809521:0:155769:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480600/0xbef42c0221d6efcd lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1bcb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec67e expref: -99 pid: 155769 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.809531:0:155769:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480600/0xbef42c0221d6efcd lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1bcb:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec67e expref: -99 pid: 155769 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.809548:0:155769:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480600/0xbef42c0221d6efcd lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bcb:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec67e expref: -99 pid: 155769 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.809558:0:155769:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480600/0xbef42c0221d6efcd lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bcb:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec67e expref: -99 pid: 155769 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.809571:0:155769:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480600/0xbef42c0221d6efcd lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1bcb:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec67e expref: -99 pid: 155769 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.809578:0:155769:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480600/0xbef42c0221d6efcd lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1bcb:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec67e expref: -99 pid: 155769 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.809584:0:155769:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480600/0xbef42c0221d6efcd lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bcb:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec67e expref: -99 pid: 155769 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826024.809596:0:155769:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c5637b80 x1874663042942720/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/672 e 0 to 0 dl 1787826040 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826024.809611:0:155769:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480600/0xbef42c0221d6efcd lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1bcb:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec67e expref: -99 pid: 155769 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.809621:0:155769:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/7115 (0 0) 00000080:00200000:3.0:1787826024.809629:0:155769:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00000002:00010000:3.0:1787826024.809683:0:155769:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200026560:0x3f:0x0]) in obj [0x200026560:0x3f:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826024.809727:0:155769:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6efd4 lrc: 3/1,0 mode: --/CR res: [0x200026560:0x3f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 155769 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.809739:0:155769:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6efd4 lrc: 3/1,0 mode: --/CR res: [0x200026560:0x3f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155769 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.809782:0:155769:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6efd4 lrc: 3/1,0 mode: --/CR res: [0x200026560:0x3f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155769 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826024.809802:0:155769:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155769:1874663042942848:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:3.0:1787826024.809871:0:155769:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5cb6480 going to sleep for 11 seconds 00000100:00100000:3.0:1787826024.813450:0:155769:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155769:1874663042942848:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:3.0:1787826024.813475:0:155769:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6efd4 lrc: 4/1,0 mode: --/CR res: [0x200026560:0x3f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec68c expref: -99 pid: 155769 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.813487:0:155769:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6efd4 lrc: 4/1,0 mode: --/PR res: [0x200026560:0x3f:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec68c expref: -99 pid: 155769 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.813499:0:155769:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6efd4 lrc: 4/1,0 mode: PR/PR res: [0x200026560:0x3f:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec68c expref: -99 pid: 155769 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.813509:0:155769:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6efd4 lrc: 4/1,0 mode: PR/PR res: [0x200026560:0x3f:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec68c expref: -99 pid: 155769 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.813531:0:155769:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6efd4 lrc: 5/2,0 mode: PR/PR res: [0x200026560:0x3f:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec68c expref: -99 pid: 155769 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.813539:0:155769:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6efd4 lrc: 5/2,0 mode: PR/PR res: [0x200026560:0x3f:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec68c expref: -99 pid: 155769 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.813555:0:155769:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6efd4 lrc: 4/1,0 mode: PR/PR res: [0x200026560:0x3f:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec68c expref: -99 pid: 155769 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826024.813570:0:155769:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c5635f80 x1874663042942848/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787826040 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826024.813583:0:155769:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6efd4 lrc: 3/1,0 mode: PR/PR res: [0x200026560:0x3f:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec68c expref: -99 pid: 155769 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.813596:0:155769:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590091616/63 (0 0) 00010000:00010000:3.0:1787826024.813603:0:155769:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6efd4 lrc: 3/1,0 mode: PR/PR res: [0x200026560:0x3f:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec68c expref: -99 pid: 155769 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.813611:0:155769:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6efd4 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x3f:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec68c expref: -99 pid: 155769 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000002:00010000:3.0:1787826024.813646:0:155769:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001f58:0x3f:0x0]) in obj [0x240001f58:0x3f:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826024.813690:0:155769:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481000/0xbef42c0221d6efdb lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x3f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 155769 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.813708:0:155769:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481000/0xbef42c0221d6efdb lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x3f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155769 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.813716:0:155769:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481000/0xbef42c0221d6efdb lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x3f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155769 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826024.813733:0:155769:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155769:1874663042942976:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:3.0:1787826024.813826:0:155769:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5cb6480 going to sleep for 11 seconds 00000100:00100000:0.0:1787826024.826454:0:155769:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155769:1874663042942976:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:0.0:1787826024.826479:0:155769:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481000/0xbef42c0221d6efdb lrc: 4/1,0 mode: --/CR res: [0x240001f58:0x3f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec69a expref: -99 pid: 155769 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.826492:0:155769:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481000/0xbef42c0221d6efdb lrc: 4/1,0 mode: --/PR res: [0x240001f58:0x3f:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec69a expref: -99 pid: 155769 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.826518:0:155769:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481000/0xbef42c0221d6efdb lrc: 4/1,0 mode: PR/PR res: [0x240001f58:0x3f:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec69a expref: -99 pid: 155769 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.826529:0:155769:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481000/0xbef42c0221d6efdb lrc: 4/1,0 mode: PR/PR res: [0x240001f58:0x3f:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec69a expref: -99 pid: 155769 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.826542:0:155769:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481000/0xbef42c0221d6efdb lrc: 5/2,0 mode: PR/PR res: [0x240001f58:0x3f:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec69a expref: -99 pid: 155769 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.826555:0:155769:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481000/0xbef42c0221d6efdb lrc: 5/2,0 mode: PR/PR res: [0x240001f58:0x3f:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec69a expref: -99 pid: 155769 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.826564:0:155769:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481000/0xbef42c0221d6efdb lrc: 4/1,0 mode: PR/PR res: [0x240001f58:0x3f:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec69a expref: -99 pid: 155769 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826024.826577:0:155769:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c5635f80 x1874663042942976/t0(0) o101->lustre-MDT0001-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787826040 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787826024.826591:0:155769:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481000/0xbef42c0221d6efdb lrc: 3/1,0 mode: PR/PR res: [0x240001f58:0x3f:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec69a expref: -99 pid: 155769 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.826608:0:155769:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d0f54800 type 13 mode 4 res 9663684440/63 (0 0) 00010000:00010000:0.0:1787826024.826618:0:155769:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481000/0xbef42c0221d6efdb lrc: 3/1,0 mode: PR/PR res: [0x240001f58:0x3f:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec69a expref: -99 pid: 155769 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.826625:0:155769:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481000/0xbef42c0221d6efdb lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x3f:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec69a expref: -99 pid: 155769 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826024.826660:0:155769:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1bcb:0x0], updating i_size 8192 00000080:00200000:0.0:1787826024.826664:0:155769:0:(namei.c:147:ll_iget()) got inode: [0x20001a9e3:0x1bcb:0x0](ffff9181c83fc2c8): rc = 0 00000080:00010000:0.0:1787826024.826673:0:155769:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x1bcb:0x0](ffff9181c83fc2c8) for lock 0xbef42c0221d6efcd 00000080:00010000:0.0:1787826024.826689:0:155769:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x1bcb:0x0](ffff9181c83fc2c8) 00000080:00010000:0.0:1787826024.826693:0:155769:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6efcd from it ffffb975019d3e08 00010000:00010000:0.0:1787826024.826697:0:155769:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480600/0xbef42c0221d6efcd lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1bcb:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec67e expref: -99 pid: 155769 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826024.826711:0:155769:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480600/0xbef42c0221d6efcd lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1bcb:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec67e expref: -99 pid: 155769 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826024.826732:0:155769:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 3 00000080:00200000:0.0:1787826024.826740:0:155769:0:(namei.c:2328:ll_rmdir()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00010000:00010000:0.0:1787826024.826813:0:155769:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480600/0xbef42c0221d6efcd lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1bcb:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00cec67e expref: -99 pid: 155769 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826024.826859:0:155769:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480600/0xbef42c0221d6efcd lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x1bcb:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cec67e expref: -99 pid: 155769 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826024.826866:0:155769:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826024.826869:0:155769:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480600/0xbef42c0221d6efcd lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x1bcb:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cec67e expref: -99 pid: 155769 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:0.0:1787826024.826892:0:155769:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787826024.826900:0:155769:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c55a7800 x1874663042943232/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826024.826921:0:155769:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155769:1874663042943232:192.168.203.147@tcp:36:rmdir.0 00000100:00100000:0.0:1787826024.826994:0:155769:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c55da600 going to sleep for 11 seconds 00000100:00100000:0.0:1787826024.849255:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826024.849299:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049201152 00000100:00100000:0.0:1787826024.849308:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049201152 00000100:00100000:0.0:1787826024.849364:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9259 00000100:00100000:0.0:1787826024.849374:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a5180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187906:x1874663049201152:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826024.849395:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6efd4 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x3f:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cec68c expref: -99 pid: 155769 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826024.849482:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826024.849489:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a5180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187906:x1874663049201152:12345-192.168.203.147@tcp:104: Request processed in 116us (244us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826024.849497:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9259 00010000:00010000:3.0:1787826024.850446:0:15161:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826024.850455:0:15161:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6efd4 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x3f:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cec68c expref: -99 pid: 155769 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826024.850467:0:15161:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3480e00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:3.0:1787826024.850472:0:15161:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6efd4 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x3f:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00cec68c expref: -99 pid: 155769 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826024.850482:0:15161:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6efd4 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x3f:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00cec68c expref: -99 pid: 155769 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826024.850491:0:15161:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6efd4 lrc: 3/0,0 mode: PR/PR res: [0x200026560:0x3f:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x408400000000 nid: local remote: 0x878a5eea00cec68c expref: -99 pid: 155769 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826024.850563:0:15161:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6efd4 lrc: 2/0,0 mode: --/PR res: [0x200026560:0x3f:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cec68c expref: -99 pid: 155769 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826024.850570:0:15161:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826024.850593:0:15161:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6efd4 lrc: 1/0,0 mode: --/PR res: [0x200026560:0x3f:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cec68c expref: -99 pid: 155769 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826024.850602:0:15161:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480e00/0xbef42c0221d6efd4 lrc: 0/0,0 mode: --/PR res: [0x200026560:0x3f:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cec68c expref: -99 pid: 155769 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826024.850622:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826024.850624:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826024.850625:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826024.850640:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2392:1874663042943360:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1787826024.870551:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826024.871143:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049201408 00000100:00100000:1.0:1787826024.871155:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049201408 00000100:00100000:0.0:1787826024.871292:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9260 00000100:00100000:0.0:1787826024.871305:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a6300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663049201408:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826024.871327:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481000/0xbef42c0221d6efdb lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x3f:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cec69a expref: -99 pid: 155769 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826024.871398:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826024.871405:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a6300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663049201408:12345-192.168.203.147@tcp:104: Request processed in 104us (865us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826024.871412:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9260 00010000:00010000:0.0:1787826024.871437:0:33329:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826024.871443:0:33329:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481000/0xbef42c0221d6efdb lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x3f:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cec69a expref: -99 pid: 155769 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826024.871452:0:33329:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3481000 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787826024.871457:0:33329:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481000/0xbef42c0221d6efdb lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x3f:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00cec69a expref: -99 pid: 155769 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826024.871466:0:33329:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481000/0xbef42c0221d6efdb lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x3f:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00cec69a expref: -99 pid: 155769 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826024.871475:0:33329:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481000/0xbef42c0221d6efdb lrc: 3/0,0 mode: PR/PR res: [0x240001f58:0x3f:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x408400000000 nid: local remote: 0x878a5eea00cec69a expref: -99 pid: 155769 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826024.871529:0:33329:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481000/0xbef42c0221d6efdb lrc: 2/0,0 mode: --/PR res: [0x240001f58:0x3f:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cec69a expref: -99 pid: 155769 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826024.871536:0:33329:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826024.871555:0:33329:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481000/0xbef42c0221d6efdb lrc: 1/0,0 mode: --/PR res: [0x240001f58:0x3f:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cec69a expref: -99 pid: 155769 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826024.871563:0:33329:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481000/0xbef42c0221d6efdb lrc: 0/0,0 mode: --/PR res: [0x240001f58:0x3f:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cec69a expref: -99 pid: 155769 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826024.871582:0:33329:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826024.871584:0:33329:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826024.871586:0:33329:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826024.871601:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042943488:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1787826024.874733:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2392:1874663042943360:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1787826024.884243:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042943488:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1787826024.891687:0:155769:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155769:1874663042943232:192.168.203.147@tcp:36:rmdir.0 00000080:00200000:0.0:1787826024.891730:0:155769:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1bcb:0x0](ffff9181c83fc2c8) 00000100:00100000:3.0:1787826024.927740:0:155772:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f0000 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155772:1874663042943616:192.168.203.147@tcp:101:lfs.0 00010000:00010000:3.0:1787826024.928197:0:155772:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6efe2 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155772 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.928212:0:155772:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6efe2 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155772 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.928220:0:155772:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6efe2 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155772 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.928228:0:155772:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6efe2 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155772 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.928236:0:155772:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6efe2 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155772 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.928242:0:155772:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e06f6000 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:3.0:1787826024.928246:0:155772:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6efe2 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155772 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.928252:0:155772:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6efe2 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155772 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.928260:0:155772:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6efe2 lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 155772 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.928268:0:155772:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826024.928273:0:155772:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6efe2 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155772 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.928280:0:155772:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6efe2 lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155772 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826024.928297:0:155772:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c71f0000 x1874663042943616/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787826040 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:3.0:1787826024.928312:0:155772:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:3.0:1787826024.928337:0:155772:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:3.0:1787826024.928403:0:155772:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:3.0:1787826024.928417:0:155772:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826024.928426:0:155772:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826024.928436:0:155772:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826024.928442:0:155772:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826024.928448:0:155772:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:3.0:1787826024.928451:0:155772:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:3.0:1787826024.928456:0:155772:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97500d4bbe8 00010000:00010000:3.0:1787826024.928459:0:155772:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826024.928466:0:155772:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826024.928477:0:155772:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826024.928483:0:155772:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=16408 00000080:00200000:3.0:1787826024.928488:0:155772:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826024.928498:0:155772:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=16392 00000080:00200000:3.0:1787826024.928502:0:155772:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:3.0:1787826024.930334:0:155772:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:3.0:1787826024.930398:0:155772:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481400/0xbef42c0221d6efe9 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155772 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.930408:0:155772:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481400/0xbef42c0221d6efe9 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155772 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.930417:0:155772:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481400/0xbef42c0221d6efe9 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155772 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826024.930440:0:155772:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155772:1874663042943744:192.168.203.147@tcp:101:lfs.0 00000100:00100000:3.0:1787826024.930639:0:155772:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5cb6a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826024.947771:0:155772:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f3800 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155772:1874663042943872:192.168.203.147@tcp:101:lfs.0 00010000:00010000:3.0:1787826024.947800:0:155772:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6eff0 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec6fc expref: -99 pid: 155772 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.947819:0:155772:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6eff0 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec6fc expref: -99 pid: 155772 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.947828:0:155772:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6eff0 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec6fc expref: -99 pid: 155772 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.947838:0:155772:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6eff0 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec6fc expref: -99 pid: 155772 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.947849:0:155772:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6eff0 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec6fc expref: -99 pid: 155772 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.947936:0:155772:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6eff0 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec6fc expref: -99 pid: 155772 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.947943:0:155772:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6eff0 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec6fc expref: -99 pid: 155772 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.947951:0:155772:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6eff0 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec6fc expref: -99 pid: 155772 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826024.947964:0:155772:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c71f3800 x1874663042943872/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826040 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826024.947978:0:155772:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6eff0 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec6fc expref: -99 pid: 155772 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.947988:0:155772:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/6112 (0 0) 00000080:00200000:3.0:1787826024.947994:0:155772:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:3.0:1787826024.948032:0:155772:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x17e0:0x0], updating i_size 4096 00000080:00010000:3.0:1787826024.948040:0:155772:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) for lock 0xbef42c0221d6eff0 00000080:00010000:3.0:1787826024.948049:0:155772:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00000080:00010000:3.0:1787826024.948053:0:155772:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6eff0 from it ffffb97500d4bcb8 00010000:00010000:3.0:1787826024.948056:0:155772:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6eff0 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec6fc expref: -99 pid: 155772 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826024.949509:0:155772:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6eff0 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec6fc expref: -99 pid: 155772 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000100:00100000:1.0:1787826025.016887:0:155775:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f2a00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155775:1874663042944256:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:1.0:1787826025.016914:0:155775:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7200/0xbef42c0221d6eff7 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec742 expref: -99 pid: 155775 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826025.016934:0:155775:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7200/0xbef42c0221d6eff7 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1bcc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec742 expref: -99 pid: 155775 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826025.016943:0:155775:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7200/0xbef42c0221d6eff7 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1bcc:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec742 expref: -99 pid: 155775 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826025.016954:0:155775:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7200/0xbef42c0221d6eff7 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bcc:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec742 expref: -99 pid: 155775 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826025.016964:0:155775:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7200/0xbef42c0221d6eff7 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bcc:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec742 expref: -99 pid: 155775 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826025.016978:0:155775:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7200/0xbef42c0221d6eff7 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1bcc:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec742 expref: -99 pid: 155775 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826025.016987:0:155775:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7200/0xbef42c0221d6eff7 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1bcc:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec742 expref: -99 pid: 155775 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826025.016994:0:155775:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7200/0xbef42c0221d6eff7 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bcc:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec742 expref: -99 pid: 155775 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826025.017017:0:155775:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c71f2a00 x1874663042944256/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/672 e 0 to 0 dl 1787826041 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826025.017032:0:155775:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7200/0xbef42c0221d6eff7 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1bcc:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec742 expref: -99 pid: 155775 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826025.017042:0:155775:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/7116 (0 0) 00000080:00200000:1.0:1787826025.017059:0:155775:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00000002:00010000:1.0:1787826025.017114:0:155775:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200026560:0x40:0x0]) in obj [0x200026560:0x40:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826025.017159:0:155775:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6effe lrc: 3/1,0 mode: --/CR res: [0x200026560:0x40:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 155775 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826025.017167:0:155775:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6effe lrc: 3/1,0 mode: --/CR res: [0x200026560:0x40:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155775 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826025.017175:0:155775:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6effe lrc: 3/1,0 mode: --/CR res: [0x200026560:0x40:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155775 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826025.017193:0:155775:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155775:1874663042944384:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:1.0:1787826025.017372:0:155775:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b59c00 going to sleep for 11 seconds 00000100:00100000:3.0:1787826025.049562:0:155775:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f3800 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155775:1874663042944512:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:3.0:1787826025.049584:0:155775:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6f005 lrc: 4/1,0 mode: --/CR res: [0x240001f58:0x40:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec75e expref: -99 pid: 155775 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.049593:0:155775:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6f005 lrc: 4/1,0 mode: --/PR res: [0x240001f58:0x40:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec75e expref: -99 pid: 155775 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.049604:0:155775:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6f005 lrc: 4/1,0 mode: PR/PR res: [0x240001f58:0x40:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec75e expref: -99 pid: 155775 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.049611:0:155775:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6f005 lrc: 4/1,0 mode: PR/PR res: [0x240001f58:0x40:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec75e expref: -99 pid: 155775 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.049621:0:155775:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6f005 lrc: 5/2,0 mode: PR/PR res: [0x240001f58:0x40:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec75e expref: -99 pid: 155775 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.049629:0:155775:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6f005 lrc: 5/2,0 mode: PR/PR res: [0x240001f58:0x40:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec75e expref: -99 pid: 155775 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.049636:0:155775:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6f005 lrc: 4/1,0 mode: PR/PR res: [0x240001f58:0x40:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec75e expref: -99 pid: 155775 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826025.049649:0:155775:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c71f3800 x1874663042944512/t0(0) o101->lustre-MDT0001-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787826041 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826025.049663:0:155775:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6f005 lrc: 3/1,0 mode: PR/PR res: [0x240001f58:0x40:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec75e expref: -99 pid: 155775 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.049677:0:155775:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d0f54800 type 13 mode 4 res 9663684440/64 (0 0) 00010000:00010000:3.0:1787826025.049685:0:155775:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6f005 lrc: 3/1,0 mode: PR/PR res: [0x240001f58:0x40:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec75e expref: -99 pid: 155775 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.049693:0:155775:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6f005 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x40:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec75e expref: -99 pid: 155775 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826025.049729:0:155775:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1bcc:0x0], updating i_size 8192 00000080:00200000:3.0:1787826025.049734:0:155775:0:(namei.c:147:ll_iget()) got inode: [0x20001a9e3:0x1bcc:0x0](ffff9181d08063c8): rc = 0 00000080:00010000:3.0:1787826025.049774:0:155775:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x1bcc:0x0](ffff9181d08063c8) for lock 0xbef42c0221d6eff7 00000080:00010000:3.0:1787826025.049791:0:155775:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x1bcc:0x0](ffff9181d08063c8) 00000080:00010000:3.0:1787826025.049794:0:155775:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6eff7 from it ffffb975019d3e08 00010000:00010000:3.0:1787826025.049798:0:155775:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7200/0xbef42c0221d6eff7 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1bcc:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec742 expref: -99 pid: 155775 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.049806:0:155775:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7200/0xbef42c0221d6eff7 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1bcc:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec742 expref: -99 pid: 155775 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826025.049828:0:155775:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 3 00000080:00200000:3.0:1787826025.049836:0:155775:0:(namei.c:2328:ll_rmdir()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00010000:00010000:3.0:1787826025.049878:0:155775:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7200/0xbef42c0221d6eff7 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1bcc:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00cec742 expref: -99 pid: 155775 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826025.049928:0:155775:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7200/0xbef42c0221d6eff7 lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x1bcc:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cec742 expref: -99 pid: 155775 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826025.049936:0:155775:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826025.049939:0:155775:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7200/0xbef42c0221d6eff7 lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x1bcc:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cec742 expref: -99 pid: 155775 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:3.0:1787826025.049969:0:155775:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:3.0:1787826025.049976:0:155775:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c5635180 x1874663042944768/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826025.049998:0:155775:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635180 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155775:1874663042944768:192.168.203.147@tcp:36:rmdir.0 00000100:00100000:3.0:1787826025.050073:0:155775:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5cb6780 going to sleep for 11 seconds 00000100:00100000:0.0:1787826025.055966:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826025.056007:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049202816 00000100:00100000:0.0:1787826025.056017:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049202816 00000100:00100000:0.0:1787826025.056066:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9261 00000100:00100000:0.0:1787826025.056076:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a7b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187904:x1874663049202816:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826025.056095:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6effe lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x40:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cec750 expref: -99 pid: 155775 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826025.056171:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826025.056178:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a7b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187904:x1874663049202816:12345-192.168.203.147@tcp:104: Request processed in 104us (223us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826025.056185:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9261 00010000:00010000:3.0:1787826025.058135:0:15160:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826025.058148:0:15160:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6effe lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x40:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cec750 expref: -99 pid: 155775 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826025.058159:0:15160:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4c00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:3.0:1787826025.058164:0:15160:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6effe lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x40:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00cec750 expref: -99 pid: 155775 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826025.058173:0:15160:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6effe lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x40:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00cec750 expref: -99 pid: 155775 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826025.058316:0:15160:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6effe lrc: 3/0,0 mode: PR/PR res: [0x200026560:0x40:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x408400000000 nid: local remote: 0x878a5eea00cec750 expref: -99 pid: 155775 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826025.058385:0:15160:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6effe lrc: 2/0,0 mode: --/PR res: [0x200026560:0x40:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cec750 expref: -99 pid: 155775 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826025.058392:0:15160:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826025.058426:0:15160:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6effe lrc: 1/0,0 mode: --/PR res: [0x200026560:0x40:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cec750 expref: -99 pid: 155775 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826025.058439:0:15160:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4c00/0xbef42c0221d6effe lrc: 0/0,0 mode: --/PR res: [0x200026560:0x40:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cec750 expref: -99 pid: 155775 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826025.058456:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826025.058458:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826025.058460:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826025.058468:0:2392:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787826025.058488:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2392:1874663042944896:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1787826025.077063:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826025.077098:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049203072 00000100:00100000:0.0:1787826025.077105:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049203072 00000100:00100000:3.0:1787826025.077138:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2392:1874663042944896:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1787826025.077157:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9262 00000100:00100000:0.0:1787826025.077165:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663049203072:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826025.077182:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6f005 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x40:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cec75e expref: -99 pid: 155775 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826025.077251:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826025.077258:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177386:x1874663049203072:12345-192.168.203.147@tcp:104: Request processed in 94us (206us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826025.077265:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9262 00010000:00010000:0.0:1787826025.077286:0:33350:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826025.077292:0:33350:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6f005 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x40:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cec75e expref: -99 pid: 155775 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826025.077302:0:33350:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e06f6000 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787826025.077306:0:33350:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6f005 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x40:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00cec75e expref: -99 pid: 155775 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826025.077319:0:33350:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6f005 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x40:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00cec75e expref: -99 pid: 155775 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826025.077327:0:33350:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6f005 lrc: 3/0,0 mode: PR/PR res: [0x240001f58:0x40:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x408400000000 nid: local remote: 0x878a5eea00cec75e expref: -99 pid: 155775 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826025.077387:0:33350:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6f005 lrc: 2/0,0 mode: --/PR res: [0x240001f58:0x40:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cec75e expref: -99 pid: 155775 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826025.077394:0:33350:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826025.077411:0:33350:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6f005 lrc: 1/0,0 mode: --/PR res: [0x240001f58:0x40:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cec75e expref: -99 pid: 155775 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826025.077419:0:33350:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181e06f6000/0xbef42c0221d6f005 lrc: 0/0,0 mode: --/PR res: [0x240001f58:0x40:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cec75e expref: -99 pid: 155775 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826025.077437:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826025.077439:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826025.077441:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826025.077452:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042945024:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1787826025.089120:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042945024:192.168.203.147@tcp:103:ldlm_bl.0 00000080:00200000:0.0:1787826025.141933:0:155781:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826025.141967:0:155781:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826025.142341:0:155781:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826025.142365:0:155781:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826025.142374:0:155781:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826025.142381:0:155781:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826025.142385:0:155781:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826025.142392:0:155781:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97500c97b58 00010000:00010000:0.0:1787826025.142397:0:155781:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826025.142406:0:155781:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826025.142419:0:155781:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826025.142425:0:155781:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=29 00000080:00200000:0.0:1787826025.142430:0:155781:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826025.142440:0:155781:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=13 00000080:00200000:0.0:1787826025.142444:0:155781:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787826025.142455:0:155781:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787826025.142497:0:155781:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6f00c lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155781 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826025.142505:0:155781:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6f00c lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155781 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826025.142514:0:155781:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512600/0xbef42c0221d6f00c lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155781 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826025.142541:0:155781:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4000 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155781:1874663042945152:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787826025.142637:0:155781:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c55dad80 going to sleep for 11 seconds 00000100:00100000:0.0:1787826025.155702:0:155781:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f0000 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155781:1874663042945280:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787826025.155728:0:155781:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6c00/0xbef42c0221d6f013 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155781 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826025.155766:0:155781:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6c00/0xbef42c0221d6f013 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155781 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826025.155776:0:155781:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6c00/0xbef42c0221d6f013 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155781 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826025.155782:0:155781:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6c00/0xbef42c0221d6f013 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155781 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826025.155792:0:155781:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6c00/0xbef42c0221d6f013 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155781 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826025.155800:0:155781:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e06f6c00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787826025.155805:0:155781:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6c00/0xbef42c0221d6f013 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155781 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826025.155812:0:155781:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6c00/0xbef42c0221d6f013 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155781 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826025.155819:0:155781:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6c00/0xbef42c0221d6f013 lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 155781 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826025.155826:0:155781:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787826025.155834:0:155781:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6c00/0xbef42c0221d6f013 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155781 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826025.155840:0:155781:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6c00/0xbef42c0221d6f013 lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155781 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826025.155870:0:155781:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c71f0000 x1874663042945280/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787826041 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787826025.155885:0:155781:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:3.0:1787826025.156111:0:155781:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:3.0:1787826025.156195:0:155781:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:3.0:1787826025.156209:0:155781:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826025.156218:0:155781:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826025.156228:0:155781:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826025.156234:0:155781:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826025.156240:0:155781:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:3.0:1787826025.156243:0:155781:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:3.0:1787826025.156248:0:155781:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97500c97b58 00010000:00010000:3.0:1787826025.156251:0:155781:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826025.156265:0:155781:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826025.156276:0:155781:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826025.156284:0:155781:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=13 00000080:00200008:3.0:1787826025.156292:0:155781:0:(file.c:6296:ll_getattr_dentry()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), request_mask 0, flags 0, foreign 0 00000080:00200000:3.0:1787826025.156297:0:155781:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08),name=d71a.replay-single 00000002:00010000:3.0:1787826025.156308:0:155781:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x20001a9e3:0x17e0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826025.156318:0:155781:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00010000:00010000:3.0:1787826025.156352:0:155781:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481600/0xbef42c0221d6f01a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 155781 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.156359:0:155781:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481600/0xbef42c0221d6f01a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 155781 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.156368:0:155781:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481600/0xbef42c0221d6f01a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 155781 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826025.156388:0:155781:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635880 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155781:1874663042945408:192.168.203.147@tcp:101:lfs.0 00000100:00100000:3.0:1787826025.156457:0:155781:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5cb6e40 going to sleep for 11 seconds 00000100:00100000:3.0:1787826025.193591:0:155785:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f0380 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155785:1874663042945792:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:3.0:1787826025.193956:0:155785:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6600/0xbef42c0221d6f021 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec806 expref: -99 pid: 155785 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.194317:0:155785:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6600/0xbef42c0221d6f021 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1bcd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec806 expref: -99 pid: 155785 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.194334:0:155785:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6600/0xbef42c0221d6f021 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1bcd:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec806 expref: -99 pid: 155785 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.194346:0:155785:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6600/0xbef42c0221d6f021 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bcd:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec806 expref: -99 pid: 155785 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.194356:0:155785:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6600/0xbef42c0221d6f021 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bcd:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec806 expref: -99 pid: 155785 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.194369:0:155785:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6600/0xbef42c0221d6f021 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1bcd:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec806 expref: -99 pid: 155785 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.194377:0:155785:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6600/0xbef42c0221d6f021 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1bcd:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec806 expref: -99 pid: 155785 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.194385:0:155785:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6600/0xbef42c0221d6f021 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bcd:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec806 expref: -99 pid: 155785 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826025.194398:0:155785:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c71f0380 x1874663042945792/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/672 e 0 to 0 dl 1787826041 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826025.194411:0:155785:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6600/0xbef42c0221d6f021 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1bcd:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec806 expref: -99 pid: 155785 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.194423:0:155785:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/7117 (0 0) 00000080:00200000:3.0:1787826025.194429:0:155785:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00000002:00010000:3.0:1787826025.194481:0:155785:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200026560:0x41:0x0]) in obj [0x200026560:0x41:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826025.194525:0:155785:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6f028 lrc: 3/1,0 mode: --/CR res: [0x200026560:0x41:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 155785 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.194533:0:155785:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6f028 lrc: 3/1,0 mode: --/CR res: [0x200026560:0x41:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155785 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.194541:0:155785:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6f028 lrc: 3/1,0 mode: --/CR res: [0x200026560:0x41:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155785 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826025.194559:0:155785:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155785:1874663042945920:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:3.0:1787826025.195156:0:155785:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5cb6f00 going to sleep for 11 seconds 00000100:00100000:3.0:1787826025.216242:0:155785:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155785:1874663042945920:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:3.0:1787826025.216267:0:155785:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6f028 lrc: 4/1,0 mode: --/CR res: [0x200026560:0x41:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec814 expref: -99 pid: 155785 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.216279:0:155785:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6f028 lrc: 4/1,0 mode: --/PR res: [0x200026560:0x41:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec814 expref: -99 pid: 155785 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.216290:0:155785:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6f028 lrc: 4/1,0 mode: PR/PR res: [0x200026560:0x41:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec814 expref: -99 pid: 155785 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.216300:0:155785:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6f028 lrc: 4/1,0 mode: PR/PR res: [0x200026560:0x41:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec814 expref: -99 pid: 155785 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.216312:0:155785:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6f028 lrc: 5/2,0 mode: PR/PR res: [0x200026560:0x41:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec814 expref: -99 pid: 155785 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.216321:0:155785:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6f028 lrc: 5/2,0 mode: PR/PR res: [0x200026560:0x41:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec814 expref: -99 pid: 155785 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.216333:0:155785:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6f028 lrc: 4/1,0 mode: PR/PR res: [0x200026560:0x41:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec814 expref: -99 pid: 155785 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826025.216344:0:155785:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c5637480 x1874663042945920/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787826041 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826025.216358:0:155785:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6f028 lrc: 3/1,0 mode: PR/PR res: [0x200026560:0x41:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec814 expref: -99 pid: 155785 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.216370:0:155785:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590091616/65 (0 0) 00010000:00010000:3.0:1787826025.216377:0:155785:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6f028 lrc: 3/1,0 mode: PR/PR res: [0x200026560:0x41:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec814 expref: -99 pid: 155785 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.216385:0:155785:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6f028 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x41:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec814 expref: -99 pid: 155785 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000002:00010000:3.0:1787826025.216409:0:155785:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001f58:0x41:0x0]) in obj [0x240001f58:0x41:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826025.216452:0:155785:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480000/0xbef42c0221d6f02f lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x41:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 155785 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.216459:0:155785:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480000/0xbef42c0221d6f02f lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x41:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155785 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.216467:0:155785:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480000/0xbef42c0221d6f02f lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x41:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155785 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826025.216483:0:155785:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155785:1874663042946048:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:3.0:1787826025.216591:0:155785:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5cb6f00 going to sleep for 11 seconds 00000100:00100000:3.0:1787826025.229831:0:155785:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155785:1874663042946048:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:3.0:1787826025.229856:0:155785:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480000/0xbef42c0221d6f02f lrc: 4/1,0 mode: --/CR res: [0x240001f58:0x41:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec822 expref: -99 pid: 155785 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.229868:0:155785:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480000/0xbef42c0221d6f02f lrc: 4/1,0 mode: --/PR res: [0x240001f58:0x41:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec822 expref: -99 pid: 155785 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.229878:0:155785:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480000/0xbef42c0221d6f02f lrc: 4/1,0 mode: PR/PR res: [0x240001f58:0x41:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec822 expref: -99 pid: 155785 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.229893:0:155785:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480000/0xbef42c0221d6f02f lrc: 4/1,0 mode: PR/PR res: [0x240001f58:0x41:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec822 expref: -99 pid: 155785 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.229906:0:155785:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480000/0xbef42c0221d6f02f lrc: 5/2,0 mode: PR/PR res: [0x240001f58:0x41:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec822 expref: -99 pid: 155785 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.229914:0:155785:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480000/0xbef42c0221d6f02f lrc: 5/2,0 mode: PR/PR res: [0x240001f58:0x41:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec822 expref: -99 pid: 155785 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.229920:0:155785:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480000/0xbef42c0221d6f02f lrc: 4/1,0 mode: PR/PR res: [0x240001f58:0x41:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec822 expref: -99 pid: 155785 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826025.229932:0:155785:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c5637480 x1874663042946048/t0(0) o101->lustre-MDT0001-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787826041 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826025.229945:0:155785:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480000/0xbef42c0221d6f02f lrc: 3/1,0 mode: PR/PR res: [0x240001f58:0x41:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec822 expref: -99 pid: 155785 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.229958:0:155785:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d0f54800 type 13 mode 4 res 9663684440/65 (0 0) 00010000:00010000:3.0:1787826025.229965:0:155785:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480000/0xbef42c0221d6f02f lrc: 3/1,0 mode: PR/PR res: [0x240001f58:0x41:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec822 expref: -99 pid: 155785 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.229973:0:155785:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480000/0xbef42c0221d6f02f lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x41:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec822 expref: -99 pid: 155785 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826025.230016:0:155785:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1bcd:0x0], updating i_size 8192 00000080:00200000:3.0:1787826025.230021:0:155785:0:(namei.c:147:ll_iget()) got inode: [0x20001a9e3:0x1bcd:0x0](ffff9181d08063c8): rc = 0 00000080:00010000:3.0:1787826025.230029:0:155785:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x1bcd:0x0](ffff9181d08063c8) for lock 0xbef42c0221d6f021 00000080:00010000:3.0:1787826025.230047:0:155785:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x1bcd:0x0](ffff9181d08063c8) 00000080:00010000:3.0:1787826025.230051:0:155785:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f021 from it ffffb97501813e08 00010000:00010000:3.0:1787826025.230054:0:155785:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6600/0xbef42c0221d6f021 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1bcd:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec806 expref: -99 pid: 155785 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.230074:0:155785:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6600/0xbef42c0221d6f021 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1bcd:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec806 expref: -99 pid: 155785 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826025.230095:0:155785:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 3 00000080:00200000:3.0:1787826025.230104:0:155785:0:(namei.c:2328:ll_rmdir()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00010000:00010000:3.0:1787826025.230132:0:155785:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6600/0xbef42c0221d6f021 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1bcd:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00cec806 expref: -99 pid: 155785 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826025.230179:0:155785:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6600/0xbef42c0221d6f021 lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x1bcd:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cec806 expref: -99 pid: 155785 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826025.230185:0:155785:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826025.230189:0:155785:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6600/0xbef42c0221d6f021 lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x1bcd:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cec806 expref: -99 pid: 155785 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:3.0:1787826025.230218:0:155785:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:3.0:1787826025.230224:0:155785:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c5637480 x1874663042946304/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826025.230244:0:155785:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155785:1874663042946304:192.168.203.147@tcp:36:rmdir.0 00000100:00100000:3.0:1787826025.230314:0:155785:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5cb6f00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826025.237492:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826025.237597:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049204480 00000100:00100000:0.0:1787826025.237607:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049204480 00000100:00100000:0.0:1787826025.237677:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9263 00000100:00100000:0.0:1787826025.237687:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b0e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187905:x1874663049204480:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826025.237712:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6f028 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x41:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cec814 expref: -99 pid: 155785 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826025.237844:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826025.237851:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b0e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187905:x1874663049204480:12345-192.168.203.147@tcp:104: Request processed in 166us (369us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826025.237860:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9263 00010000:00010000:3.0:1787826025.238261:0:15161:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826025.238269:0:15161:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6f028 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x41:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cec814 expref: -99 pid: 155785 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826025.238277:0:15161:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3481e00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:3.0:1787826025.238282:0:15161:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6f028 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x41:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00cec814 expref: -99 pid: 155785 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826025.238290:0:15161:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6f028 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x41:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00cec814 expref: -99 pid: 155785 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826025.238298:0:15161:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6f028 lrc: 3/0,0 mode: PR/PR res: [0x200026560:0x41:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x408400000000 nid: local remote: 0x878a5eea00cec814 expref: -99 pid: 155785 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826025.238354:0:15161:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6f028 lrc: 2/0,0 mode: --/PR res: [0x200026560:0x41:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cec814 expref: -99 pid: 155785 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826025.238361:0:15161:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826025.238391:0:15161:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6f028 lrc: 1/0,0 mode: --/PR res: [0x200026560:0x41:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cec814 expref: -99 pid: 155785 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826025.238404:0:15161:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6f028 lrc: 0/0,0 mode: --/PR res: [0x200026560:0x41:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cec814 expref: -99 pid: 155785 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826025.238422:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826025.238424:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826025.238426:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826025.250869:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826025.250992:0:15150:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049204736 00000100:00100000:1.0:1787826025.251028:0:15150:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049204736 00000100:00100000:1.0:1787826025.251095:0:15150:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9264 00000100:00100000:1.0:1787826025.251106:0:15150:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a6680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663049204736:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826025.251125:0:15150:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3480000/0xbef42c0221d6f02f lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x41:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cec822 expref: -99 pid: 155785 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826025.251214:0:15150:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826025.251222:0:15150:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a6680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:177390:x1874663049204736:12345-192.168.203.147@tcp:104: Request processed in 118us (362us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826025.251229:0:15150:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9264 00000100:00100000:3.0:1787826025.268353:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2392:1874663042946560:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1787826025.288013:0:155785:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155785:1874663042946304:192.168.203.147@tcp:36:rmdir.0 00000080:00200000:3.0:1787826025.288063:0:155785:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1bcd:0x0](ffff9181d08063c8) 00000080:00200000:3.0:1787826025.305689:0:155789:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:3.0:1787826025.305721:0:155789:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826025.305739:0:155789:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000100:00100000:3.0:1787826025.330667:0:155789:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f0700 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155789:1874663042946816:192.168.203.147@tcp:101:lfs.0 00010000:00010000:3.0:1787826025.330690:0:155789:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6f03d lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155789 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.330704:0:155789:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6f03d lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155789 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.330712:0:155789:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6f03d lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155789 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.330720:0:155789:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6f03d lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155789 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.330730:0:155789:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6f03d lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155789 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.330736:0:155789:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e06f6e00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:3.0:1787826025.330740:0:155789:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6f03d lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155789 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.330762:0:155789:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6f03d lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155789 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.330778:0:155789:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6f03d lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 155789 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.330785:0:155789:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826025.330792:0:155789:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6f03d lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155789 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.330799:0:155789:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6e00/0xbef42c0221d6f03d lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155789 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826025.330816:0:155789:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c71f0700 x1874663042946816/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787826041 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:3.0:1787826025.330831:0:155789:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:3.0:1787826025.330857:0:155789:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:3.0:1787826025.330921:0:155789:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:3.0:1787826025.330935:0:155789:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826025.330944:0:155789:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826025.330958:0:155789:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826025.330964:0:155789:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826025.330969:0:155789:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:3.0:1787826025.330972:0:155789:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:3.0:1787826025.330976:0:155789:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501483b58 00010000:00010000:3.0:1787826025.330980:0:155789:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826025.330987:0:155789:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826025.331008:0:155789:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826025.331015:0:155789:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=13 00000080:00200008:3.0:1787826025.331023:0:155789:0:(file.c:6296:ll_getattr_dentry()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), request_mask 0, flags 0, foreign 0 00000080:00200000:3.0:1787826025.331029:0:155789:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08),name=d71a.replay-single 00000002:00010000:3.0:1787826025.331041:0:155789:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x20001a9e3:0x17e0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826025.331046:0:155789:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00010000:00010000:3.0:1787826025.331076:0:155789:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6f044 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 155789 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.331083:0:155789:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6f044 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 155789 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.331090:0:155789:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6f044 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 155789 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826025.331111:0:155789:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636680 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155789:1874663042946944:192.168.203.147@tcp:101:lfs.0 00000100:00100000:3.0:1787826025.331185:0:155789:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5cb6f00 going to sleep for 11 seconds 00000100:00100000:3.0:1787826025.338558:0:155789:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636680 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155789:1874663042946944:192.168.203.147@tcp:101:lfs.0 00010000:00010000:3.0:1787826025.338584:0:155789:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6f044 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec884 expref: -99 pid: 155789 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.338600:0:155789:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6f044 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec884 expref: -99 pid: 155789 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.338608:0:155789:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6f044 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec884 expref: -99 pid: 155789 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.338619:0:155789:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6f044 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec884 expref: -99 pid: 155789 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.338629:0:155789:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6f044 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec884 expref: -99 pid: 155789 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.338642:0:155789:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6f044 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec884 expref: -99 pid: 155789 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.338650:0:155789:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6f044 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec884 expref: -99 pid: 155789 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.338656:0:155789:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6f044 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec884 expref: -99 pid: 155789 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826025.338672:0:155789:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c5636680 x1874663042946944/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826041 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826025.338685:0:155789:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6f044 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec884 expref: -99 pid: 155789 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.338694:0:155789:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/6112 (0 0) 00000080:00200000:3.0:1787826025.338701:0:155789:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:3.0:1787826025.338728:0:155789:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x17e0:0x0], updating i_size 4096 00000080:00010000:3.0:1787826025.338734:0:155789:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) for lock 0xbef42c0221d6f044 00000080:00010000:3.0:1787826025.338741:0:155789:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00000080:00010000:3.0:1787826025.338759:0:155789:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f044 from it ffffb97501483cb8 00010000:00010000:3.0:1787826025.338763:0:155789:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6f044 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec884 expref: -99 pid: 155789 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.338783:0:155789:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6f044 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec884 expref: -99 pid: 155789 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:3.0:1787826025.339361:0:155789:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), request_mask 0, flags 0, foreign 0 00000080:00200000:3.0:1787826025.339420:0:155789:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:3.0:1787826025.339439:0:155789:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826025.339449:0:155789:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826025.339462:0:155789:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826025.339474:0:155789:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826025.339483:0:155789:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:3.0:1787826025.339487:0:155789:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:3.0:1787826025.339491:0:155789:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501483cb8 00010000:00010000:3.0:1787826025.339495:0:155789:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826025.339503:0:155789:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826025.339618:0:155789:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:3.0:1787826025.339627:0:155789:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826025.339634:0:155789:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826025.339643:0:155789:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826025.339650:0:155789:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826025.339655:0:155789:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:3.0:1787826025.339657:0:155789:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:3.0:1787826025.339661:0:155789:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb97501483bf8 00010000:00010000:3.0:1787826025.339664:0:155789:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826025.339671:0:155789:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826025.339680:0:155789:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826025.339687:0:155789:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=267 00000080:00200000:3.0:1787826025.339692:0:155789:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 44 00000080:00200008:3.0:1787826025.339696:0:155789:0:(file.c:1090:ll_file_open()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00000080:00200080:3.0:1787826025.339716:0:155789:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) cmd=c03066f0 arg=7fff5e224130 00000080:00200000:3.0:1787826025.339728:0:155789:0:(dir.c:451:ll_dir_setdirstripe()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) name=test stripe_offset=0 stripe_count=2, hash_type=0 00010000:00010000:3.0:1787826025.339784:0:155789:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6f044 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00cec884 expref: -99 pid: 155789 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826025.339801:0:155789:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6800/0xbef42c0221d64b3f lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9634 expref: -99 pid: 152202 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826025.339838:0:155789:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6f044 lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cec884 expref: -99 pid: 155789 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826025.339845:0:155789:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826025.339848:0:155789:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6f044 lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cec884 expref: -99 pid: 155789 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:3.0:1787826025.339865:0:155789:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:3.0:1787826025.339873:0:155789:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c5636680 x1874663042947200/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 552/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826025.339895:0:155789:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636680 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155789:1874663042947200:192.168.203.147@tcp:36:lfs.0 00000100:00100000:3.0:1787826025.339969:0:155789:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5cb6f00 going to sleep for 11 seconds 00000100:00100000:3.0:1787826025.360847:0:155789:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636680 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155789:1874663042947200:192.168.203.147@tcp:36:lfs.0 00000080:00200000:3.0:1787826025.360908:0:155789:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x20001a9e3:0x1bce:0x0](ffff9181c83f9148) 00000080:00200000:3.0:1787826025.360916:0:155789:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1bce:0x0], updating i_size 4096 00000080:00200008:3.0:1787826025.360949:0:155789:0:(file.c:479:ll_file_release()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00010000:00010000:3.0:1787826025.360978:0:155789:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513000/0xbef42c0221d64bee lrc: 2/0,0 mode: CR/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce962d expref: -99 pid: 152210 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826025.367348:0:155796:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:3.0:1787826025.367375:0:155796:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826025.367393:0:155796:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826025.367407:0:155796:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826025.367415:0:155796:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826025.367422:0:155796:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:3.0:1787826025.367425:0:155796:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:3.0:1787826025.367432:0:155796:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb975015ebc20 00010000:00010000:3.0:1787826025.367436:0:155796:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826025.367445:0:155796:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826025.367458:0:155796:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826025.367464:0:155796:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=24 00000080:00200000:3.0:1787826025.367470:0:155796:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826025.367481:0:155796:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=0 00000080:00200000:3.0:1787826025.367486:0:155796:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:3.0:1787826025.367498:0:155796:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:3.0:1787826025.367540:0:155796:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6f04b lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155796 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.367547:0:155796:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6f04b lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155796 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.367556:0:155796:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6f04b lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155796 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826025.367580:0:155796:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155796:1874663042947328:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:3.0:1787826025.367672:0:155796:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5cb6f00 going to sleep for 11 seconds 00000100:00100000:3.0:1787826025.384026:0:155796:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155796:1874663042947328:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:3.0:1787826025.384130:0:155796:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6f04b lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec8ca expref: -99 pid: 155796 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.384155:0:155796:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6f04b lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1bce:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec8ca expref: -99 pid: 155796 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.384164:0:155796:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6f04b lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1bce:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec8ca expref: -99 pid: 155796 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.384174:0:155796:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6f04b lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bce:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec8ca expref: -99 pid: 155796 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.384185:0:155796:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6f04b lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bce:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec8ca expref: -99 pid: 155796 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.384202:0:155796:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6f04b lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1bce:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec8ca expref: -99 pid: 155796 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.384210:0:155796:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6f04b lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1bce:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec8ca expref: -99 pid: 155796 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.384216:0:155796:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6f04b lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bce:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec8ca expref: -99 pid: 155796 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826025.384227:0:155796:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c5636a00 x1874663042947328/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/672 e 0 to 0 dl 1787826041 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826025.384240:0:155796:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6f04b lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1bce:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec8ca expref: -99 pid: 155796 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.384250:0:155796:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/7118 (0 0) 00000080:00200000:3.0:1787826025.384256:0:155796:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00000002:00010000:3.0:1787826025.384334:0:155796:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200026560:0x42:0x0]) in obj [0x200026560:0x42:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826025.384379:0:155796:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6f052 lrc: 3/1,0 mode: --/CR res: [0x200026560:0x42:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 155796 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.384387:0:155796:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6f052 lrc: 3/1,0 mode: --/CR res: [0x200026560:0x42:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155796 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.384395:0:155796:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6f052 lrc: 3/1,0 mode: --/CR res: [0x200026560:0x42:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155796 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826025.384414:0:155796:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635880 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155796:1874663042947456:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:3.0:1787826025.384489:0:155796:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5cb6f00 going to sleep for 11 seconds 00000100:00100000:3.0:1787826025.392057:0:155796:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635880 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155796:1874663042947456:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:3.0:1787826025.392083:0:155796:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6f052 lrc: 4/1,0 mode: --/CR res: [0x200026560:0x42:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec8d8 expref: -99 pid: 155796 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.392097:0:155796:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6f052 lrc: 4/1,0 mode: --/PR res: [0x200026560:0x42:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec8d8 expref: -99 pid: 155796 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.392108:0:155796:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6f052 lrc: 4/1,0 mode: PR/PR res: [0x200026560:0x42:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec8d8 expref: -99 pid: 155796 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.392119:0:155796:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6f052 lrc: 4/1,0 mode: PR/PR res: [0x200026560:0x42:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec8d8 expref: -99 pid: 155796 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.392131:0:155796:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6f052 lrc: 5/2,0 mode: PR/PR res: [0x200026560:0x42:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec8d8 expref: -99 pid: 155796 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.392140:0:155796:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6f052 lrc: 5/2,0 mode: PR/PR res: [0x200026560:0x42:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec8d8 expref: -99 pid: 155796 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.392147:0:155796:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6f052 lrc: 4/1,0 mode: PR/PR res: [0x200026560:0x42:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec8d8 expref: -99 pid: 155796 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826025.392159:0:155796:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c5635880 x1874663042947456/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787826041 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826025.392173:0:155796:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6f052 lrc: 3/1,0 mode: PR/PR res: [0x200026560:0x42:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec8d8 expref: -99 pid: 155796 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.392191:0:155796:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590091616/66 (0 0) 00010000:00010000:3.0:1787826025.392197:0:155796:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6f052 lrc: 3/1,0 mode: PR/PR res: [0x200026560:0x42:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec8d8 expref: -99 pid: 155796 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.392205:0:155796:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6f052 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x42:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec8d8 expref: -99 pid: 155796 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000002:00010000:3.0:1787826025.392233:0:155796:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001f58:0x42:0x0]) in obj [0x240001f58:0x42:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826025.392278:0:155796:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481c00/0xbef42c0221d6f059 lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x42:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 155796 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.392285:0:155796:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481c00/0xbef42c0221d6f059 lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x42:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155796 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.392295:0:155796:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481c00/0xbef42c0221d6f059 lrc: 3/1,0 mode: --/CR res: [0x240001f58:0x42:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 155796 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826025.392314:0:155796:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635880 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155796:1874663042947584:192.168.203.147@tcp:101:rmdir.0 00000100:00100000:3.0:1787826025.392388:0:155796:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5cb6f00 going to sleep for 11 seconds 00000100:00100000:3.0:1787826025.407490:0:155796:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635880 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155796:1874663042947584:192.168.203.147@tcp:101:rmdir.0 00010000:00010000:3.0:1787826025.407518:0:155796:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481c00/0xbef42c0221d6f059 lrc: 4/1,0 mode: --/CR res: [0x240001f58:0x42:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec8e6 expref: -99 pid: 155796 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.407528:0:155796:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481c00/0xbef42c0221d6f059 lrc: 4/1,0 mode: --/PR res: [0x240001f58:0x42:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec8e6 expref: -99 pid: 155796 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.407541:0:155796:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481c00/0xbef42c0221d6f059 lrc: 4/1,0 mode: PR/PR res: [0x240001f58:0x42:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec8e6 expref: -99 pid: 155796 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.407563:0:155796:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481c00/0xbef42c0221d6f059 lrc: 4/1,0 mode: PR/PR res: [0x240001f58:0x42:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec8e6 expref: -99 pid: 155796 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.407575:0:155796:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481c00/0xbef42c0221d6f059 lrc: 5/2,0 mode: PR/PR res: [0x240001f58:0x42:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec8e6 expref: -99 pid: 155796 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.407583:0:155796:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481c00/0xbef42c0221d6f059 lrc: 5/2,0 mode: PR/PR res: [0x240001f58:0x42:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec8e6 expref: -99 pid: 155796 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.407590:0:155796:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481c00/0xbef42c0221d6f059 lrc: 4/1,0 mode: PR/PR res: [0x240001f58:0x42:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec8e6 expref: -99 pid: 155796 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826025.407601:0:155796:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c5635880 x1874663042947584/t0(0) o101->lustre-MDT0001-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787826041 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826025.407614:0:155796:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481c00/0xbef42c0221d6f059 lrc: 3/1,0 mode: PR/PR res: [0x240001f58:0x42:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec8e6 expref: -99 pid: 155796 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.407626:0:155796:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d0f54800 type 13 mode 4 res 9663684440/66 (0 0) 00010000:00010000:3.0:1787826025.407635:0:155796:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481c00/0xbef42c0221d6f059 lrc: 3/1,0 mode: PR/PR res: [0x240001f58:0x42:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec8e6 expref: -99 pid: 155796 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.407643:0:155796:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481c00/0xbef42c0221d6f059 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x42:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec8e6 expref: -99 pid: 155796 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826025.407674:0:155796:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1bce:0x0], updating i_size 8192 00000080:00200000:3.0:1787826025.407679:0:155796:0:(namei.c:147:ll_iget()) got inode: [0x20001a9e3:0x1bce:0x0](ffff9181c83f9148): rc = 0 00000080:00010000:3.0:1787826025.407688:0:155796:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x1bce:0x0](ffff9181c83f9148) for lock 0xbef42c0221d6f04b 00000080:00010000:3.0:1787826025.407705:0:155796:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x1bce:0x0](ffff9181c83f9148) 00000080:00010000:3.0:1787826025.407709:0:155796:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f04b from it ffffb975015ebe08 00010000:00010000:3.0:1787826025.407713:0:155796:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6f04b lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1bce:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec8ca expref: -99 pid: 155796 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.407721:0:155796:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6f04b lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1bce:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec8ca expref: -99 pid: 155796 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826025.407768:0:155796:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 3 00000080:00200000:3.0:1787826025.407786:0:155796:0:(namei.c:2328:ll_rmdir()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00010000:00010000:3.0:1787826025.407814:0:155796:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6f04b lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1bce:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00cec8ca expref: -99 pid: 155796 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826025.407858:0:155796:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6f04b lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x1bce:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cec8ca expref: -99 pid: 155796 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826025.407864:0:155796:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826025.407867:0:155796:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481e00/0xbef42c0221d6f04b lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x1bce:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cec8ca expref: -99 pid: 155796 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:3.0:1787826025.407889:0:155796:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:3.0:1787826025.407896:0:155796:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c5636a00 x1874663042947840/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826025.407916:0:155796:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155796:1874663042947840:192.168.203.147@tcp:36:rmdir.0 00000100:00100000:3.0:1787826025.407992:0:155796:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5cb6f00 going to sleep for 11 seconds 00000100:00100000:0.0:1787826025.414194:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826025.414242:0:15151:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:0.0:1787826025.414259:0:15151:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049206144 00000100:00100000:0.0:1787826025.414267:0:15151:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049206144 00000100:00100000:0.0:1787826025.414317:0:15151:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9265 00000100:00100000:0.0:1787826025.414327:0:15151:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181e053c050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187905:x1874663049206144:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826025.414346:0:15151:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6f052 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x42:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cec8d8 expref: -99 pid: 155796 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826025.414438:0:15151:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826025.414446:0:15151:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181e053c050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:187905:x1874663049206144:12345-192.168.203.147@tcp:104: Request processed in 120us (262us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826025.414453:0:15151:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9265 00010000:00010000:3.0:1787826025.422398:0:33350:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826025.422412:0:33350:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6f052 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x42:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cec8d8 expref: -99 pid: 155796 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826025.422426:0:33350:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3480800 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:3.0:1787826025.422431:0:33350:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6f052 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x42:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00cec8d8 expref: -99 pid: 155796 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826025.422440:0:33350:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6f052 lrc: 2/0,0 mode: PR/PR res: [0x200026560:0x42:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00cec8d8 expref: -99 pid: 155796 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826025.422450:0:33350:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6f052 lrc: 3/0,0 mode: PR/PR res: [0x200026560:0x42:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x408400000000 nid: local remote: 0x878a5eea00cec8d8 expref: -99 pid: 155796 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826025.422520:0:33350:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6f052 lrc: 2/0,0 mode: --/PR res: [0x200026560:0x42:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cec8d8 expref: -99 pid: 155796 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826025.422526:0:33350:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826025.422553:0:33350:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6f052 lrc: 1/0,0 mode: --/PR res: [0x200026560:0x42:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cec8d8 expref: -99 pid: 155796 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826025.422561:0:33350:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6f052 lrc: 0/0,0 mode: --/PR res: [0x200026560:0x42:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cec8d8 expref: -99 pid: 155796 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826025.422585:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826025.422587:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826025.422589:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826025.422624:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2392:1874663042947968:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1787826025.426879:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2392:1874663042947968:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787826025.428864:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826025.429155:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049206400 00000100:00100000:1.0:1787826025.429164:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049206400 00000100:00100000:1.0:1787826025.429215:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9266 00000100:00100000:1.0:1787826025.429226:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663049206400:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826025.429244:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481c00/0xbef42c0221d6f059 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x42:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cec8e6 expref: -99 pid: 155796 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826025.429323:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826025.429330:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:a9ce6693-0634-46c5-bf21-5ca7d026d44e+5:8415:x1874663049206400:12345-192.168.203.147@tcp:104: Request processed in 106us (474us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826025.429337:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9266 00010000:00010000:3.0:1787826025.430203:0:15161:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826025.430213:0:15161:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481c00/0xbef42c0221d6f059 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x42:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cec8e6 expref: -99 pid: 155796 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826025.430224:0:15161:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3481c00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:3.0:1787826025.430229:0:15161:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481c00/0xbef42c0221d6f059 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x42:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00cec8e6 expref: -99 pid: 155796 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826025.430238:0:15161:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481c00/0xbef42c0221d6f059 lrc: 2/0,0 mode: PR/PR res: [0x240001f58:0x42:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00cec8e6 expref: -99 pid: 155796 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826025.430247:0:15161:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481c00/0xbef42c0221d6f059 lrc: 3/0,0 mode: PR/PR res: [0x240001f58:0x42:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x408400000000 nid: local remote: 0x878a5eea00cec8e6 expref: -99 pid: 155796 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826025.430304:0:15161:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481c00/0xbef42c0221d6f059 lrc: 2/0,0 mode: --/PR res: [0x240001f58:0x42:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cec8e6 expref: -99 pid: 155796 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826025.430311:0:15161:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826025.430347:0:15161:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481c00/0xbef42c0221d6f059 lrc: 1/0,0 mode: --/PR res: [0x240001f58:0x42:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cec8e6 expref: -99 pid: 155796 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826025.430354:0:15161:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c4d2b000 lock: ffff9181c3481c00/0xbef42c0221d6f059 lrc: 0/0,0 mode: --/PR res: [0x240001f58:0x42:0x0].0x0 bits 0x12/0x12 rrc: 2 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cec8e6 expref: -99 pid: 155796 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826025.430371:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826025.430373:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826025.430375:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826025.430382:0:2392:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787826025.430396:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2392:1874663042948096:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1787826025.441202:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2392:1874663042948096:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1787826025.455127:0:155796:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job rmdir:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155796:1874663042947840:192.168.203.147@tcp:36:rmdir.0 00000080:00200000:3.0:1787826025.455174:0:155796:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1bce:0x0](ffff9181c83f9148) 00000080:00200008:1.0:1787826025.470135:0:155790:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1787826025.470150:0:155790:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826025.470174:0:155790:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826025.470194:0:155790:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826025.470211:0:155790:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826025.470220:0:155790:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826025.470228:0:155790:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826025.470231:0:155790:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787826025.470239:0:155790:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb9750149bcb8 00010000:00010000:1.0:1787826025.470243:0:155790:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826025.470252:0:155790:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200008:1.0:1787826025.470857:0:155790:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), request_mask 0, flags 0, foreign 0 00000080:00200000:1.0:1787826025.470864:0:155790:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826025.470874:0:155790:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826025.470883:0:155790:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826025.470894:0:155790:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826025.470901:0:155790:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826025.470906:0:155790:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826025.470908:0:155790:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787826025.470913:0:155790:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb9750149bcb8 00010000:00010000:1.0:1787826025.470915:0:155790:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826025.470927:0:155790:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826025.471025:0:155790:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826025.471034:0:155790:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826025.471042:0:155790:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826025.471052:0:155790:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826025.471058:0:155790:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826025.471062:0:155790:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826025.471065:0:155790:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:1.0:1787826025.471069:0:155790:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb9750149bbb0 00010000:00010000:1.0:1787826025.471072:0:155790:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:1.0:1787826025.471080:0:155790:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:1.0:1787826025.471089:0:155790:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 44 00000080:00200008:1.0:1787826025.471095:0:155790:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), flags 100000 00000080:00200080:1.0:1787826025.471171:0:155790:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe0fcc2b20 00800000:00000080:1.0:1787826025.471186:0:155790:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053fc00 uarg=0000000000000000 00800000:00000004:1.0:1787826025.471192:0:155790:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9181c4d2b000: age 8084, max_age 8096 00000100:00100000:1.0:1787826025.471235:0:155790:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155790:1874663042948224:192.168.203.147@tcp:41:lfs.0 00000100:00100000:1.0:1787826025.471452:0:155790:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b59480 going to sleep for 11 seconds 00000080:00200000:3.0:1787826025.477101:0:155806:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:3.0:1787826025.477129:0:155806:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826025.477145:0:155806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826025.477160:0:155806:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826025.477168:0:155806:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826025.477176:0:155806:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:3.0:1787826025.477179:0:155806:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:3.0:1787826025.477187:0:155806:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb9750181bb58 00010000:00010000:3.0:1787826025.477190:0:155806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826025.477200:0:155806:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826025.477213:0:155806:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826025.477220:0:155806:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=29 00000080:00200000:3.0:1787826025.477226:0:155806:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826025.477236:0:155806:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=13 00000080:00200000:3.0:1787826025.477240:0:155806:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:3.0:1787826025.477251:0:155806:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:3.0:1787826025.477288:0:155806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481c00/0xbef42c0221d6f060 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.477296:0:155806:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481c00/0xbef42c0221d6f060 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.477304:0:155806:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481c00/0xbef42c0221d6f060 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826025.477324:0:155806:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635880 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155806:1874663042948480:192.168.203.147@tcp:101:lfs.0 00000100:00100000:3.0:1787826025.477394:0:155806:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5cb6f00 going to sleep for 11 seconds 00000100:00100000:0.0:1787826025.483203:0:155806:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635880 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155806:1874663042948480:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787826025.483218:0:155806:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481c00/0xbef42c0221d6f060 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826025.483229:0:155806:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481c00/0xbef42c0221d6f060 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826025.483243:0:155806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481c00/0xbef42c0221d6f060 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826025.483251:0:155806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481c00/0xbef42c0221d6f060 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826025.483260:0:155806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481c00/0xbef42c0221d6f060 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826025.483266:0:155806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3481c00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787826025.483270:0:155806:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481c00/0xbef42c0221d6f060 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826025.483277:0:155806:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481c00/0xbef42c0221d6f060 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826025.483284:0:155806:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481c00/0xbef42c0221d6f060 lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 155806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826025.483292:0:155806:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787826025.483297:0:155806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481c00/0xbef42c0221d6f060 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826025.483304:0:155806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481c00/0xbef42c0221d6f060 lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155806 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826025.483322:0:155806:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c5635880 x1874663042948480/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787826041 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787826025.483336:0:155806:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787826025.483360:0:155806:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:0.0:1787826025.483423:0:155806:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826025.483438:0:155806:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826025.483446:0:155806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826025.483459:0:155806:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826025.483466:0:155806:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826025.483470:0:155806:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826025.483473:0:155806:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:0.0:1787826025.483478:0:155806:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb9750181bbe8 00010000:00010000:0.0:1787826025.483481:0:155806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826025.483488:0:155806:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:0.0:1787826025.483500:0:155806:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826025.483507:0:155806:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=16408 00000080:00200000:0.0:1787826025.483512:0:155806:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826025.483523:0:155806:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags=16392 00000080:00200000:0.0:1787826025.483528:0:155806:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=test, dir=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), intent=getattr 00000002:00010000:0.0:1787826025.483538:0:155806:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: test,[0x0:0x0:0x0]) in obj [0x20001a9e3:0x17e0:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787826025.483573:0:155806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6f067 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 155806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826025.483581:0:155806:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6f067 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826025.483591:0:155806:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6f067 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826025.483609:0:155806:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155806:1874663042948608:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787826025.483823:0:155806:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c7749300 going to sleep for 11 seconds 00000100:00100000:3.0:1787826025.487557:0:155806:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155806:1874663042948608:192.168.203.147@tcp:101:lfs.0 00010000:00010000:3.0:1787826025.487573:0:155806:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6f067 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 155806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.487585:0:155806:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6f067 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.487593:0:155806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6f067 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.487600:0:155806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6f067 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 155806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.487610:0:155806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6f067 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 155806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.487616:0:155806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0512800 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:3.0:1787826025.487621:0:155806:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6f067 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.487628:0:155806:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6f067 lrc: 4/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x206400000000 pid: 155806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.487637:0:155806:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6f067 lrc: 5/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x20e400000000 pid: 155806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.487643:0:155806:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826025.487650:0:155806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6f067 lrc: 3/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.487656:0:155806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512800/0xbef42c0221d6f067 lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x2 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 155806 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826025.487674:0:155806:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c55a5180 x1874663042948608/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/584 e 0 to 0 dl 1787826041 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:3.0:1787826025.487689:0:155806:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x17e0:0x0] suppgids 0 -1: rc 0 00010000:00010000:3.0:1787826025.487724:0:155806:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00000080:00200000:3.0:1787826025.488070:0:155806:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:3.0:1787826025.488088:0:155806:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826025.488097:0:155806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826025.488108:0:155806:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826025.488115:0:155806:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826025.488121:0:155806:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:3.0:1787826025.488124:0:155806:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:3.0:1787826025.488130:0:155806:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb9750181bb58 00010000:00010000:3.0:1787826025.488133:0:155806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826025.488143:0:155806:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826025.488155:0:155806:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826025.488161:0:155806:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=13 00000080:00200008:3.0:1787826025.488170:0:155806:0:(file.c:6296:ll_getattr_dentry()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), request_mask 0, flags 0, foreign 0 00000080:00200000:3.0:1787826025.488174:0:155806:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08),name=d71a.replay-single 00000002:00010000:3.0:1787826025.488188:0:155806:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x20001a9e3:0x17e0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826025.488195:0:155806:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/6112 (0 0) 00010000:00010000:3.0:1787826025.488223:0:155806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6f06e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 155806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.488230:0:155806:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6f06e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 155806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.488239:0:155806:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6f06e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 155806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826025.488253:0:155806:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155806:1874663042948992:192.168.203.147@tcp:101:lfs.0 00000100:00100000:3.0:1787826025.488298:0:155806:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5cb6c00 going to sleep for 11 seconds 00000100:00100000:3.0:1787826025.492435:0:155806:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155806:1874663042948992:192.168.203.147@tcp:101:lfs.0 00010000:00010000:3.0:1787826025.492456:0:155806:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6f06e lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec948 expref: -99 pid: 155806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.492471:0:155806:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6f06e lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec948 expref: -99 pid: 155806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.492483:0:155806:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6f06e lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec948 expref: -99 pid: 155806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.492495:0:155806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6f06e lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec948 expref: -99 pid: 155806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.492505:0:155806:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6f06e lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec948 expref: -99 pid: 155806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.492516:0:155806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6f06e lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec948 expref: -99 pid: 155806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.492524:0:155806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6f06e lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec948 expref: -99 pid: 155806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.492531:0:155806:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6f06e lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec948 expref: -99 pid: 155806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826025.492544:0:155806:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c5636d80 x1874663042948992/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826041 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826025.492558:0:155806:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6f06e lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec948 expref: -99 pid: 155806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.492568:0:155806:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/6112 (0 0) 00000080:00200000:3.0:1787826025.492574:0:155806:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:3.0:1787826025.492599:0:155806:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x17e0:0x0], updating i_size 4096 00000080:00010000:3.0:1787826025.492605:0:155806:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) for lock 0xbef42c0221d6f06e 00000080:00010000:3.0:1787826025.492611:0:155806:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00000080:00010000:3.0:1787826025.492614:0:155806:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f06e from it ffffb9750181bcb8 00010000:00010000:3.0:1787826025.492626:0:155806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6f06e lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec948 expref: -99 pid: 155806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826025.492636:0:155806:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6f06e lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec948 expref: -99 pid: 155806 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:3.0:1787826025.493542:0:155806:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), request_mask 0, flags 0, foreign 0 00000080:00200000:3.0:1787826025.493550:0:155806:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:3.0:1787826025.493567:0:155806:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826025.493578:0:155806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826025.493590:0:155806:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826025.493597:0:155806:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826025.493602:0:155806:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:3.0:1787826025.493605:0:155806:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:3.0:1787826025.493610:0:155806:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb9750181bcb8 00010000:00010000:3.0:1787826025.493613:0:155806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826025.493684:0:155806:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826025.494125:0:155806:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:3.0:1787826025.494135:0:155806:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826025.494144:0:155806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826025.494153:0:155806:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826025.494160:0:155806:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826025.494165:0:155806:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:3.0:1787826025.494167:0:155806:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6d18f 00000080:00010000:3.0:1787826025.494171:0:155806:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6d18f from it ffffb9750181bbf8 00010000:00010000:3.0:1787826025.494174:0:155806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826025.494181:0:155806:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512200/0xbef42c0221d6d18f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9626 expref: -99 pid: 153969 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000080:00200000:3.0:1787826025.494190:0:155806:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826025.494196:0:155806:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d71a.replay-single, flags=267 00000080:00200000:3.0:1787826025.494202:0:155806:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), inode mode 41ed mask 44 00000080:00200008:3.0:1787826025.494208:0:155806:0:(file.c:1090:ll_file_open()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00000080:00200080:3.0:1787826025.494230:0:155806:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) cmd=c03066f0 arg=7ffd16d66df0 00000080:00200000:3.0:1787826025.494242:0:155806:0:(dir.c:451:ll_dir_setdirstripe()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) name=test stripe_offset=0 stripe_count=2, hash_type=0 00010000:00010000:3.0:1787826025.494267:0:155806:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6f06e lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00cec948 expref: -99 pid: 155806 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826025.494286:0:155806:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6800/0xbef42c0221d64b3f lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9634 expref: -99 pid: 152202 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:3.0:1787826025.494331:0:155806:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6f06e lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cec948 expref: -99 pid: 155806 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826025.494338:0:155806:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826025.494341:0:155806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3480800/0xbef42c0221d6f06e lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cec948 expref: -99 pid: 155806 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:3.0:1787826025.494361:0:155806:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:3.0:1787826025.494371:0:155806:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c5636d80 x1874663042949248/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 552/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826025.494393:0:155806:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155806:1874663042949248:192.168.203.147@tcp:36:lfs.0 00000100:00100000:3.0:1787826025.494476:0:155806:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5cb6c00 going to sleep for 11 seconds 00000100:00100000:3.0:1787826025.551107:0:155806:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:155806:1874663042949248:192.168.203.147@tcp:36:lfs.0 00000080:00200000:3.0:1787826025.551154:0:155806:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x20001a9e3:0x1bcf:0x0](ffff9181d080aa08) 00000080:00200000:3.0:1787826025.551162:0:155806:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1bcf:0x0], updating i_size 4096 00000080:00200008:3.0:1787826025.551192:0:155806:0:(file.c:479:ll_file_release()) START file d71a.replay-single:[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08), flags 300000 00010000:00010000:3.0:1787826025.551220:0:155806:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513000/0xbef42c0221d64bee lrc: 2/0,0 mode: CR/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce962d expref: -99 pid: 152210 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:0.0:1787826026.337932:0:33329:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826027.363396:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826028.385843:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826029.089166:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7528148b-9bee-4d31-b597-8aae3b0b4982:2392:1874663042949376:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826029.089268:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2392:1874663042949632:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826029.089298:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2392:1874663042949888:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826029.089326:0:2392:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1787826029.089333:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2392:1874663042949504:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826029.089354:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2392:1874663042949760:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826029.103112:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2392:1874663042949504:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826029.103147:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2392:1874663042949632:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826029.120467:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2392:1874663042949760:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826029.120484:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7528148b-9bee-4d31-b597-8aae3b0b4982:2392:1874663042949376:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826029.120495:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2392:1874663042949888:192.168.203.147@tcp:400:kworker.0 00010000:00010000:3.0:1787826029.409263:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826030.442796:0:33329:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826032.481367:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826033.504901:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:0.0:1787826033.870982:0:156383:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == replay-single test 73a: open(O_CREAT), unlink, replay, reconnect before open replay, close ========================================================== 06:20:32 (1787826032) 00000100:00100000:0.0:1787826034.215118:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042950144:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826034.215661:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042950400:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826034.215706:0:2390:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:0.0:1787826034.215714:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7528148b-9bee-4d31-b597-8aae3b0b4982:2390:1874663042950016:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826034.216171:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042950272:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826034.216209:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042950528:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826034.228426:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7528148b-9bee-4d31-b597-8aae3b0b4982:2390:1874663042950016:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826034.228458:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042950272:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826034.228472:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042950528:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826034.228483:0:2390:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9181c4d2b000: committing for last_committed 335007449218 gen 11 00000100:00100000:0.0:1787826034.228720:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042950144:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826034.228739:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042950400:192.168.203.147@tcp:400:kworker.0 00000080:00200000:0.0:1787826034.266414:0:156398:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826034.266451:0:156398:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826034.266527:0:156398:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6f07c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 156398 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826034.266538:0:156398:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6f07c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 156398 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826034.266551:0:156398:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513a00/0xbef42c0221d6f07c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 156398 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826034.267099:0:156398:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job rm:a9ce6693-0634-46c5-bf21-5ca7d026d44e:156398:1874663042950912:192.168.203.147@tcp:101:rm.0 00000100:00100000:0.0:1787826034.267255:0:156398:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d800ca80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826034.273197:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2391:1874663042951040:192.168.203.147@tcp:101:rm.0 00010000:00010000:3.0:1787826034.289336:0:2391:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff9181c4d2b000 lock: 00000000a58f462c/0xbef42c0221d6f083 lrc: 6/2,0 mode: --/PR res: [0x280000401:0x1842:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 156398 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826034.289372:0:2391:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0000-osc-ffff9181c4d2b000 lock: 00000000a58f462c/0xbef42c0221d6f083 lrc: 6/2,0 mode: --/PR res: [0x280000401:0x1842:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec9aa expref: -99 pid: 156398 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826034.289390:0:2391:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff9181c4d2b000 lock: 00000000a58f462c/0xbef42c0221d6f083 lrc: 6/2,0 mode: PR/PR res: [0x280000401:0x1842:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec9aa expref: -99 pid: 156398 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826034.289411:0:2391:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0000-osc-ffff9181c4d2b000 lock: 00000000a58f462c/0xbef42c0221d6f083 lrc: 6/2,0 mode: PR/PR res: [0x280000401:0x1842:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec9aa expref: -99 pid: 156398 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:3.0:1787826034.289428:0:2391:0:(osc_lock.c:187:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0000-osc-ffff9181c4d2b000 lock: 00000000a58f462c/0xbef42c0221d6f083 lrc: 5/2,0 mode: PR/PR res: [0x280000401:0x1842:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00cec9aa expref: -99 pid: 156398 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:3.0:1787826034.289441:0:2391:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00010000:00010000:3.0:1787826034.289461:0:2391:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9181c4d2b000 lock: 00000000a58f462c/0xbef42c0221d6f083 lrc: 6/2,0 mode: PR/PR res: [0x280000401:0x1842:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x878a5eea00cec9aa expref: -99 pid: 156398 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826034.289471:0:2391:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0000-osc-ffff9181c4d2b000 lock: 00000000a58f462c/0xbef42c0221d6f083 lrc: 5/1,0 mode: PR/PR res: [0x280000401:0x1842:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x878a5eea00cec9aa expref: -99 pid: 156398 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826034.289480:0:2391:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9181c4d2b000 lock: 00000000a58f462c/0xbef42c0221d6f083 lrc: 5/1,0 mode: PR/PR res: [0x280000401:0x1842:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x878a5eea00cec9aa expref: -99 pid: 156398 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826034.289490:0:2391:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff9181c4d2b000 lock: 00000000a58f462c/0xbef42c0221d6f083 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x1842:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x878a5eea00cec9aa expref: -99 pid: 156398 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000100:00100000:3.0:1787826034.289517:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2391:1874663042951040:192.168.203.147@tcp:101:rm.0 00000100:00100000:3.0:1787826035.304225:0:156407:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f3480 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:156407:1874663042951552:192.168.203.147@tcp:41:lfs.0 00800000:00000004:3.0:1787826035.304265:0:156407:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0000-mdc-ffff9181c4d2b000: update ffff9181c87b0450 cache blocks 317053/353529 objects 1023645/1024000 00000080:00200080:3.0:1787826035.304365:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.304391:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00800000:00000004:3.0:1787826035.304400:0:156407:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9181c4d2b000: age 8097, max_age 8106 00000100:00100000:3.0:1787826035.304443:0:156407:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:156407:1874663042951680:192.168.203.147@tcp:41:lfs.0 00000100:00100000:3.0:1787826035.304535:0:156407:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c7da1780 going to sleep for 11 seconds 00000100:00100000:3.0:1787826035.307858:0:156407:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:156407:1874663042951680:192.168.203.147@tcp:41:lfs.0 00800000:00000004:3.0:1787826035.307894:0:156407:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0001-mdc-ffff9181c4d2b000: update ffff9181c87b1e08 cache blocks 317194/353529 objects 1023702/1024000 00000080:00200080:3.0:1787826035.307990:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308010:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308018:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308023:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308027:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308031:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308035:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308038:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308042:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308046:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308050:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308054:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308058:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308061:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308066:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308069:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308073:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308076:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308082:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308085:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308089:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308093:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308097:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308100:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308104:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308107:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308111:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308115:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308119:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308122:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308126:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308130:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308146:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308150:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308154:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308157:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308161:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308165:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308168:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308171:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308175:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308178:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308182:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308186:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308189:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308193:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308197:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308200:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308204:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308207:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308211:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308214:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308218:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308221:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308224:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308227:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308230:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308233:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308236:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308239:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308242:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308245:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308248:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308251:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308254:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308256:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308260:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308262:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308266:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308269:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308272:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308275:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308278:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308281:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308284:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308286:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308289:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308292:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308296:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308299:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308303:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308307:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308311:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308314:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308317:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308321:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308324:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308328:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308331:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308335:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308339:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308342:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308346:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308351:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308355:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308359:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308363:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308366:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308371:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308374:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308378:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308381:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308385:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308388:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308392:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308395:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308398:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308401:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308405:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308408:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308412:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308416:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308420:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308423:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308428:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308432:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308435:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308438:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308443:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308447:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308450:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308454:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308459:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308462:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308467:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00800000:00000080:3.0:1787826035.308471:0:156407:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.308476:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.308484:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00020000:00000004:3.0:1787826035.308494:0:156407:0:(obd_class.h:1090:obd_statfs()) lustre-OST0000-osc-ffff9181c4d2b000: age 8097, max_age 8106 00000100:00100000:3.0:1787826035.308538:0:156407:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:156407:1874663042951808:192.168.203.147@tcp:13:lfs.0 00000100:00100000:3.0:1787826035.308644:0:156407:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c7da1780 going to sleep for 11 seconds 00000100:00100000:3.0:1787826035.312360:0:156407:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:156407:1874663042951808:192.168.203.147@tcp:13:lfs.0 00020000:00000004:3.0:1787826035.312398:0:156407:0:(obd_class.h:1116:obd_statfs()) lustre-OST0000-osc-ffff9181c4d2b000: update ffff9181c87b37c0 cache blocks 901356/958279 objects 261710/262144 00000080:00200080:3.0:1787826035.312482:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.312503:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00020000:00000004:3.0:1787826035.312512:0:156407:0:(obd_class.h:1090:obd_statfs()) lustre-OST0001-osc-ffff9181c4d2b000: age 8097, max_age 8106 00000100:00100000:3.0:1787826035.312809:0:156407:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:156407:1874663042951936:192.168.203.147@tcp:13:lfs.0 00000100:00100000:3.0:1787826035.312999:0:156407:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c7da1780 going to sleep for 11 seconds 00000100:00100000:3.0:1787826035.319042:0:156407:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:156407:1874663042951936:192.168.203.147@tcp:13:lfs.0 00020000:00000004:3.0:1787826035.319080:0:156407:0:(obd_class.h:1116:obd_statfs()) lustre-OST0001-osc-ffff9181c4d2b000: update ffff9181c87b5178 cache blocks 901360/958279 objects 261727/262144 00000080:00200080:3.0:1787826035.319286:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.319309:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.319318:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.319323:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.319327:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.319331:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.319335:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.319339:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.319343:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.319346:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.319350:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.319353:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.319357:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.319361:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.319365:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.319368:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.319371:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.319374:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.319378:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.319381:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.319385:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.319389:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.319393:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.319397:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.319402:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.319405:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.319409:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.319414:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.319418:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.319422:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.319425:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.319429:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.319433:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.319436:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.319441:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.319444:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.319448:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.319451:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.319455:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.319458:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.319463:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.319466:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.319470:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.319473:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.319477:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.319480:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.319483:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.319487:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.319490:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.319494:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.319498:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.319501:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.319505:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.319509:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.319513:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.319517:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.319521:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.319524:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.319528:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.319532:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.319536:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.319539:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.319543:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.319546:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.319549:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.319670:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.319679:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.319683:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.319688:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.319691:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.320081:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.320090:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.320095:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.320099:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.320103:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.320107:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.320112:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.320116:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.320120:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.320124:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.320128:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.320223:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.320232:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.320237:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.320241:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.320245:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.320249:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.320252:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.320256:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.320259:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.320263:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.320266:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.320269:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.320272:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.320276:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.320279:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.320283:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.320286:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.320290:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.320293:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.320297:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.320301:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.320305:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.320309:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.320312:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.320316:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.320320:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.320323:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.320326:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.320329:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.320332:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.320335:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.320338:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.320341:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.320344:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.320348:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.320352:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.320355:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.320359:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.320362:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.320366:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.320369:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.320374:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.320378:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200080:3.0:1787826035.320382:0:156407:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffc4fd98030 00020000:00000080:3.0:1787826035.320385:0:156407:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442f800 uarg=0000000000000000 00000080:00200008:3.0:1787826035.320402:0:156407:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), flags 100000 00000080:00200000:0.0:1787826035.506411:0:156417:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826035.506453:0:156417:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826035.506543:0:156417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6f091 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 156417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826035.507705:0:156417:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6f091 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 156417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826035.507725:0:156417:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6f091 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 156417 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826035.508100:0:156417:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4a80 pname:cluuid:pid:xid:nid:opc:job rm:a9ce6693-0634-46c5-bf21-5ca7d026d44e:156417:1874663042952320:192.168.203.147@tcp:101:rm.0 00000100:00100000:0.0:1787826035.508224:0:156417:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d800c000 going to sleep for 11 seconds 00000100:00100000:1.0:1787826035.511487:0:156417:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4a80 pname:cluuid:pid:xid:nid:opc:job rm:a9ce6693-0634-46c5-bf21-5ca7d026d44e:156417:1874663042952320:192.168.203.147@tcp:101:rm.0 00010000:00010000:1.0:1787826035.511522:0:156417:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6f091 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec9e9 expref: -99 pid: 156417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826035.511537:0:156417:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6f091 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec9e9 expref: -99 pid: 156417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826035.511560:0:156417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6f091 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec9e9 expref: -99 pid: 156417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826035.511572:0:156417:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6f091 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec9e9 expref: -99 pid: 156417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826035.511587:0:156417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6f091 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec9e9 expref: -99 pid: 156417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826035.511594:0:156417:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6f091 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec9e9 expref: -99 pid: 156417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826035.511602:0:156417:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6f091 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec9e9 expref: -99 pid: 156417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826035.511615:0:156417:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c55a4a80 x1874663042952320/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826051 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826035.511637:0:156417:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6f091 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec9e9 expref: -99 pid: 156417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826035.511653:0:156417:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:1.0:1787826035.511660:0:156417:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787826035.511707:0:156417:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 69632 00000080:00010000:1.0:1787826035.511714:0:156417:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6f091 00000080:00010000:1.0:1787826035.511721:0:156417:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826035.511725:0:156417:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f091 from it ffffb97501bd7b58 00010000:00010000:1.0:1787826035.511728:0:156417:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6f091 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec9e9 expref: -99 pid: 156417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826035.511737:0:156417:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6f091 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec9e9 expref: -99 pid: 156417 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826035.511795:0:156417:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826035.511820:0:156417:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=fsa-oleg347-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9181c41acb08), flags=8 00000080:00200000:1.0:1787826035.511826:0:156417:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=fsa-oleg347-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9181c41acb08), intent=getattr 00000002:00010000:1.0:1787826035.511854:0:156417:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: fsa-oleg347-client.virtnet,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 01 00010000:00010000:1.0:1787826035.511905:0:156417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6f098 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 156417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826035.511914:0:156417:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6f098 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 156417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826035.511924:0:156417:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6f098 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 156417 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826035.511961:0:156417:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job rm:a9ce6693-0634-46c5-bf21-5ca7d026d44e:156417:1874663042952448:192.168.203.147@tcp:101:rm.0 00000100:00100000:1.0:1787826035.512060:0:156417:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b593c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826035.516376:0:156417:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job rm:a9ce6693-0634-46c5-bf21-5ca7d026d44e:156417:1874663042952448:192.168.203.147@tcp:101:rm.0 00010000:00010000:1.0:1787826035.516411:0:156417:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6f098 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceca05 expref: -99 pid: 156417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826035.516437:0:156417:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6f098 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1bd1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceca05 expref: -99 pid: 156417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826035.516446:0:156417:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6f098 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1bd1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceca05 expref: -99 pid: 156417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826035.516458:0:156417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6f098 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bd1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceca05 expref: -99 pid: 156417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826035.516470:0:156417:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6f098 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bd1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceca05 expref: -99 pid: 156417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826035.516484:0:156417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6f098 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1bd1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceca05 expref: -99 pid: 156417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826035.516492:0:156417:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6f098 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1bd1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceca05 expref: -99 pid: 156417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826035.516501:0:156417:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6f098 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bd1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceca05 expref: -99 pid: 156417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826035.516519:0:156417:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c35b3480 x1874663042952448/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 600/584 e 0 to 0 dl 1787826051 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826035.516534:0:156417:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6f098 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1bd1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceca05 expref: -99 pid: 156417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826035.516545:0:156417:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/7121 (0 0) 00000080:00200000:1.0:1787826035.516566:0:156417:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1787826035.516592:0:156417:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1bd1:0x0], updating i_size 0 00000080:00200000:1.0:1787826035.516596:0:156417:0:(namei.c:147:ll_iget()) got inode: [0x20001a9e3:0x1bd1:0x0](ffff9181d08021c8): rc = 0 00000080:00200000:1.0:1787826035.516614:0:156417:0:(file.c:6843:ll_layout_conf()) [0x20001a9e3:0x1bd1:0x0]: layout version change: 4294967294 -> 4294967295 00000080:00010000:1.0:1787826035.516621:0:156417:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x1bd1:0x0](ffff9181d08021c8) for lock 0xbef42c0221d6f098 00000080:00010000:1.0:1787826035.516634:0:156417:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x1bd1:0x0](ffff9181d08021c8) 00000080:00010000:1.0:1787826035.516638:0:156417:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f098 from it ffffb97501bd7b20 00010000:00010000:1.0:1787826035.516641:0:156417:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6f098 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1bd1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00ceca05 expref: -99 pid: 156417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826035.516650:0:156417:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6f098 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1bd1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00ceca05 expref: -99 pid: 156417 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:1.0:1787826035.516695:0:156417:0:(file.c:6296:ll_getattr_dentry()) START file fsa-oleg347-client.virtnet:[0x20001a9e3:0x1bd1:0x0](ffff9181d08021c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1787826035.516703:0:156417:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x20001a9e3:0x1bd1:0x0](ffff9181d08021c8),name=fsa-oleg347-client.virtnet 00000002:00010000:1.0:1787826035.516728:0:156417:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x20001a9e3:0x1bd1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826035.516738:0:156417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6f098 lrc: 2/1,0 mode: PR/PR res: [0x20001a9e3:0x1bd1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00ceca05 expref: -99 pid: 156417 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826035.516794:0:156417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6f098 lrc: 2/1,0 mode: PR/PR res: [0x20001a9e3:0x1bd1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00ceca05 expref: -99 pid: 156417 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826035.516801:0:156417:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826035.516806:0:156417:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x1bd1:0x0](ffff9181d08021c8) 00000080:00010000:1.0:1787826035.516809:0:156417:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x1bd1:0x0](ffff9181d08021c8) for lock 0xbef42c0221d6f098 00000080:00010000:1.0:1787826035.516813:0:156417:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f098 from it ffffb97501bd7cb8 00010000:00010000:1.0:1787826035.516816:0:156417:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6f098 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1bd1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00ceca05 expref: -99 pid: 156417 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826035.516824:0:156417:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6f098 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1bd1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00ceca05 expref: -99 pid: 156417 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826035.516892:0:156417:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826035.516900:0:156417:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826035.516908:0:156417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6f091 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec9e9 expref: -99 pid: 156417 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826035.516915:0:156417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6f091 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec9e9 expref: -99 pid: 156417 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826035.516921:0:156417:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826035.516925:0:156417:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826035.516937:0:156417:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6f091 00000080:00010000:1.0:1787826035.516941:0:156417:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f091 from it ffffb97501bd7c10 00010000:00010000:1.0:1787826035.516945:0:156417:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6f091 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec9e9 expref: -99 pid: 156417 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826035.516953:0:156417:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6f091 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec9e9 expref: -99 pid: 156417 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826035.516961:0:156417:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826035.516970:0:156417:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=fsa-oleg347-client.virtnet, flags=0 00000080:00200000:1.0:1787826035.516976:0:156417:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826035.517337:0:156417:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826035.517347:0:156417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6f091 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec9e9 expref: -99 pid: 156417 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826035.517356:0:156417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6f091 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec9e9 expref: -99 pid: 156417 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826035.517363:0:156417:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826035.517367:0:156417:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:1.0:1787826035.517370:0:156417:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6f091 00000080:00010000:1.0:1787826035.517374:0:156417:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f091 from it ffffb97501bd7d68 00010000:00010000:1.0:1787826035.517377:0:156417:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6f091 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cec9e9 expref: -99 pid: 156417 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826035.517385:0:156417:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6f091 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cec9e9 expref: -99 pid: 156417 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787826035.517394:0:156417:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 3 00000080:00200000:1.0:1787826035.517400:0:156417:0:(namei.c:2418:ll_unlink()) VFS Op:name=fsa-oleg347-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9181c41acb08) 00010000:00010000:1.0:1787826035.517426:0:156417:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6f091 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00cec9e9 expref: -99 pid: 156417 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787826035.517453:0:156417:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6f098 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1bd1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x878a5eea00ceca05 expref: -99 pid: 156417 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826035.517462:0:156417:0:(vvp_object.c:117:vvp_conf_set()) [0x20001a9e3:0x1bd1:0x0]: losing layout lock 00010000:00010000:1.0:1787826035.517501:0:156417:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6f098 lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x1bd1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00ceca05 expref: -99 pid: 156417 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826035.517507:0:156417:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6f091 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cec9e9 expref: -99 pid: 156417 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787826035.517513:0:156417:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:1.0:1787826035.517515:0:156417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5e00/0xbef42c0221d6f098 lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x1bd1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00ceca05 expref: -99 pid: 156417 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826035.517535:0:156417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512e00/0xbef42c0221d6f091 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cec9e9 expref: -99 pid: 156417 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000020:00100000:1.0:1787826035.518143:0:156417:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787826035.518159:0:156417:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c35b3480 x1874663042952704/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 512/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826035.518190:0:156417:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job rm:a9ce6693-0634-46c5-bf21-5ca7d026d44e:156417:1874663042952704:192.168.203.147@tcp:36:rm.0 00000100:00100000:1.0:1787826035.518329:0:156417:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b593c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826035.541010:0:156417:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job rm:a9ce6693-0634-46c5-bf21-5ca7d026d44e:156417:1874663042952704:192.168.203.147@tcp:36:rm.0 00000080:00200000:1.0:1787826035.541099:0:156417:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1bd1:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826035.541113:0:156417:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1bd1:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826035.541127:0:156417:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1bd1:0x0](ffff9181d08021c8) 00010000:00010000:3.0:1787826036.581054:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826037.600963:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826039.329693:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042952960:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826039.329850:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042953216:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826039.329888:0:2389:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:0.0:1787826039.329895:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7528148b-9bee-4d31-b597-8aae3b0b4982:2389:1874663042952832:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826039.329941:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042953088:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826039.329972:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042953344:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826039.339128:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7528148b-9bee-4d31-b597-8aae3b0b4982:2389:1874663042952832:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826039.339157:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042953088:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826039.339170:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042953344:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826039.339184:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042952960:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826039.339197:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042953216:192.168.203.147@tcp:400:kworker.0 00010000:00010000:3.0:1787826040.672932:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826041.696957:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826044.769940:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826045.793030:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826048.865060:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826049.569376:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2392:1874663042954240:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826049.570877:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2392:1874663042954496:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826049.576701:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2392:1874663042954496:192.168.203.147@tcp:400:kworker.0 00000100:00080000:3.0:1787826049.578658:0:2392:0:(recover.c:205:ptlrpc_request_handle_notconn()) import lustre-MDT0000-mdc-ffff9181c4d2b000 of lustre-MDT0000_UUID@192.168.203.147@tcp abruptly disconnected: reconnecting 00000100:00080000:3.0:1787826049.578665:0:2392:0:(import.c:65:import_set_state_nolock()) ffff9181c39f6800 lustre-MDT0000_UUID: changing import state from FULL to DISCONN 00000100:02000400:3.0:1787826049.578671:0:2392:0:(import.c:182:ptlrpc_set_import_discon()) lustre-MDT0000-mdc-ffff9181c4d2b000: Connection to lustre-MDT0000 (at 192.168.203.147@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00080000:3.0:1787826049.578677:0:2392:0:(import.c:65:import_set_state_nolock()) ffff9181c39f6800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826049.578682:0:2392:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c4d2b000: connect to NID 192.168.203.147@tcp last attempt 8080 00000100:00080000:3.0:1787826049.578692:0:2392:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c4d2b000: import ffff9181c39f6800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826049.578703:0:2392:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826049.578739:0:2392:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634700 x1874663042954752/t0(0) o38->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826049.578897:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2392:1874663042954240:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826049.578966:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042954752:192.168.203.147@tcp:38:ptlrpcd_01_01.0 00000100:00080000:0.0:1787826049.583342:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c39f6800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826049.583356:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-19) 00000100:00100000:0.0:1787826049.583361:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042954752:192.168.203.147@tcp:38:ptlrpcd_01_01.0 00010000:00010000:3.0:1787826049.890782:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826052.962988:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826053.984943:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826054.694395:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042955008:192.168.203.147@tcp:38:kworker.0 00000100:00080000:1.0:1787826054.711299:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c39f6800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787826054.711312:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-19) 00000100:00100000:1.0:1787826054.711329:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042955008:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1787826057.058100:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826058.080955:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826059.809345:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042955648:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787826059.809553:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042956032:192.168.203.147@tcp:9:kworker.0 00000100:00080000:1.0:1787826059.818347:0:2388:0:(import.c:2011:ptlrpc_disconnect_idle_interpret()) @@@ inflight=1, refcount=3: rc = 0 req@ffff9181c71f1f80 x1874663042956032/t0(0) o9->lustre-OST0001-osc-ffff9181c4d2b000@192.168.203.147@tcp:28/4 lens 224/224 e 0 to 0 dl 1787826069 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787826059.818360:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181e0e59800 lustre-OST0001_UUID: changing import state from CONNECTING to IDLE 00000100:00100000:1.0:1787826059.818365:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042956032:192.168.203.147@tcp:9:kworker.0 00000100:00080000:1.0:1787826059.821580:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c39f6800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787826059.821587:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-19) 00000100:00100000:1.0:1787826059.821592:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042955648:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1787826061.157832:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826062.177319:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826063.212553:0:33329:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787826064.932071:0:152015:0:(pinger.c:229:ptlrpc_pinger_process_import()) a9ce6693-0634-46c5-bf21-5ca7d026d44e->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826064.932079:0:152015:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787826064.932082:0:152015:0:(import.c:65:import_set_state_nolock()) ffff9181c39f6800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826064.932093:0:152015:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c4d2b000: connect to NID 192.168.203.147@tcp last attempt 8131 00000100:00080000:1.0:1787826064.932105:0:152015:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c4d2b000: tried all connections, increasing latency to 15s 00000100:00080000:1.0:1787826064.932113:0:152015:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c4d2b000: import ffff9181c39f6800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787826064.932124:0:152015:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787826064.932144:0:152015:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3b80 x1874663042956288/t0(0) o38->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826064.932178:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7528148b-9bee-4d31-b597-8aae3b0b4982:2389:1874663042956160:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826064.932279:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042956544:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826064.932306:0:2389:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1787826064.932313:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042956416:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826064.932374:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042956288:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787826064.937346:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042956544:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826064.969424:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042956416:192.168.203.147@tcp:400:kworker.0 00010000:00010000:0.0:1787826065.250234:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000400:3.0:1787826065.957828:0:2391:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787826049/real 1787826049] req@ffff9181c5635500 x1874663042954112/t0(0) o400->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 224/224 e 0 to 1 dl 1787826065 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826065.957862:0:2391:0:(client.c:2547:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff9181c5635500 x1874663042954112/t0(0) o400->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 224/224 e 0 to 1 dl 1787826065 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826065.957871:0:2391:0:(import.c:65:import_set_state_nolock()) ffff9181c5a6c800 MGS: changing import state from FULL to DISCONN 00000100:02020000:3.0:1787826065.957874:0:2391:0:(import.c:186:ptlrpc_set_import_discon()) MGC192.168.203.147@tcp: Connection to MGS (at 192.168.203.147@tcp) was lost; in progress operations using this service will fail 00000100:00080000:3.0:1787826065.957878:0:2391:0:(import.c:192:ptlrpc_set_import_discon()) import MGS@192.168.203.147@tcp for MGC192.168.203.147@tcp not replayable, auto-deactivating 00000100:00080000:3.0:1787826065.957880:0:2391:0:(import.c:138:ptlrpc_deactivate_import_nolock()) setting import MGS INVALID 00000100:00100000:3.0:1787826065.957884:0:2391:0:(client.c:3589:ptlrpc_abort_inflight()) @@@ inflight req@ffff9181c5635500 x1874663042954112/t0(0) o400->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 224/224 e 0 to 1 dl 1787826065 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826065.957893:0:2391:0:(client.c:3589:ptlrpc_abort_inflight()) @@@ inflight req@ffff9181c71f0e00 x1874663042954880/t0(0) o400->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 224/224 e 0 to 0 dl 1787826070 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826065.957902:0:2391:0:(client.c:3589:ptlrpc_abort_inflight()) @@@ inflight req@ffff9181c71f3b80 x1874663042955520/t0(0) o400->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 224/224 e 0 to 0 dl 1787826075 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826065.957912:0:2391:0:(client.c:3589:ptlrpc_abort_inflight()) @@@ inflight req@ffff9181c35b1f80 x1874663042956160/t0(0) o400->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 224/224 e 0 to 0 dl 1787826080 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 10000000:01000000:3.0:1787826065.957928:0:2391:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808001 00000100:00080000:3.0:1787826065.957931:0:2391:0:(pinger.c:193:ptlrpc_pinger_ir_down()) IR down 10000000:01000000:3.0:1787826065.957932:0:2391:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808002 00000100:00080000:3.0:1787826065.957934:0:2391:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00100000:3.0:1787826065.957963:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7528148b-9bee-4d31-b597-8aae3b0b4982:2391:1874663042955520:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826065.957971:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7528148b-9bee-4d31-b597-8aae3b0b4982:2391:1874663042954880:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826065.957976:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7528148b-9bee-4d31-b597-8aae3b0b4982:2391:1874663042954112:192.168.203.147@tcp:400:kworker.0 00000100:00080000:3.0:1787826065.958080:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7528148b-9bee-4d31-b597-8aae3b0b4982->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826065.958084:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787826065.958086:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5a6c800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826065.958092:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8071 00000100:00080000:3.0:1787826065.958103:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c5a6c800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787826065.958127:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663042956672/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826065.958142:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) a9ce6693-0634-46c5-bf21-5ca7d026d44e->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826065.958145:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787826065.958147:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c39f6800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826065.958150:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c4d2b000: connect to NID 192.168.203.147@tcp last attempt 8136 00000100:00080000:3.0:1787826065.958153:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c4d2b000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826065.958156:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c4d2b000: import ffff9181c39f6800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826065.958164:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826065.958173:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637b80 x1874663042956800/t0(0) o38->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826065.958216:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7528148b-9bee-4d31-b597-8aae3b0b4982:2388:1874663042956672:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787826065.958288:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042956800:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787826065.958324:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2391:1874663042956928:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826065.958350:0:2391:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787826065.958355:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2391:1874663042957056:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826065.958971:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7528148b-9bee-4d31-b597-8aae3b0b4982:2389:1874663042956160:192.168.203.147@tcp:400:kworker.0 00000100:00080000:3.0:1787826065.965322:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c39f6800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826065.965329:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-19) 00000100:00100000:3.0:1787826065.965333:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042956800:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787826066.273590:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826069.349001:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826070.369019:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787826071.080055:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7528148b-9bee-4d31-b597-8aae3b0b4982->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826071.080066:0:99984:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7528148b-9bee-4d31-b597-8aae3b0b4982->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1787826071.080069:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) a9ce6693-0634-46c5-bf21-5ca7d026d44e->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826071.080072:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787826071.080076:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c39f6800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826071.080086:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c4d2b000: connect to NID 192.168.203.147@tcp last attempt 8137 00000100:00080000:1.0:1787826071.080096:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c4d2b000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787826071.080103:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c4d2b000: import ffff9181c39f6800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787826071.080115:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787826071.080146:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3800 x1874663042957184/t0(0) o38->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826071.080246:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042957312:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826071.080330:0:2390:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1787826071.080338:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042957440:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826071.080350:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042957184:192.168.203.147@tcp:38:kworker.0 00000100:00080000:3.0:1787826071.086115:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c39f6800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826071.086129:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-19) 00000100:00100000:3.0:1787826071.086136:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042957184:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787826071.093591:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042957440:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826071.093606:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042957312:192.168.203.147@tcp:400:kworker.0 00010000:00010000:0.0:1787826073.441699:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826074.464891:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826075.170248:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787826065/real 1787826065] req@ffff9181c5635500 x1874663042956672/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826075 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826075.170279:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663042956672/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826075 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826075.170296:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5a6c800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826075.170305:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826075.172240:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7528148b-9bee-4d31-b597-8aae3b0b4982:2388:1874663042956672:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787826075.172263:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7528148b-9bee-4d31-b597-8aae3b0b4982:2388:1874663042957568:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787826075.173449:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042957696:192.168.203.147@tcp:38:kworker.0 00000100:00080000:3.0:1787826075.193912:0:2388:0:(import.c:1132:ptlrpc_connect_interpret()) MGC192.168.203.147@tcp: connect to target with instance 0 10000000:01000000:3.0:1787826075.193929:0:2388:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808005 00000100:02000400:3.0:1787826075.193937:0:2388:0:(import.c:1284:ptlrpc_connect_interpret()) Evicted from MGS (at 192.168.203.147@tcp) after server handle changed from 0x878a5eea00ce748b to 0x878a5eea00cecb63 00000100:00080000:3.0:1787826075.193942:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5a6c800 MGS: changing import state from CONNECTING to EVICTED 00000100:00080000:3.0:1787826075.193947:0:2388:0:(import.c:1674:ptlrpc_import_recovery_state_machine()) evicted from MGS@192.168.203.147@tcp; invalidating 00000100:00100000:3.0:1787826075.195481:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7528148b-9bee-4d31-b597-8aae3b0b4982:2388:1874663042957568:192.168.203.147@tcp:250:kworker.0 00010000:00080000:3.0:1787826075.195530:0:2388:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=0) req@ffff9181c71f1c00 x1874663042957696/t0(0) o38->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 520/416 e 0 to 0 dl 1787826100 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826075.195548:0:2388:0:(import.c:1132:ptlrpc_connect_interpret()) lustre-MDT0000-mdc-ffff9181c4d2b000: connect to target with instance 79 00000100:00080000:3.0:1787826075.195563:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c39f6800 lustre-MDT0000_UUID: changing import state from CONNECTING to REPLAY 00000100:00080000:3.0:1787826075.195569:0:2388:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:3.0:1787826075.195572:0:2388:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9181c4d2b000: committing for last_committed 335007449220 gen 11 00000100:00100000:3.0:1787826075.195579:0:2388:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff9181c71f1880 x1874663042952192/t335007449221(335007449221) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 528/448 e 0 to 0 dl 1787826051 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826075.195587:0:2388:0:(recover.c:53:ptlrpc_replay_next()) import ffff9181c39f6800 from lustre-MDT0000_UUID committed 335007449220 last 0 00000100:00080000:3.0:1787826075.195594:0:2388:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9181c3573480 x1874663041393408/t330712482189(330712482189) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 0 dl 1787826024 ref 1 fl New:RPQU/604/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:0 00000100:00100000:3.0:1787826075.195606:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042957696:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787826075.195659:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c3573480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663041393408:192.168.203.147@tcp:101:lfs.0 00000100:00080000:1.0:1787826075.196428:0:156721:0:(import.c:1598:ptlrpc_invalidate_import_thread()) thread invalidate import MGC192.168.203.147@tcp to MGS@192.168.203.147@tcp 00000100:00100000:1.0:1787826075.196433:0:156721:0:(import.c:323:ptlrpc_invalidate_import()) Sleeping 20s for inflight to error out 10000000:01000000:1.0:1787826075.196437:0:156721:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808003 00010000:00010000:1.0:1787826075.196452:0:156721:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e06f7600/0xbef42c0221d6e5bd lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x110e400000000 nid: local remote: 0x878a5eea00ce7499 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787826075.196464:0:156721:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e06f7600/0xbef42c0221d6e5bd lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x110f400000000 nid: local remote: 0x878a5eea00ce7499 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826075.196470:0:156721:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1787826075.196480:0:156721:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787826075.196488:0:156721:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e06f7600/0xbef42c0221d6e5bd lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x590f400000000 nid: local remote: 0x878a5eea00ce7499 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826075.196514:0:156721:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b5a00/0xbef42c0221d6e5cb lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0x878a5eea00ce74a7 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787826075.196521:0:156721:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5a00/0xbef42c0221d6e5cb lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0x878a5eea00ce74a7 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826075.196526:0:156721:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1787826075.196530:0:156721:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787826075.196533:0:156721:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b5a00/0xbef42c0221d6e5cb lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0x878a5eea00ce74a7 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826075.196551:0:156721:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e06f6200/0xbef42c0221d6e5b6 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0x878a5eea00ce7492 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787826075.196558:0:156721:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e06f6200/0xbef42c0221d6e5b6 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0x878a5eea00ce7492 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826075.196563:0:156721:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:1.0:1787826075.196567:0:156721:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787826075.196570:0:156721:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e06f6200/0xbef42c0221d6e5b6 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0x878a5eea00ce7492 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826075.196582:0:156721:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6e5c4 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0x878a5eea00ce74a0 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787826075.196588:0:156721:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6e5c4 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0x878a5eea00ce74a0 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826075.196592:0:156721:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:1.0:1787826075.196595:0:156721:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787826075.196597:0:156721:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6e5c4 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0x878a5eea00ce74a0 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787826075.196609:0:156721:0:(import.c:65:import_set_state_nolock()) ffff9181c5a6c800 MGS: changing import state from EVICTED to RECOVER 00000100:00080000:1.0:1787826075.196623:0:156721:0:(import.c:65:import_set_state_nolock()) ffff9181c5a6c800 MGS: changing import state from RECOVER to FULL 10000000:01000000:1.0:1787826075.196625:0:156721:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808004 00000100:00080000:1.0:1787826075.196629:0:156721:0:(pinger.c:186:ptlrpc_pinger_ir_up()) IR up 00000100:02000000:1.0:1787826075.196632:0:156721:0:(import.c:1743:ptlrpc_import_recovery_state_machine()) MGC192.168.203.147@tcp: Connection restored to 192.168.203.147@tcp (at 192.168.203.147@tcp) 10000000:01000000:3.0:1787826076.448822:0:15163:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:3.0:1787826076.448834:0:15163:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff9181c4d2b000 from 324 10000000:01000000:3.0:1787826076.448842:0:15163:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:3.0:1787826076.448916:0:15163:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f09f lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826076.448930:0:15163:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f09f lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826076.448945:0:15163:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f09f lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826076.448983:0:15163:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:7528148b-9bee-4d31-b597-8aae3b0b4982:15163:1874663042958080:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00100000:3.0:1787826076.449132:0:15163:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c7da1540 going to sleep for 11 seconds 00000100:00100000:3.0:1787826076.464544:0:15163:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:7528148b-9bee-4d31-b597-8aae3b0b4982:15163:1874663042958080:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:3.0:1787826076.464595:0:15163:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f09f lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00cecb6a expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826076.464611:0:15163:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f09f lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x878a5eea00cecb6a expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826076.464735:0:15163:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:7528148b-9bee-4d31-b597-8aae3b0b4982:15163:1874663042958208:192.168.203.147@tcp:256:ll_cfg_requeue.0 00000100:00100000:3.0:1787826076.464856:0:15163:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c7da1540 going to sleep for 11 seconds 00000100:00100000:3.0:1787826076.491128:0:15163:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:7528148b-9bee-4d31-b597-8aae3b0b4982:15163:1874663042958208:192.168.203.147@tcp:256:ll_cfg_requeue.0 10000000:01000000:3.0:1787826076.491253:0:15163:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-cliir' succeeded (0). 00010000:00010000:3.0:1787826076.491267:0:15163:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f09f lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00cecb6a expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826076.491280:0:15163:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f09f lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00cecb6a expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787826076.491292:0:15163:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:3.0:1787826076.491295:0:15163:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff9181c4d2b000 from 52 10000000:01000000:3.0:1787826076.491299:0:15163:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:3.0:1787826076.491348:0:15163:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f0a6 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826076.491357:0:15163:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f0a6 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826076.491373:0:15163:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f0a6 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826076.491399:0:15163:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:7528148b-9bee-4d31-b597-8aae3b0b4982:15163:1874663042958336:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00100000:3.0:1787826076.491497:0:15163:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c7da1540 going to sleep for 11 seconds 00000100:00100000:3.0:1787826076.511994:0:15163:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f0a80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:7528148b-9bee-4d31-b597-8aae3b0b4982:15163:1874663042958464:192.168.203.147@tcp:501:ll_cfg_requeue.0 00000100:00100000:3.0:1787826076.512087:0:15163:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:7528148b-9bee-4d31-b597-8aae3b0b4982:15163:1874663042958592:192.168.203.147@tcp:503:ll_cfg_requeue.0 00000100:00100000:3.0:1787826076.512212:0:15163:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c7da1e40 going to sleep for 11 seconds 00010000:00010000:1.0:1787826076.528972:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826076.529554:0:15163:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:7528148b-9bee-4d31-b597-8aae3b0b4982:15163:1874663042958592:192.168.203.147@tcp:503:ll_cfg_requeue.0 00000040:00080000:1.0:1787826076.530593:0:156737:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x3:0x0] index 64768 count 51 00000020:01000000:0.0:1787826076.530898:0:15163:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-client gen 52-51 (rc=0) 10000000:01000000:0.0:1787826076.530932:0:15163:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-client' succeeded (0). 00010000:00010000:0.0:1787826076.530948:0:15163:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f0a6 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00cecb71 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826076.530962:0:15163:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f0a6 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00cecb71 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787826076.530972:0:15163:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:0.0:1787826076.530975:0:15163:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff9181c4d2b000 from 7 10000000:01000000:0.0:1787826076.530979:0:15163:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00010000:00010000:0.0:1787826076.531032:0:15163:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512e00/0xbef42c0221d6f0ad lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826076.531045:0:15163:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e0512e00/0xbef42c0221d6f0ad lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826076.531058:0:15163:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e0512e00/0xbef42c0221d6f0ad lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826076.531103:0:15163:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:7528148b-9bee-4d31-b597-8aae3b0b4982:15163:1874663042958720:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00100000:0.0:1787826076.531213:0:15163:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d800c240 going to sleep for 11 seconds 00000100:00100000:1.0:1787826076.556978:0:15163:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:7528148b-9bee-4d31-b597-8aae3b0b4982:15163:1874663042958720:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787826076.557018:0:15163:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.147@tcp lock: ffff9181e0512e00/0xbef42c0221d6f0ad lrc: 4/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00cecb7f expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826076.557034:0:15163:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.147@tcp lock: ffff9181e0512e00/0xbef42c0221d6f0ad lrc: 4/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x878a5eea00cecb7f expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826076.557163:0:15163:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:7528148b-9bee-4d31-b597-8aae3b0b4982:15163:1874663042958848:192.168.203.147@tcp:501:ll_cfg_requeue.0 00000100:00100000:1.0:1787826076.557274:0:15163:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b59c00 going to sleep for 11 seconds 00000100:00100000:0.0:1787826076.575902:0:15163:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:7528148b-9bee-4d31-b597-8aae3b0b4982:15163:1874663042958848:192.168.203.147@tcp:501:ll_cfg_requeue.0 00000100:00100000:0.0:1787826076.576005:0:15163:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:7528148b-9bee-4d31-b597-8aae3b0b4982:15163:1874663042958976:192.168.203.147@tcp:503:ll_cfg_requeue.0 00000100:00100000:0.0:1787826076.576141:0:15163:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d800c9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826076.585560:0:15163:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:7528148b-9bee-4d31-b597-8aae3b0b4982:15163:1874663042958976:192.168.203.147@tcp:503:ll_cfg_requeue.0 00000020:01000000:1.0:1787826076.589708:0:15163:0:(obd_config.c:2145:class_config_parse_llog()) Processed log params gen 7-6 (rc=0) 10000000:01000000:1.0:1787826076.589735:0:15163:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'params' succeeded (0). 00010000:00010000:1.0:1787826076.589773:0:15163:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512e00/0xbef42c0221d6f0ad lrc: 3/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00cecb7f expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826076.589787:0:15163:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.147@tcp lock: ffff9181e0512e00/0xbef42c0221d6f0ad lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00cecb7f expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826076.589799:0:15163:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:1.0:1787826076.589803:0:15163:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff9181c3d3cd28 from 1 10000000:01000000:1.0:1787826076.589806:0:15163:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00010000:00010000:1.0:1787826076.589862:0:15163:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5e00/0xbef42c0221d6f0b4 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826076.589871:0:15163:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b5e00/0xbef42c0221d6f0b4 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826076.589890:0:15163:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b5e00/0xbef42c0221d6f0b4 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826076.589935:0:15163:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:7528148b-9bee-4d31-b597-8aae3b0b4982:15163:1874663042959104:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00100000:1.0:1787826076.590052:0:15163:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b593c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826076.600539:0:15163:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:7528148b-9bee-4d31-b597-8aae3b0b4982:15163:1874663042959104:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787826076.600582:0:15163:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.147@tcp lock: ffff9181e01b5e00/0xbef42c0221d6f0b4 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00cecbe1 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826076.600598:0:15163:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.147@tcp lock: ffff9181e01b5e00/0xbef42c0221d6f0b4 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x878a5eea00cecbe1 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826076.601884:0:15163:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:7528148b-9bee-4d31-b597-8aae3b0b4982:15163:1874663042959232:192.168.203.147@tcp:501:ll_cfg_requeue.0 00000100:00100000:1.0:1787826076.602011:0:15163:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1b593c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826076.637088:0:15163:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:7528148b-9bee-4d31-b597-8aae3b0b4982:15163:1874663042959232:192.168.203.147@tcp:501:ll_cfg_requeue.0 10000000:01000000:0.0:1787826076.637157:0:15163:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-sptlrpc' failed (-2). 00010000:00010000:0.0:1787826076.637171:0:15163:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5e00/0xbef42c0221d6f0b4 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00cecbe1 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826076.637185:0:15163:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.147@tcp lock: ffff9181e01b5e00/0xbef42c0221d6f0b4 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00cecbe1 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826077.537186:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826078.561108:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826080.293772:0:15160:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 0 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826080.293788:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826080.293790:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826080.293792:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826080.609668:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826081.634856:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200008:3.0:1787826084.567680:0:156822:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1787826084.567702:0:156822:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:3.0:1787826084.567735:0:156822:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826084.567833:0:156822:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481600/0xbef42c0221d6f0bb lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 156822 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826084.567846:0:156822:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481600/0xbef42c0221d6f0bb lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 156822 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826084.567858:0:156822:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481600/0xbef42c0221d6f0bb lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 156822 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787826084.567890:0:156822:0:(client.c:1845:ptlrpc_send_new_req()) @@@ req waiting for recovery: (FULL != REPLAY) req@ffff9181c5637800 x1874663042959744/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/1152 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/600/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:0 00000100:00100000:3.0:1787826084.567904:0:156822:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c714e240 going to sleep for 0 seconds 00010000:00010000:0.0:1787826084.709224:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787826085.409847:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) a9ce6693-0634-46c5-bf21-5ca7d026d44e->lustre-MDT0000_UUID: level REPLAY/5 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826085.409856:0:111835:0:(pinger.c:247:ptlrpc_pinger_process_import()) a9ce6693-0634-46c5-bf21-5ca7d026d44e->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: REPLAY) 00000100:00100000:1.0:1787826085.410394:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042960000:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826085.410516:0:2389:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1787826085.410523:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7528148b-9bee-4d31-b597-8aae3b0b4982:2389:1874663042959872:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826085.410557:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042960128:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826085.421484:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7528148b-9bee-4d31-b597-8aae3b0b4982:2389:1874663042959872:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826085.421520:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042960128:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826085.421534:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042960000:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826085.603653:0:156822:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c714e240 going to sleep for 0 seconds 00010000:00010000:3.0:1787826085.728912:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826086.628837:0:156822:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c714e240 going to sleep for 0 seconds 00000100:00100000:3.0:1787826087.650706:0:156822:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c714e240 going to sleep for 0 seconds 00000100:00100000:3.0:1787826088.672917:0:156822:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c714e240 going to sleep for 0 seconds 00010000:00010000:0.0:1787826088.800893:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826089.696882:0:156822:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c714e240 going to sleep for 0 seconds 00010000:00010000:3.0:1787826089.824906:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787826090.529348:0:152015:0:(pinger.c:229:ptlrpc_pinger_process_import()) a9ce6693-0634-46c5-bf21-5ca7d026d44e->lustre-MDT0000_UUID: level REPLAY/5 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826090.529356:0:152015:0:(pinger.c:247:ptlrpc_pinger_process_import()) a9ce6693-0634-46c5-bf21-5ca7d026d44e->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: REPLAY) 00000100:00100000:0.0:1787826090.529472:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042960384:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826090.529606:0:2390:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1787826090.529614:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7528148b-9bee-4d31-b597-8aae3b0b4982:2390:1874663042960256:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826090.529648:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042960512:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826090.535161:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7528148b-9bee-4d31-b597-8aae3b0b4982:2390:1874663042960256:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826090.541617:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042960512:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826090.552104:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042960384:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826090.720888:0:156822:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c714e240 going to sleep for 0 seconds 00000100:00000400:1.0:1787826091.556084:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787826075/real 1787826075] req@ffff9181c3573480 x1874663041393408/t330712482189(330712482189) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 1 dl 1787826091 ref 2 fl Rpc:XPQr/604/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:0 00000100:00100000:1.0:1787826091.556128:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=REPLAY (now=REPLAY) req@ffff9181c3573480 x1874663041393408/t330712482189(330712482189) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 1 dl 1787826091 ref 2 fl Rpc:XPQU/604/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:0 00000100:00020000:1.0:1787826091.556151:0:2388:0:(client.c:3398:ptlrpc_replay_interpret()) @@@ request replay timed out req@ffff9181c3573480 x1874663041393408/t330712482189(330712482189) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 1 dl 1787826091 ref 2 fl Interpret:EXPQU/604/ffffffff rc -110/-1 job:'lfs.0' uid:0 gid:0 projid:0 00000100:00080000:1.0:1787826091.589087:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c39f6800 lustre-MDT0000_UUID: changing import state from REPLAY to CONNECTING 00000100:00080000:1.0:1787826091.589109:0:2388:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c4d2b000: connect to NID 192.168.203.147@tcp last attempt 8146 00000100:00080000:1.0:1787826091.589130:0:2388:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c4d2b000: import ffff9181c39f6800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787826091.589146:0:2388:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787826091.589912:0:2388:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1880 x1874663042960640/t0(0) o38->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826091.589941:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c3573480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663041393408:192.168.203.147@tcp:101:lfs.0 00000100:00100000:1.0:1787826091.589965:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042960640:192.168.203.147@tcp:38:ptlrpcd_rcv.0 00010000:00080000:3.0:1787826091.624581:0:2388:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffff9181c35b1880 x1874663042960640/t0(0) o38->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 520/416 e 0 to 0 dl 1787826101 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'ptlrpcd_rcv.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826091.624672:0:2388:0:(import.c:1132:ptlrpc_connect_interpret()) lustre-MDT0000-mdc-ffff9181c4d2b000: connect to target with instance 79 00000100:00080000:3.0:1787826091.624692:0:2388:0:(import.c:1297:ptlrpc_connect_interpret()) reconnected to lustre-MDT0000_UUID@192.168.203.147@tcp after partition 00000100:00080000:3.0:1787826091.624694:0:2388:0:(import.c:1307:ptlrpc_connect_interpret()) lustre-MDT0000-mdc-ffff9181c4d2b000: reconnected to lustre-MDT0000_UUID during replay 00000100:00080000:3.0:1787826091.624697:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c39f6800 lustre-MDT0000_UUID: changing import state from CONNECTING to REPLAY 00000100:00080000:3.0:1787826091.624701:0:2388:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:3.0:1787826091.624704:0:2388:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9181c4d2b000: committing for last_committed 335007449220 gen 11 00000100:00100000:3.0:1787826091.624711:0:2388:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff9181c71f1880 x1874663042952192/t335007449221(335007449221) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 528/448 e 0 to 0 dl 1787826051 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826091.624720:0:2388:0:(recover.c:53:ptlrpc_replay_next()) import ffff9181c39f6800 from lustre-MDT0000_UUID committed 335007449220 last 0 00000100:00080000:3.0:1787826091.625149:0:2388:0:(recover.c:118:ptlrpc_replay_next()) @@@ resend_replay=1, last_transno=0 req@ffff9181c3573480 x1874663041393408/t330712482189(330712482189) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 1 dl 1787826091 ref 1 fl Complete:EXPQU/606/ffffffff rc -110/-1 job:'lfs.0' uid:0 gid:0 projid:0 00000100:00080000:3.0:1787826091.625162:0:2388:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9181c3573480 x1874663041393408/t330712482189(330712482189) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 1 dl 1787826091 ref 1 fl New:EXPQU/606/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:0 00000100:00100000:3.0:1787826091.625176:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042960640:192.168.203.147@tcp:38:ptlrpcd_rcv.0 00000100:00100000:3.0:1787826091.625206:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c3573480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663041393408:192.168.203.147@tcp:101:lfs.0 00000100:00080000:3.0:1787826091.690502:0:2388:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9181c3573480 x1874663041393408/t330712482189(330712482189) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 0 dl 1787826107 ref 2 fl Interpret:RPQU/604/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00080000:3.0:1787826091.690526:0:2388:0:(mdc_request.c:696:mdc_replay_open()) updating handle from 0x878a5eea00ce74b5 to 0x878a5eea00cecc6d 00000100:00020000:3.0:1787826091.690533:0:2388:0:(client.c:3448:ptlrpc_replay_interpret()) @@@ status 301, old was 0 req@ffff9181c3573480 x1874663041393408/t330712482189(330712482189) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 0 dl 1787826107 ref 2 fl Interpret:RPQU/604/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000100:00080000:3.0:1787826091.690847:0:2388:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:3.0:1787826091.690851:0:2388:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9181c4d2b000: committing for last_committed 335007449220 gen 11 00000100:00100000:3.0:1787826091.690859:0:2388:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff9181c71f1880 x1874663042952192/t335007449221(335007449221) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 528/448 e 0 to 0 dl 1787826051 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826091.690868:0:2388:0:(recover.c:53:ptlrpc_replay_next()) import ffff9181c39f6800 from lustre-MDT0000_UUID committed 335007449220 last 330712482189 00000100:00080000:3.0:1787826091.690878:0:2388:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9181c71f2300 x1874663042950784/t335007449219(335007449219) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826050 ref 1 fl New:RPQU/604/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00000100:00100000:3.0:1787826091.690893:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c3573480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663041393408:192.168.203.147@tcp:101:lfs.0 00000100:00100000:3.0:1787826091.690925:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042950784:192.168.203.147@tcp:101:multiop.0 00000100:00080000:3.0:1787826091.696949:0:2388:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9181c71f2300 x1874663042950784/t335007449219(335007449219) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 592/608 e 0 to 0 dl 1787826107 ref 2 fl Interpret:RPQU/604/0 rc 301/301 job:'multiop.0' uid:0 gid:0 projid:0 00000002:00080000:3.0:1787826091.696974:0:2388:0:(mdc_request.c:696:mdc_replay_open()) updating handle from 0x878a5eea00cec995 to 0x878a5eea00cecc7b 00000100:00020000:3.0:1787826091.696980:0:2388:0:(client.c:3448:ptlrpc_replay_interpret()) @@@ status 301, old was 0 req@ffff9181c71f2300 x1874663042950784/t335007449219(335007449219) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 592/608 e 0 to 0 dl 1787826107 ref 2 fl Interpret:RPQU/604/0 rc 301/301 job:'multiop.0' uid:0 gid:0 projid:0 00000100:00080000:3.0:1787826091.696988:0:2388:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:3.0:1787826091.696992:0:2388:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9181c4d2b000: committing for last_committed 335007449220 gen 11 00000100:00100000:3.0:1787826091.697068:0:2388:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff9181c71f1880 x1874663042952192/t335007449221(335007449221) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 528/448 e 0 to 0 dl 1787826051 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826091.697077:0:2388:0:(recover.c:53:ptlrpc_replay_next()) import ffff9181c39f6800 from lustre-MDT0000_UUID committed 335007449220 last 335007449219 00000100:00080000:3.0:1787826091.697086:0:2388:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9181c71f1880 x1874663042952192/t335007449221(335007449221) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 528/448 e 0 to 0 dl 1787826051 ref 1 fl New:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826091.697099:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042950784:192.168.203.147@tcp:101:multiop.0 00000100:00100000:3.0:1787826091.697130:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042952192:192.168.203.147@tcp:36:mcreate.0 00000100:00080000:3.0:1787826091.701725:0:2388:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9181c71f1880 x1874663042952192/t335007449221(335007449221) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 528/448 e 0 to 0 dl 1787826107 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826091.701765:0:2388:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:3.0:1787826091.701769:0:2388:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9181c4d2b000: committing for last_committed 335007449220 gen 11 00000100:00100000:3.0:1787826091.701775:0:2388:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff9181c71f1880 x1874663042952192/t335007449221(335007449221) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 528/448 e 0 to 0 dl 1787826107 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826091.701785:0:2388:0:(recover.c:53:ptlrpc_replay_next()) import ffff9181c39f6800 from lustre-MDT0000_UUID committed 335007449220 last 335007449221 00000100:00080000:3.0:1787826091.701796:0:2388:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9181c35b3480 x1874663042952704/t335007449222(335007449222) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 512/456 e 0 to 0 dl 1787826051 ref 1 fl New:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826091.701811:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042952192:192.168.203.147@tcp:36:mcreate.0 00000100:00100000:3.0:1787826091.701833:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042952704:192.168.203.147@tcp:36:rm.0 00000100:00080000:3.0:1787826091.705637:0:2388:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9181c35b3480 x1874663042952704/t335007449222(335007449222) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 512/456 e 0 to 0 dl 1787826107 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826091.705658:0:2388:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:3.0:1787826091.705661:0:2388:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9181c4d2b000: committing for last_committed 335007449220 gen 11 00000100:00100000:3.0:1787826091.705669:0:2388:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff9181c71f1880 x1874663042952192/t335007449221(335007449221) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 528/448 e 0 to 0 dl 1787826107 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826091.705678:0:2388:0:(recover.c:53:ptlrpc_replay_next()) import ffff9181c39f6800 from lustre-MDT0000_UUID committed 335007449220 last 335007449222 00000100:00080000:3.0:1787826091.705685:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c39f6800 lustre-MDT0000_UUID: changing import state from REPLAY to REPLAY_LOCKS 00000100:00100000:1.0:1787826091.706597:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042952704:192.168.203.147@tcp:36:rm.0 00010000:00080000:3.0:1787826091.706730:0:156838:0:(ldlm_request.c:2972:ldlm_lock_replay_thread()) lock replay thread lustre-MDT0000-mdc-ffff9181c4d2b000 to lustre-MDT0000_UUID@192.168.203.147@tcp 00010000:00010000:3.0:1787826091.706737:0:156838:0:(ldlm_request.c:2888:ldlm_cancel_unused_locks_for_replay()) Dropping as many unused locks as possible before replay for namespace lustre-MDT0000-mdc-ffff9181c4d2b000 (2) 00010000:00010000:3.0:1787826091.706770:0:156838:0:(ldlm_request.c:2901:ldlm_cancel_unused_locks_for_replay()) Canceled 0 unused locks from namespace lustre-MDT0000-mdc-ffff9181c4d2b000 00010000:00010000:3.0:1787826091.709447:0:156838:0:(ldlm_request.c:2860:replay_one_lock()) ### replaying lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513000/0xbef42c0221d64bee lrc: 2/0,0 mode: CR/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce962d expref: -99 pid: 152210 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00080000:3.0:1787826091.709494:0:156838:0:(ldlm_request.c:2910:lock_can_replay()) check lock replay limit, inflights = 1(8) 00010000:00010000:3.0:1787826091.709504:0:156838:0:(ldlm_request.c:2860:replay_one_lock()) ### replaying lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6800/0xbef42c0221d64b3f lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ce9634 expref: -99 pid: 152202 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00080000:3.0:1787826091.709514:0:156838:0:(ldlm_request.c:2910:lock_can_replay()) check lock replay limit, inflights = 2(8) 00000100:00100000:1.0:1787826091.709915:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042960768:192.168.203.147@tcp:101:ldlm_lock_repla.0 00000100:00100000:1.0:1787826091.710034:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042960896:192.168.203.147@tcp:101:ldlm_lock_repla.0 00010000:00010000:3.0:1787826091.720635:0:2388:0:(ldlm_request.c:2777:replay_lock_interpret()) ### replayed lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513000/0xbef42c0221d64bee lrc: 2/0,0 mode: CR/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cecca5 expref: -99 pid: 152210 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000100:00100000:3.0:1787826091.720668:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042960768:192.168.203.147@tcp:101:ldlm_lock_repla.0 00000100:02000000:1.0:1787826091.732135:0:2388:0:(import.c:1743:ptlrpc_import_recovery_state_machine()) lustre-MDT0000-mdc-ffff9181c4d2b000: Connection restored to 192.168.203.147@tcp (at 192.168.203.147@tcp) 00000100:00080000:1.0:1787826091.732146:0:2388:0:(recover.c:191:ptlrpc_wake_delayed()) @@@ waking (set 0000000015271a7b): req@ffff9181c5637800 x1874663042959744/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/1152 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/600/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:0 00000100:00100000:1.0:1787826091.732168:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042961024:192.168.203.147@tcp:400:ptlrpcd_rcv.0 00000100:00100000:3.0:1787826091.732372:0:156822:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c714e240 going to sleep for 0 seconds 00000100:00100000:3.0:1787826091.732485:0:156822:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c714e240 going to sleep for 11 seconds 00000100:00100000:3.0:1787826091.739410:0:156822:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:156822:1874663042959744:192.168.203.147@tcp:101:lfs.0 00010000:00010000:3.0:1787826091.739453:0:156822:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481600/0xbef42c0221d6f0bb lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceccba expref: -99 pid: 156822 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826091.739469:0:156822:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481600/0xbef42c0221d6f0bb lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceccba expref: -99 pid: 156822 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826091.739484:0:156822:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481600/0xbef42c0221d6f0bb lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceccba expref: -99 pid: 156822 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826091.739503:0:156822:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481600/0xbef42c0221d6f0bb lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceccba expref: -99 pid: 156822 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826091.739519:0:156822:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481600/0xbef42c0221d6f0bb lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceccba expref: -99 pid: 156822 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826091.739528:0:156822:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481600/0xbef42c0221d6f0bb lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceccba expref: -99 pid: 156822 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826091.739536:0:156822:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481600/0xbef42c0221d6f0bb lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceccba expref: -99 pid: 156822 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826091.739561:0:156822:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c5637800 x1874663042959744/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826107 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826091.739578:0:156822:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481600/0xbef42c0221d6f0bb lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceccba expref: -99 pid: 156822 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826091.739607:0:156822:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:3.0:1787826091.739616:0:156822:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:3.0:1787826091.739652:0:156822:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 69632 00000080:00010000:3.0:1787826091.739659:0:156822:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6f0bb 00000080:00010000:3.0:1787826091.739665:0:156822:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:3.0:1787826091.739669:0:156822:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f0bb from it ffffb97500d53cb8 00010000:00010000:3.0:1787826091.739672:0:156822:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481600/0xbef42c0221d6f0bb lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceccba expref: -99 pid: 156822 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826091.739688:0:156822:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481600/0xbef42c0221d6f0bb lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceccba expref: -99 pid: 156822 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:3.0:1787826091.740599:0:156822:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), request_mask 0, flags 0, foreign 0 00000080:00200000:3.0:1787826091.740610:0:156822:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:3.0:1787826091.740633:0:156822:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826091.740648:0:156822:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481600/0xbef42c0221d6f0bb lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceccba expref: -99 pid: 156822 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826091.740663:0:156822:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481600/0xbef42c0221d6f0bb lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceccba expref: -99 pid: 156822 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826091.740670:0:156822:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826091.740676:0:156822:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:3.0:1787826091.740679:0:156822:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6f0bb 00000080:00010000:3.0:1787826091.740685:0:156822:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f0bb from it ffffb97500d53cb8 00010000:00010000:3.0:1787826091.740689:0:156822:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481600/0xbef42c0221d6f0bb lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceccba expref: -99 pid: 156822 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826091.740697:0:156822:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481600/0xbef42c0221d6f0bb lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceccba expref: -99 pid: 156822 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:3.0:1787826091.740866:0:156822:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:3.0:1787826091.740876:0:156822:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826091.740884:0:156822:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481600/0xbef42c0221d6f0bb lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceccba expref: -99 pid: 156822 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826091.740900:0:156822:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481600/0xbef42c0221d6f0bb lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceccba expref: -99 pid: 156822 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:3.0:1787826091.740907:0:156822:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826091.740911:0:156822:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:3.0:1787826091.740914:0:156822:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6f0bb 00000080:00010000:3.0:1787826091.740918:0:156822:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f0bb from it ffffb97500d53bb0 00010000:00010000:3.0:1787826091.740921:0:156822:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481600/0xbef42c0221d6f0bb lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceccba expref: -99 pid: 156822 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826091.740929:0:156822:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481600/0xbef42c0221d6f0bb lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceccba expref: -99 pid: 156822 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:3.0:1787826091.740939:0:156822:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 44 00000080:00200008:3.0:1787826091.740946:0:156822:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), flags 100000 00000080:00200080:3.0:1787826091.741064:0:156822:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffd810cbe80 00800000:00000080:3.0:1787826091.741082:0:156822:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442c000 uarg=0000000000000000 00800000:00000004:3.0:1787826091.741091:0:156822:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9181c4d2b000: age 8107, max_age 8162 00000100:00100000:3.0:1787826091.741142:0:156822:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:156822:1874663042961152:192.168.203.147@tcp:41:lfs.0 00000100:00100000:3.0:1787826091.741251:0:156822:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c714e240 going to sleep for 11 seconds 00000100:00100000:3.0:1787826091.752517:0:156822:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:156822:1874663042961152:192.168.203.147@tcp:41:lfs.0 00800000:00000004:3.0:1787826091.752572:0:156822:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0000-mdc-ffff9181c4d2b000: update ffff9181c87b0450 cache blocks 317234/353529 objects 1023645/1024000 00000080:00200080:3.0:1787826091.752700:0:156822:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffd810cbe80 00800000:00000080:3.0:1787826091.752725:0:156822:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442c000 uarg=0000000000000000 00800000:00000004:3.0:1787826091.752735:0:156822:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9181c4d2b000: age 8107, max_age 8162 00000100:00100000:3.0:1787826091.752813:0:156822:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:156822:1874663042961280:192.168.203.147@tcp:41:lfs.0 00000100:00100000:3.0:1787826091.752898:0:156822:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c714e240 going to sleep for 11 seconds 00000100:00100000:3.0:1787826091.761934:0:156822:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:156822:1874663042961280:192.168.203.147@tcp:41:lfs.0 00800000:00000004:3.0:1787826091.761977:0:156822:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0001-mdc-ffff9181c4d2b000: update ffff9181c87b1e08 cache blocks 317336/353529 objects 1023704/1024000 00000100:00100000:1.0:1787826091.773025:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042961536:192.168.203.147@tcp:8:lfs.0 00000100:00100000:1.0:1787826091.782885:0:2388:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-ffff9181c4d2b000: committing for last_committed 4294975530 gen 33 00000100:00080000:1.0:1787826091.782912:0:2388:0:(import.c:1132:ptlrpc_connect_interpret()) lustre-OST0001-osc-ffff9181c4d2b000: connect to target with instance 1 00000100:00080000:1.0:1787826091.782927:0:2388:0:(import.c:956:ptlrpc_connect_set_flags()) lustre-OST0001-osc-ffff9181c4d2b000: Resetting ns_connect_flags to server flags: 0xa0427af2e3440038 00000100:00080000:1.0:1787826091.782945:0:2388:0:(import.c:1213:ptlrpc_connect_interpret()) connected to replayable target: lustre-OST0001_UUID 00000100:00080000:1.0:1787826091.782950:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181e0e59800 lustre-OST0001_UUID: changing import state from CONNECTING to FULL 00000080:00000004:1.0:1787826091.782965:0:2388:0:(lcommon_misc.c:94:cl_ocd_update()) Changing connect_flags: 0xa0427af2e3440038 -> 0xa0427af2e3440038 00000080:00080000:1.0:1787826091.782972:0:2388:0:(lcommon_misc.c:56:cl_init_ea_size()) updating def/max_easize: 72/65536 00000100:00080000:1.0:1787826091.782992:0:2388:0:(recover.c:191:ptlrpc_wake_delayed()) @@@ waking (set 000000009947d58c): req@ffff9181c71f1880 x1874663042961664/t0(0) o13->lustre-OST0001-osc-ffff9181c4d2b000@192.168.203.147@tcp:7/4 lens 224/368 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787826091.789154:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042961536:192.168.203.147@tcp:8:lfs.0 00010000:00010000:0.0:1787826092.897081:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826094.945082:0:33329:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826095.658759:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042961920:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826095.658886:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042962176:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826095.658927:0:2389:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:0.0:1787826095.658933:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7528148b-9bee-4d31-b597-8aae3b0b4982:2389:1874663042961792:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826095.658964:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042962048:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826095.658987:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042962304:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826095.663661:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7528148b-9bee-4d31-b597-8aae3b0b4982:2389:1874663042961792:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826095.663688:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042962048:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826095.663700:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042962304:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826095.663711:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042961920:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826095.663720:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042962176:192.168.203.147@tcp:400:kworker.0 00010000:00010000:0.0:1787826097.010033:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:3.0:1787826098.299204:0:157106:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: oleg347-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 00000001:02000400:0.0:1787826099.725147:0:157255:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec 00000100:00100000:1.0:1787826100.771971:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7528148b-9bee-4d31-b597-8aae3b0b4982:2389:1874663042962432:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826100.777548:0:2389:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1787826100.777573:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042962560:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826100.778503:0:2390:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1787826100.778519:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042962688:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826100.778673:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042962816:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826100.778741:0:2390:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1787826100.778780:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042962944:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826100.780638:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042962560:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826100.780655:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7528148b-9bee-4d31-b597-8aae3b0b4982:2389:1874663042962432:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826100.786294:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042962688:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826100.791139:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042962944:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826100.791159:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042962816:192.168.203.147@tcp:400:kworker.0 00010000:00010000:0.0:1787826101.092123:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200008:3.0:1787826101.680238:0:157293:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1787826101.681047:0:157293:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:3.0:1787826101.681072:0:157293:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826101.681088:0:157293:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481600/0xbef42c0221d6f0bb lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceccba expref: -99 pid: 156822 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:3.0:1787826101.681103:0:157293:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481600/0xbef42c0221d6f0bb lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceccba expref: -99 pid: 156822 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:3.0:1787826101.681111:0:157293:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826101.681118:0:157293:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:3.0:1787826101.681122:0:157293:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6f0bb 00000080:00010000:3.0:1787826101.681129:0:157293:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f0bb from it ffffb97500da3cb8 00010000:00010000:3.0:1787826101.681132:0:157293:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481600/0xbef42c0221d6f0bb lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceccba expref: -99 pid: 156822 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:3.0:1787826101.681140:0:157293:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481600/0xbef42c0221d6f0bb lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceccba expref: -99 pid: 156822 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00000080:00200008:3.0:1787826101.684117:0:157293:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), request_mask 0, flags 0, foreign 0 00000080:00200000:3.0:1787826101.684128:0:157293:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:3.0:1787826101.684149:0:157293:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826101.684167:0:157293:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481600/0xbef42c0221d6f0bb lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceccba expref: -99 pid: 156822 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:3.0:1787826101.684182:0:157293:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481600/0xbef42c0221d6f0bb lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceccba expref: -99 pid: 156822 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00000080:00200000:3.0:1787826101.684189:0:157293:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826101.684196:0:157293:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:3.0:1787826101.684199:0:157293:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6f0bb 00000080:00010000:3.0:1787826101.684207:0:157293:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f0bb from it ffffb97500da3cb8 00010000:00010000:3.0:1787826101.684211:0:157293:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481600/0xbef42c0221d6f0bb lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceccba expref: -99 pid: 156822 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:3.0:1787826101.684220:0:157293:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481600/0xbef42c0221d6f0bb lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceccba expref: -99 pid: 156822 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00000080:00200000:3.0:1787826101.684379:0:157293:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:3.0:1787826101.684388:0:157293:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826101.684396:0:157293:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481600/0xbef42c0221d6f0bb lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceccba expref: -99 pid: 156822 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00010000:00010000:3.0:1787826101.684405:0:157293:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481600/0xbef42c0221d6f0bb lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceccba expref: -99 pid: 156822 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00000080:00200000:3.0:1787826101.684410:0:157293:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826101.684414:0:157293:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:3.0:1787826101.684417:0:157293:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6f0bb 00000080:00010000:3.0:1787826101.684420:0:157293:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f0bb from it ffffb97500da3bb0 00010000:00010000:3.0:1787826101.684424:0:157293:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481600/0xbef42c0221d6f0bb lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceccba expref: -99 pid: 156822 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00010000:00010000:3.0:1787826101.684431:0:157293:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481600/0xbef42c0221d6f0bb lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceccba expref: -99 pid: 156822 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00000080:00200000:3.0:1787826101.684439:0:157293:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 44 00000080:00200008:3.0:1787826101.684447:0:157293:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), flags 100000 00000080:00200080:3.0:1787826101.685054:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:3.0:1787826101.685070:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442d800 uarg=0000000000000000 00800000:00000004:3.0:1787826101.685077:0:157293:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9181c4d2b000: age 8163, max_age 8172 00000100:00100000:3.0:1787826101.685117:0:157293:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:157293:1874663042963072:192.168.203.147@tcp:41:lfs.0 00000100:00100000:3.0:1787826101.685193:0:157293:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d81a9480 going to sleep for 11 seconds 00000100:00100000:3.0:1787826101.692564:0:157293:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:157293:1874663042963072:192.168.203.147@tcp:41:lfs.0 00800000:00000004:3.0:1787826101.692591:0:157293:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0000-mdc-ffff9181c4d2b000: update ffff9181c87b0450 cache blocks 317234/353529 objects 1023646/1024000 00000080:00200080:3.0:1787826101.692695:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:3.0:1787826101.692712:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442d800 uarg=0000000000000000 00800000:00000004:3.0:1787826101.692720:0:157293:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9181c4d2b000: age 8163, max_age 8172 00000100:00100000:3.0:1787826101.692821:0:157293:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:157293:1874663042963200:192.168.203.147@tcp:41:lfs.0 00000100:00100000:0.0:1787826101.693064:0:157293:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d81a9480 going to sleep for 11 seconds 00000100:00100000:1.0:1787826101.701801:0:157293:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:157293:1874663042963200:192.168.203.147@tcp:41:lfs.0 00800000:00000004:1.0:1787826101.701830:0:157293:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0001-mdc-ffff9181c4d2b000: update ffff9181c87b1e08 cache blocks 317336/353529 objects 1023704/1024000 00000080:00200080:1.0:1787826101.701896:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.701913:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.701922:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.701926:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.701930:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.701934:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.701939:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.701942:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.701946:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.701949:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.701953:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.701956:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.701960:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.701964:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.701968:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.701972:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.701977:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.701981:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.701985:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.701989:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.701996:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.701999:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.702003:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.702006:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.702011:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.702015:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.702020:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.702023:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.702027:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.702030:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.702034:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.702038:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.702043:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.702047:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.702052:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.702056:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.702061:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.702065:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.702069:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.702072:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.702076:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.702080:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.702084:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.702087:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.702091:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.702095:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.702099:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.702103:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.702107:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.702111:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.702115:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.702120:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.702125:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.702129:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.702133:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.702136:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.702141:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.702145:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.702150:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.702154:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.702159:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.702162:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.702167:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.702170:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.702174:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.702177:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.702181:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.702185:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.702189:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.702192:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.702196:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.702199:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.702203:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.702206:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.702209:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.702212:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.702216:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.702219:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.702222:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.702225:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.702229:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.702232:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.702235:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.702238:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.702241:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.702245:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.702248:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.702251:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.702255:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.702259:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.702263:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.702266:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.702270:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.702273:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.702277:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.702280:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.702283:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.702286:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.702290:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.702294:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.702298:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.702301:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.702305:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.702309:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.702313:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.702316:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.702320:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.702324:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.702327:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.702330:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.702334:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.702337:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.702340:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.702344:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.702347:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.702351:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.702357:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.702360:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.702364:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.702367:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.702371:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.702374:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.702378:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.702381:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.702385:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00800000:00000080:1.0:1787826101.702388:0:157293:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.702392:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.702397:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00020000:00000004:1.0:1787826101.702405:0:157293:0:(obd_class.h:1090:obd_statfs()) lustre-OST0000-osc-ffff9181c4d2b000: age 8163, max_age 8172 00000100:00100000:1.0:1787826101.702442:0:157293:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:157293:1874663042963328:192.168.203.147@tcp:13:lfs.0 00000100:00100000:1.0:1787826101.702527:0:157293:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c55daa80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826101.719237:0:157293:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:157293:1874663042963328:192.168.203.147@tcp:13:lfs.0 00020000:00000004:1.0:1787826101.719626:0:157293:0:(obd_class.h:1116:obd_statfs()) lustre-OST0000-osc-ffff9181c4d2b000: update ffff9181c87b37c0 cache blocks 901356/958279 objects 261709/262144 00000080:00200080:1.0:1787826101.719706:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.719725:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00020000:00000004:1.0:1787826101.719732:0:157293:0:(obd_class.h:1090:obd_statfs()) lustre-OST0001-osc-ffff9181c4d2b000: age 8163, max_age 8172 00000100:00100000:1.0:1787826101.719921:0:157293:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:157293:1874663042963456:192.168.203.147@tcp:13:lfs.0 00000100:00100000:1.0:1787826101.719993:0:157293:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c55daa80 going to sleep for 11 seconds 00000100:00100000:1.0:1787826101.723607:0:157293:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:157293:1874663042963456:192.168.203.147@tcp:13:lfs.0 00020000:00000004:1.0:1787826101.723635:0:157293:0:(obd_class.h:1116:obd_statfs()) lustre-OST0001-osc-ffff9181c4d2b000: update ffff9181c87b5178 cache blocks 901360/958279 objects 261727/262144 00000080:00200080:1.0:1787826101.723698:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.723714:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.723721:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.723726:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.723730:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724110:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724122:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724128:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724133:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724137:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724141:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724145:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724149:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724152:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724156:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724160:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724164:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724168:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724172:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724177:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724181:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724184:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724189:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724192:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724197:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724201:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724205:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724208:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724212:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724216:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724220:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724224:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724228:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724231:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724235:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724239:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724243:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724261:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724266:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724270:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724274:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724277:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724281:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724284:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724288:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724292:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724296:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724299:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724303:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724307:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724311:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724315:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724319:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724323:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724327:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724330:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724335:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724338:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724342:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724346:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724350:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724354:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724357:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724360:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724364:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724367:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724371:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724374:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724378:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724381:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724385:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724388:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724392:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724395:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724398:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724402:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724405:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724409:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724412:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724416:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724420:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724429:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724435:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724439:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724442:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724445:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724449:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724453:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724456:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724459:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724463:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724466:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724483:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724487:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724491:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724494:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724497:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724500:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724504:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724508:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724511:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724515:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724530:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724535:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724540:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724543:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724547:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724550:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724554:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724557:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724561:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724564:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724568:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724571:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724574:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724578:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724582:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724585:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724589:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724592:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724595:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724598:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724602:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724605:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200080:1.0:1787826101.724609:0:157293:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe5fc7e2d0 00020000:00000080:1.0:1787826101.724612:0:157293:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181e053dc00 uarg=0000000000000000 00000080:00200008:1.0:1787826101.724627:0:157293:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), flags 100000 00000080:00200008:1.0:1787826101.744906:0:156396:0:(llite_lib.c:2427:ll_setattr_raw()) START file f73a.replay-single:[0x20001a9e3:0x1bd0:0x0](ffff9181d0807448) current size 0, valid attrs 41, mode 8000, uid 1, gid 0, new size 18446744071748348645, atime 1787826101.000000000, mtime 1787826101.000000000, ctime 1787826101.000000000 00000020:00100000:1.0:1787826101.744989:0:156396:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787826101.745003:0:156396:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c35b2680 x1874663042963712/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 488/512 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826101.745026:0:156396:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job multiop:a9ce6693-0634-46c5-bf21-5ca7d026d44e:156396:1874663042963712:192.168.203.147@tcp:36:multiop.0 00000100:00100000:1.0:1787826101.745107:0:156396:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c55daa80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826101.757432:0:156396:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job multiop:a9ce6693-0634-46c5-bf21-5ca7d026d44e:156396:1874663042963712:192.168.203.147@tcp:36:multiop.0 00000080:00200000:3.0:1787826101.757503:0:156396:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1bd0:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00010000:3.0:1787826101.757529:0:156396:0:(file.c:7065:ll_layout_intent()) ### lustre: requeue layout lock for file [0x20001a9e3:0x1bd0:0x0](ffff9181d0807448) 00000002:00010000:3.0:1787826101.757540:0:156396:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x20001a9e3:0x1bd0:0x0]) in obj [0x20001a9e3:0x1bd0:0x0], intent: layout flags 0 00010000:00010000:3.0:1787826101.757590:0:156396:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481400/0xbef42c0221d6f0c2 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x1bd0:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 156396 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826101.757606:0:156396:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481400/0xbef42c0221d6f0c2 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x1bd0:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x0 pid: 156396 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826101.757617:0:156396:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481400/0xbef42c0221d6f0c2 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x1bd0:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x0 pid: 156396 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826101.757636:0:156396:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job multiop:a9ce6693-0634-46c5-bf21-5ca7d026d44e:156396:1874663042963840:192.168.203.147@tcp:101:multiop.0 00000100:00100000:3.0:1787826101.757711:0:156396:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d81a9000 going to sleep for 11 seconds 00000100:00100000:3.0:1787826101.772710:0:156396:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job multiop:a9ce6693-0634-46c5-bf21-5ca7d026d44e:156396:1874663042963840:192.168.203.147@tcp:101:multiop.0 00010000:00010000:3.0:1787826101.775249:0:156396:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481400/0xbef42c0221d6f0c2 lrc: 4/1,0 mode: CR/CR res: [0x20001a9e3:0x1bd0:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cece88 expref: -99 pid: 156396 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826101.775269:0:156396:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481400/0xbef42c0221d6f0c2 lrc: 4/1,0 mode: CR/CR res: [0x20001a9e3:0x1bd0:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cece88 expref: -99 pid: 156396 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826101.775288:0:156396:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x400 disposition=0x1, status=0 req@ffff9181c5634a80 x1874663042963840/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 376/400 e 0 to 0 dl 1787826117 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'multiop.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826101.775300:0:156396:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: layout, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481400/0xbef42c0221d6f0c2 lrc: 3/1,0 mode: CR/CR res: [0x20001a9e3:0x1bd0:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cece88 expref: -99 pid: 156396 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00000002:00010000:3.0:1787826101.775312:0:156396:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481400/0xbef42c0221d6f0c2 lrc: 3/1,0 mode: CR/CR res: [0x20001a9e3:0x1bd0:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cece88 expref: -99 pid: 156396 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826101.775325:0:156396:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 16 res 8590043619/7120 (0 0) 00000080:00200000:3.0:1787826101.775332:0:156396:0:(namei.c:952:ll_intent_lock()) intent lock 1024 on i1 [0x20001a9e3:0x1bd0:0x0] suppgids 0 0: rc 0 00000080:00010000:3.0:1787826101.775349:0:156396:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x1bd0:0x0](ffff9181d0807448) for lock 0xbef42c0221d6f0c2 00000080:00010000:3.0:1787826101.775357:0:156396:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f0c2 from it ffffb97501d93af8 00010000:00010000:3.0:1787826101.775361:0:156396:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481400/0xbef42c0221d6f0c2 lrc: 3/1,0 mode: CR/CR res: [0x20001a9e3:0x1bd0:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cece88 expref: -99 pid: 156396 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826101.775371:0:156396:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481400/0xbef42c0221d6f0c2 lrc: 2/0,0 mode: CR/CR res: [0x20001a9e3:0x1bd0:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cece88 expref: -99 pid: 156396 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826101.775394:0:156396:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481400/0xbef42c0221d6f0c2 lrc: 2/1,0 mode: CR/CR res: [0x20001a9e3:0x1bd0:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cece88 expref: -99 pid: 156396 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826101.775405:0:156396:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481400/0xbef42c0221d6f0c2 lrc: 2/1,0 mode: CR/CR res: [0x20001a9e3:0x1bd0:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cece88 expref: -99 pid: 156396 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00010000:3.0:1787826101.775414:0:156396:0:(file.c:6948:ll_layout_lock_set()) ### file [0x20001a9e3:0x1bd0:0x0](0000000052299c50) being reconfigured ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481400/0xbef42c0221d6f0c2 lrc: 3/1,0 mode: CR/CR res: [0x20001a9e3:0x1bd0:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cece88 expref: -99 pid: 156396 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826101.775445:0:156396:0:(file.c:6843:ll_layout_conf()) [0x20001a9e3:0x1bd0:0x0]: layout version change: 4294967294 -> 0 00010000:00010000:3.0:1787826101.775450:0:156396:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481400/0xbef42c0221d6f0c2 lrc: 3/1,0 mode: CR/CR res: [0x20001a9e3:0x1bd0:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00cece88 expref: -99 pid: 156396 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826101.775457:0:156396:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481400/0xbef42c0221d6f0c2 lrc: 2/0,0 mode: CR/CR res: [0x20001a9e3:0x1bd0:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00cece88 expref: -99 pid: 156396 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00020000:00200000:3.0:1787826101.775486:0:156396:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000100:00100000:3.0:1787826101.775565:0:2391:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787826101.775579:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2391:1874663042963968:192.168.203.147@tcp:2:multiop.0 00000100:00100000:3.0:1787826101.791160:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2391:1874663042963968:192.168.203.147@tcp:2:multiop.0 00000080:00200000:1.0:1787826101.791204:0:156396:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1bd0:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200008:1.0:1787826101.791221:0:156396:0:(llite_lib.c:2686:ll_setattr_raw()) COMPLETED file f73a.replay-single:[0x20001a9e3:0x1bd0:0x0](ffff9181d0807448) current size 0, valid attrs 10000041, mode 8000, uid 1, gid 0, new size 18446744071748348645, atime 1787826101.000000000, mtime 1787826101.000000000, ctime 1787826101.000000000, rc 0 00000080:00200008:1.0:1787826101.791259:0:156396:0:(file.c:6296:ll_getattr_dentry()) START file f73a.replay-single:[0x20001a9e3:0x1bd0:0x0](ffff9181d0807448), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1787826101.791267:0:156396:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x20001a9e3:0x1bd0:0x0](ffff9181d0807448),name=f73a.replay-single 00000002:00010000:1.0:1787826101.791285:0:156396:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x20001a9e3:0x1bd0:0x0]) in obj [0x20001a9e3:0x1bd0:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826101.791300:0:156396:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 30 res 8590043619/7120 (0 0) 00010000:00010000:1.0:1787826101.791345:0:156396:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6f0c9 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x1bd0:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 156396 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826101.791354:0:156396:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6f0c9 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x1bd0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 156396 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826101.791364:0:156396:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6f0c9 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x1bd0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 156396 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826101.791384:0:156396:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job multiop:a9ce6693-0634-46c5-bf21-5ca7d026d44e:156396:1874663042964096:192.168.203.147@tcp:101:multiop.0 00000100:00100000:1.0:1787826101.791461:0:156396:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c55da300 going to sleep for 11 seconds 00000100:00100000:3.0:1787826101.794903:0:156396:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job multiop:a9ce6693-0634-46c5-bf21-5ca7d026d44e:156396:1874663042964096:192.168.203.147@tcp:101:multiop.0 00010000:00010000:3.0:1787826101.794929:0:156396:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6f0c9 lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x1bd0:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cece96 expref: -99 pid: 156396 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826101.794941:0:156396:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6f0c9 lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1bd0:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cece96 expref: -99 pid: 156396 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826101.794952:0:156396:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6f0c9 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bd0:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cece96 expref: -99 pid: 156396 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826101.794962:0:156396:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6f0c9 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bd0:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cece96 expref: -99 pid: 156396 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826101.794974:0:156396:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6f0c9 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1bd0:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cece96 expref: -99 pid: 156396 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826101.794983:0:156396:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6f0c9 lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1bd0:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cece96 expref: -99 pid: 156396 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826101.794990:0:156396:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6f0c9 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bd0:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cece96 expref: -99 pid: 156396 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826101.795000:0:156396:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c35b0000 x1874663042964096/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787826117 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'multiop.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826101.795011:0:156396:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: getattr, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6f0c9 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1bd0:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cece96 expref: -99 pid: 156396 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:3.0:1787826101.795025:0:156396:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6f0c9 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1bd0:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cece96 expref: -99 pid: 156396 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826101.795038:0:156396:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/7120 (0 0) 00000080:00200000:3.0:1787826101.795044:0:156396:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x1bd0:0x0] suppgids 0 0: rc 0 00000080:00200000:3.0:1787826101.796640:0:156396:0:(file.c:6843:ll_layout_conf()) [0x20001a9e3:0x1bd0:0x0]: layout version change: 0 -> 0 00000080:00010000:3.0:1787826101.796652:0:156396:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x1bd0:0x0](ffff9181d0807448) for lock 0xbef42c0221d6f0c9 00000080:00010000:3.0:1787826101.796661:0:156396:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x1bd0:0x0](ffff9181d0807448) 00000080:00010000:3.0:1787826101.796663:0:156396:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f0c9 from it ffffb97501d93cd8 00010000:00010000:3.0:1787826101.796668:0:156396:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6f0c9 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1bd0:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00cece96 expref: -99 pid: 156396 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826101.796679:0:156396:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4600/0xbef42c0221d6f0c9 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1bd0:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00cece96 expref: -99 pid: 156396 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826101.796724:0:156396:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1bd0:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:3.0:1787826101.796733:0:156396:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x20001a9e3:0x1bd0:0x0] 00000008:00010000:3.0:1787826101.796775:0:156396:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: a 00000008:00010000:3.0:1787826101.796779:0:156396:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff9181c12ec430, osc lock ffff9181c7d41560, flags 0x400001002 00010000:00010000:3.0:1787826101.796797:0:156396:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff9181c4d2b000 lock: 00000000a58f462c/0xbef42c0221d6f083 lrc: 2/1,0 mode: PR/PR res: [0x280000401:0x1842:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x878a5eea00cec9aa expref: -99 pid: 156398 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826101.796811:0:156396:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 18446744073709551615) ns: lustre-OST0000-osc-ffff9181c4d2b000 lock: 00000000a58f462c/0xbef42c0221d6f083 lrc: 2/1,0 mode: PR/PR res: [0x280000401:0x1842:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x878a5eea00cec9aa expref: -99 pid: 156398 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000020:00200000:3.0:1787826101.796822:0:156396:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00010000:00010000:3.0:1787826101.796825:0:156396:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9181c4d2b000 lock: 00000000a58f462c/0xbef42c0221d6f083 lrc: 5/1,0 mode: PR/PR res: [0x280000401:0x1842:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x878a5eea00cec9aa expref: -99 pid: 156398 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826101.796838:0:156396:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff9181c4d2b000 lock: 00000000a58f462c/0xbef42c0221d6f083 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x1842:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x878a5eea00cec9aa expref: -99 pid: 156398 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000020:00200000:3.0:1787826101.796848:0:156396:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1787826101.796863:0:156396:0:(file.c:1775:ll_merge_attr_nolock()) [0x20001a9e3:0x1bd0:0x0] updating i_size 0 i_blocks 0 00000020:00010000:3.0:1787826101.796868:0:156396:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9181c7d6a750 (ffff9181c6e422d8/0) at cl_lock_release():245 00000020:00010000:3.0:1787826101.796871:0:156396:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9181c7d6a750 (ffff9181c6e422d8/0) at cl_lock_cancel():142 00000020:00010000:3.0:1787826101.796875:0:156396:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9181c12ec430 (ffff9181c44053f8/1) at cl_lock_cancel():142 00000020:00010000:3.0:1787826101.796879:0:156396:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9181c7d6a750 (ffff9181c6e422d8/0) at cl_lock_fini():72 00000020:00010000:3.0:1787826101.796881:0:156396:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9181c12ec430 (ffff9181c6e422d8/1) at cl_lock_fini():72 00000080:00200000:3.0:1787826101.796894:0:156396:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1bd0:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200008:3.0:1787826101.796919:0:156396:0:(file.c:479:ll_file_release()) START file f73a.replay-single:[0x20001a9e3:0x1bd0:0x0](ffff9181d0807448), flags 100002 00000002:00100000:3.0:1787826101.796970:0:156396:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181c71f2300 x1874663042950784/t335007449219(335007449219) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 592/608 e 0 to 0 dl 1787826107 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00000020:00100000:3.0:1787826101.796981:0:156396:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:3.0:1787826101.796986:0:156396:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c5636300 x1874663042964352/t0(0) o35->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:3.0:1787826101.797006:0:156396:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636300 pname:cluuid:pid:xid:nid:opc:job multiop:a9ce6693-0634-46c5-bf21-5ca7d026d44e:156396:1874663042964352:192.168.203.147@tcp:35:multiop.0 00000100:00100000:3.0:1787826101.797084:0:156396:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d81a9000 going to sleep for 11 seconds 00000100:00100000:3.0:1787826101.804149:0:156396:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636300 pname:cluuid:pid:xid:nid:opc:job multiop:a9ce6693-0634-46c5-bf21-5ca7d026d44e:156396:1874663042964352:192.168.203.147@tcp:35:multiop.0 00000002:00100000:3.0:1787826101.804195:0:156396:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181c71f2300 x1874663042950784/t335007449219(335007449219) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 592/608 e 0 to 0 dl 1787826107 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00000080:00200000:3.0:1787826101.804273:0:156396:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1bd0:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826101.804286:0:156396:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826101.804317:0:156396:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1bd0:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826101.804325:0:156396:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1bd0:0x0](ffff9181d0807448) 00000008:00010000:3.0:1787826101.804364:0:156396:0:(osc_object.c:202:osc_object_ast_clear()) ### update lock size 0 blocks 0 [cma]time: 0 0 0 by oinfo size 0 blocks 0 [cma]time 1787826101 0 0 ns: lustre-OST0000-osc-ffff9181c4d2b000 lock: 00000000a58f462c/0xbef42c0221d6f083 lrc: 1/0,0 mode: PR/PR res: [0x280000401:0x1842:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x878a5eea00cec9aa expref: -99 pid: 156398 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787826101.810033:0:156251:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826101.810060:0:156251:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826101.810080:0:156251:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481600/0xbef42c0221d6f0bb lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceccba expref: -99 pid: 156822 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00010000:00010000:0.0:1787826101.810516:0:156251:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481600/0xbef42c0221d6f0bb lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceccba expref: -99 pid: 156822 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00000080:00200000:0.0:1787826101.810535:0:156251:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826101.810543:0:156251:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826101.810547:0:156251:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6f0bb 00000080:00010000:0.0:1787826101.810554:0:156251:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f0bb from it ffffb97501603b58 00010000:00010000:0.0:1787826101.810558:0:156251:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481600/0xbef42c0221d6f0bb lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceccba expref: -99 pid: 156822 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00010000:00010000:0.0:1787826101.810565:0:156251:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481600/0xbef42c0221d6f0bb lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceccba expref: -99 pid: 156822 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00000080:00200000:0.0:1787826101.810581:0:156251:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826101.810598:0:156251:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f73a.replay-single, dir=[0x200000007:0x1:0x0](ffff9181c41acb08), flags=9 00000080:00200000:0.0:1787826101.810602:0:156251:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f73a.replay-single, dir=[0x200000007:0x1:0x0](ffff9181c41acb08), intent=getattr 00000002:00010000:0.0:1787826101.810615:0:156251:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f73a.replay-single,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787826101.810669:0:156251:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6f0d0 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 156251 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826101.810676:0:156251:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6f0d0 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 156251 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826101.810684:0:156251:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6f0d0 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 156251 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826101.810941:0:156251:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job bash:a9ce6693-0634-46c5-bf21-5ca7d026d44e:156251:1874663042964480:192.168.203.147@tcp:101:bash.0 00000100:00100000:0.0:1787826101.811073:0:156251:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d800c840 going to sleep for 11 seconds 00000100:00100000:0.0:1787826101.823491:0:156251:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job bash:a9ce6693-0634-46c5-bf21-5ca7d026d44e:156251:1874663042964480:192.168.203.147@tcp:101:bash.0 00010000:00010000:0.0:1787826101.823513:0:156251:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6f0d0 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 156251 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826101.823526:0:156251:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6f0d0 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 156251 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826101.823543:0:156251:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6f0d0 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 156251 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826101.823550:0:156251:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6f0d0 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 156251 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826101.823558:0:156251:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6f0d0 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 156251 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826101.823564:0:156251:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0513600 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787826101.823568:0:156251:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6f0d0 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 156251 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826101.823574:0:156251:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6f0d0 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 156251 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826101.823580:0:156251:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6f0d0 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x20e400000000 pid: 156251 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826101.823587:0:156251:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787826101.823592:0:156251:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6f0d0 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 156251 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826101.823598:0:156251:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513600/0xbef42c0221d6f0d0 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 156251 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826101.823611:0:156251:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c55a7100 x1874663042964480/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 592/584 e 0 to 0 dl 1787826117 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'bash.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787826101.823625:0:156251:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787826101.823651:0:156251:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481600/0xbef42c0221d6f0bb lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceccba expref: -99 pid: 156822 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00010000:00010000:0.0:1787826101.823664:0:156251:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481600/0xbef42c0221d6f0bb lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceccba expref: -99 pid: 156822 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00000080:00010000:0.0:1787826101.823672:0:156251:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f0bb from it ffffb97501603930 00010000:00010000:0.0:1787826101.823677:0:156251:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481600/0xbef42c0221d6f0bb lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceccba expref: -99 pid: 156822 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00010000:00010000:0.0:1787826101.823684:0:156251:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181c3481600/0xbef42c0221d6f0bb lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceccba expref: -99 pid: 156822 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00010000:00010000:3.0:1787826102.113845:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826105.185649:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826105.889146:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042964736:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826105.889341:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042964992:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826105.889377:0:2389:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:1.0:1787826105.889383:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7528148b-9bee-4d31-b597-8aae3b0b4982:2389:1874663042964608:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826105.889410:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042964864:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826105.889429:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042965120:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826105.902682:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042964992:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826105.902725:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7528148b-9bee-4d31-b597-8aae3b0b4982:2389:1874663042964608:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826105.903833:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042965120:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826105.905514:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042964736:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826105.905557:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042964864:192.168.203.147@tcp:400:kworker.0 00010000:00010000:0.0:1787826107.242068:0:33329:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826107.533642:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826107.534025:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049268736 00000100:00100000:0.0:1787826107.534036:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049268736 00000100:00100000:0.0:1787826107.534090:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9267 00000100:00100000:0.0:1787826107.534099:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:LOV_OSC_UUID+4:8403:x1874663049268736:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826107.534116:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-OST0000-osc-ffff9181c4d2b000 lock: 00000000a58f462c/0xbef42c0221d6f083 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x1842:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x420000010000 nid: local remote: 0x878a5eea00cec9aa expref: -99 pid: 156398 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787826107.534202:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826107.534208:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:LOV_OSC_UUID+4:8403:x1874663049268736:12345-192.168.203.147@tcp:104: Request processed in 111us (581us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826107.534214:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9267 00010000:00010000:3.0:1787826107.534370:0:15160:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826107.534377:0:15160:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0000-osc-ffff9181c4d2b000 lock: 00000000a58f462c/0xbef42c0221d6f083 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x1842:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x420000010000 nid: local remote: 0x878a5eea00cec9aa expref: -99 pid: 156398 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826107.534386:0:15160:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e06f7400 already unused, calling callback (ffffffffc10ec300) 00010000:00010000:3.0:1787826107.534391:0:15160:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0000-osc-ffff9181c4d2b000 lock: 00000000a58f462c/0xbef42c0221d6f083 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x1842:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x428400010000 nid: local remote: 0x878a5eea00cec9aa expref: -99 pid: 156398 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826107.534440:0:15160:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0000-osc-ffff9181c4d2b000 lock: 00000000a58f462c/0xbef42c0221d6f083 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x1842:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4c29400010020 nid: local remote: 0x878a5eea00cec9aa expref: -99 pid: 156398 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826107.534447:0:15160:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826107.534473:0:15160:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0000-osc-ffff9181c4d2b000 lock: 00000000a58f462c/0xbef42c0221d6f083 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x1842:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4c29400010020 nid: local remote: 0x878a5eea00cec9aa expref: -99 pid: 156398 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826107.534482:0:15160:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0000-osc-ffff9181c4d2b000 lock: 00000000a58f462c/0xbef42c0221d6f083 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x1842:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4c29400010020 nid: local remote: 0x878a5eea00cec9aa expref: -99 pid: 156398 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826107.534503:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826107.534505:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826107.534507:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826107.534521:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2391:1874663042965248:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1787826107.551563:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2391:1874663042965248:192.168.203.147@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826108.257042:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:1.0:1787826108.795717:0:157883:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == replay-single test 73b: open(O_CREAT), unlink, replay, reconnect at open_replay reply, close ========================================================== 06:21:47 (1787826107) 00000100:00100000:3.0:1787826108.995874:0:157888:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9181c4d2b000: committing for last_committed 339302416386 gen 11 00000100:00100000:3.0:1787826108.995907:0:157888:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff9181c71f1880 x1874663042965504/t339302416387(339302416387) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826124 ref 3 fl Rpc:RPQU/604/0 rc 0/0 job:'multiop.0' uid:0 gid:0 projid:0 00000100:00100000:3.0:1787826108.995921:0:157888:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f1880 pname:cluuid:pid:xid:nid:opc:job multiop:a9ce6693-0634-46c5-bf21-5ca7d026d44e:157888:1874663042965504:192.168.203.147@tcp:101:multiop.0 00010000:00010000:3.0:1787826108.995944:0:157888:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7600/0xbef42c0221d6f0d7 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceced5 expref: -99 pid: 157888 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826108.995969:0:157888:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7600/0xbef42c0221d6f0d7 lrc: 4/0,1 mode: --/PR res: [0x20001a9e3:0x1bd2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceced5 expref: -99 pid: 157888 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826108.995977:0:157888:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7600/0xbef42c0221d6f0d7 lrc: 4/0,1 mode: --/PR res: [0x20001a9e3:0x1bd2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceced5 expref: -99 pid: 157888 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826108.995988:0:157888:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7600/0xbef42c0221d6f0d7 lrc: 4/0,1 mode: PR/PR res: [0x20001a9e3:0x1bd2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceced5 expref: -99 pid: 157888 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826108.995999:0:157888:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7600/0xbef42c0221d6f0d7 lrc: 4/0,1 mode: PR/PR res: [0x20001a9e3:0x1bd2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceced5 expref: -99 pid: 157888 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826108.996012:0:157888:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7600/0xbef42c0221d6f0d7 lrc: 5/1,1 mode: PR/PR res: [0x20001a9e3:0x1bd2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceced5 expref: -99 pid: 157888 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826108.996026:0:157888:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7600/0xbef42c0221d6f0d7 lrc: 5/1,1 mode: PR/PR res: [0x20001a9e3:0x1bd2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceced5 expref: -99 pid: 157888 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826108.996034:0:157888:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7600/0xbef42c0221d6f0d7 lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bd2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceced5 expref: -99 pid: 157888 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826108.996043:0:157888:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9181c71f1880 x1874663042965504/t339302416387(339302416387) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826124 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'multiop.0' uid:0 gid:0 projid:0 00000002:00100000:3.0:1787826108.996055:0:157888:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c71f1880 x1874663042965504/t339302416387(339302416387) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826124 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'multiop.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826108.996068:0:157888:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7600/0xbef42c0221d6f0d7 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1bd2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceced5 expref: -99 pid: 157888 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:3.0:1787826108.996081:0:157888:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7600/0xbef42c0221d6f0d7 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1bd2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceced5 expref: -99 pid: 157888 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826108.996091:0:157888:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/7122 (0 0) 00000080:00200000:3.0:1787826108.996097:0:157888:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:3.0:1787826108.996109:0:157888:0:(namei.c:1628:ll_create_it()) VFS Op:name=f73b.replay-single, dir=[0x200000007:0x1:0x0](ffff9181c41acb08), intent=open|creat 00000080:00200000:3.0:1787826108.996135:0:157888:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x20001a9e3:0x1bd2:0x0](ffff9181d080cb08) 00000080:00200000:3.0:1787826108.996205:0:157888:0:(file.c:6843:ll_layout_conf()) [0x20001a9e3:0x1bd2:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:3.0:1787826108.996210:0:157888:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x20001a9e3:0x1bd2:0x0](ffff9181d080cb08) 00000080:00010000:3.0:1787826108.996213:0:157888:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x1bd2:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f0d7 00000080:00010000:3.0:1787826108.996221:0:157888:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x1bd2:0x0](ffff9181d080cb08) 00000080:00200008:3.0:1787826108.996234:0:157888:0:(file.c:1090:ll_file_open()) START file f73b.replay-single:[0x20001a9e3:0x1bd2:0x0](ffff9181d080cb08), flags 100102 00000002:00100000:3.0:1787826108.996254:0:157888:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c71f1880 x1874663042965504/t339302416387(339302416387) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826124 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'multiop.0' uid:0 gid:0 projid:0 00000080:00010000:3.0:1787826108.996292:0:157888:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f0d7 from it ffff9181c553e840 00010000:00010000:3.0:1787826108.996296:0:157888:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7600/0xbef42c0221d6f0d7 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1bd2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00ceced5 expref: -99 pid: 157888 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826108.996306:0:157888:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7600/0xbef42c0221d6f0d7 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1bd2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00ceced5 expref: -99 pid: 157888 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826108.996326:0:157888:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x20001a9e3:0x1bd2:0x0](ffff9181d080cb08), inode mode 81a4 mask 40 00000080:00200000:0.0:1787826109.019974:0:157893:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826109.020009:0:157893:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826109.020086:0:157893:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6f0de lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 157893 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826109.020097:0:157893:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6f0de lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 157893 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826109.020107:0:157893:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6f0de lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 157893 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826109.020130:0:157893:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job rm:a9ce6693-0634-46c5-bf21-5ca7d026d44e:157893:1874663042965632:192.168.203.147@tcp:101:rm.0 00000100:00100000:0.0:1787826109.020705:0:157893:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c7da1c00 going to sleep for 11 seconds 00000100:00100000:3.0:1787826109.048172:0:157893:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job rm:a9ce6693-0634-46c5-bf21-5ca7d026d44e:157893:1874663042965632:192.168.203.147@tcp:101:rm.0 00010000:00010000:3.0:1787826109.048199:0:157893:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6f0de lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceceea expref: -99 pid: 157893 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826109.048212:0:157893:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6f0de lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceceea expref: -99 pid: 157893 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826109.048222:0:157893:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6f0de lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceceea expref: -99 pid: 157893 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826109.048233:0:157893:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6f0de lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceceea expref: -99 pid: 157893 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826109.048245:0:157893:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6f0de lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceceea expref: -99 pid: 157893 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826109.048253:0:157893:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6f0de lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceceea expref: -99 pid: 157893 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826109.048261:0:157893:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6f0de lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceceea expref: -99 pid: 157893 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826109.048272:0:157893:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c55a7100 x1874663042965632/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826125 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826109.048292:0:157893:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6f0de lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceceea expref: -99 pid: 157893 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826109.048305:0:157893:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:3.0:1787826109.048312:0:157893:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:3.0:1787826109.048341:0:157893:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 69632 00000080:00010000:3.0:1787826109.048347:0:157893:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6f0de 00000080:00010000:3.0:1787826109.048353:0:157893:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:3.0:1787826109.048356:0:157893:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f0de from it ffffb9750147bb58 00010000:00010000:3.0:1787826109.048359:0:157893:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6f0de lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceceea expref: -99 pid: 157893 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826109.048369:0:157893:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6f0de lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceceea expref: -99 pid: 157893 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826109.048394:0:157893:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826109.048405:0:157893:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f73b.replay-single, flags=8 00000080:00200008:3.0:1787826109.048430:0:157893:0:(file.c:6296:ll_getattr_dentry()) START file f73b.replay-single:[0x20001a9e3:0x1bd2:0x0](ffff9181d080cb08), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1787826109.048437:0:157893:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x20001a9e3:0x1bd2:0x0](ffff9181d080cb08),name=f73b.replay-single 00000002:00010000:3.0:1787826109.048457:0:157893:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x20001a9e3:0x1bd2:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826109.048640:0:157893:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7600/0xbef42c0221d6f0d7 lrc: 2/1,0 mode: PR/PR res: [0x20001a9e3:0x1bd2:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00ceced5 expref: -99 pid: 157888 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826109.048653:0:157893:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7600/0xbef42c0221d6f0d7 lrc: 2/1,0 mode: PR/PR res: [0x20001a9e3:0x1bd2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00ceced5 expref: -99 pid: 157888 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826109.048660:0:157893:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826109.048665:0:157893:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x1bd2:0x0](ffff9181d080cb08) 00000080:00010000:3.0:1787826109.048668:0:157893:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x1bd2:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f0d7 00000080:00010000:3.0:1787826109.048673:0:157893:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f0d7 from it ffffb9750147bcb8 00010000:00010000:3.0:1787826109.048676:0:157893:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7600/0xbef42c0221d6f0d7 lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1bd2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00ceced5 expref: -99 pid: 157888 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826109.048684:0:157893:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7600/0xbef42c0221d6f0d7 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1bd2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00ceced5 expref: -99 pid: 157888 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:3.0:1787826109.048716:0:157893:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1bd2:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:3.0:1787826109.048728:0:157893:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x20001a9e3:0x1bd2:0x0] 00000008:00010000:3.0:1787826109.049134:0:157893:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: a 00000008:00010000:3.0:1787826109.049139:0:157893:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff9181c46709b0, osc lock ffff9181c7d41560, flags 0x400001002 00010000:00010000:3.0:1787826109.049187:0:157893:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff9181c4d2b000 lock: 00000000ffd0d697/0xbef42c0221d6f0e5 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x1862:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 157893 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826109.049200:0:157893:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x400001000 ns: lustre-OST0000-osc-ffff9181c4d2b000 lock: 00000000ffd0d697/0xbef42c0221d6f0e5 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x1862:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 157893 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:3.0:1787826109.049239:0:157893:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000100:00100000:3.0:1787826109.049255:0:2391:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787826109.049270:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2391:1874663042965760:192.168.203.147@tcp:101:rm.0 00000080:00200000:0.0:1787826109.084415:0:157893:0:(file.c:1775:ll_merge_attr_nolock()) [0x20001a9e3:0x1bd2:0x0] updating i_size 0 i_blocks 0 00000020:00010000:0.0:1787826109.084429:0:157893:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9181c7d6a750 (ffff9181c6e422d8/0) at cl_lock_release():245 00000020:00010000:0.0:1787826109.084432:0:157893:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9181c7d6a750 (ffff9181c6e422d8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1787826109.084437:0:157893:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9181c46709b0 (ffff9181c44053f8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1787826109.084441:0:157893:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9181c7d6a750 (ffff9181c6e422d8/0) at cl_lock_fini():72 00000020:00010000:0.0:1787826109.084444:0:157893:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9181c46709b0 (ffff9181c6e422d8/1) at cl_lock_fini():72 00000080:00200000:0.0:1787826109.084638:0:157893:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1bd2:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826109.084695:0:157893:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826109.084714:0:157893:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826109.084727:0:157893:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6f0de lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceceea expref: -99 pid: 157893 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826109.084739:0:157893:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6f0de lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceceea expref: -99 pid: 157893 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826109.084769:0:157893:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826109.084776:0:157893:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826109.084780:0:157893:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6f0de 00000080:00010000:0.0:1787826109.084786:0:157893:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f0de from it ffffb9750147bc10 00010000:00010000:0.0:1787826109.084789:0:157893:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6f0de lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceceea expref: -99 pid: 157893 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826109.084797:0:157893:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6f0de lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceceea expref: -99 pid: 157893 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787826109.084807:0:157893:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826109.084815:0:157893:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f73b.replay-single, flags=0 00000080:00200000:0.0:1787826109.084820:0:157893:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826109.084825:0:157893:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826109.084830:0:157893:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6f0de lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceceea expref: -99 pid: 157893 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787826109.084838:0:157893:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6f0de lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceceea expref: -99 pid: 157893 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787826109.084844:0:157893:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826109.084847:0:157893:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826109.084850:0:157893:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6f0de 00000080:00010000:0.0:1787826109.084852:0:157893:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f0de from it ffffb9750147bd68 00010000:00010000:0.0:1787826109.084856:0:157893:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6f0de lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceceea expref: -99 pid: 157893 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787826109.084864:0:157893:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6f0de lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceceea expref: -99 pid: 157893 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787826109.084871:0:157893:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 3 00000080:00200000:0.0:1787826109.084875:0:157893:0:(namei.c:2418:ll_unlink()) VFS Op:name=f73b.replay-single, dir=[0x200000007:0x1:0x0](ffff9181c41acb08) 00010000:00010000:0.0:1787826109.084899:0:157893:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6f0de lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ceceea expref: -99 pid: 157893 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787826109.084922:0:157893:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7600/0xbef42c0221d6f0d7 lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1bd2:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x878a5eea00ceced5 expref: -99 pid: 157888 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787826109.084932:0:157893:0:(vvp_object.c:117:vvp_conf_set()) [0x20001a9e3:0x1bd2:0x0]: losing layout lock 00010000:00010000:0.0:1787826109.084959:0:157893:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7600/0xbef42c0221d6f0d7 lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x1bd2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00ceced5 expref: -99 pid: 157888 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787826109.084967:0:157893:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6f0de lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ceceea expref: -99 pid: 157893 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787826109.084973:0:157893:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:0.0:1787826109.084976:0:157893:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7600/0xbef42c0221d6f0d7 lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x1bd2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00ceced5 expref: -99 pid: 157888 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787826109.084991:0:157893:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0512000/0xbef42c0221d6f0de lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ceceea expref: -99 pid: 157893 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000020:00100000:0.0:1787826109.085007:0:157893:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787826109.085015:0:157893:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c55a4380 x1874663042966016/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 504/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826109.085035:0:157893:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job rm:a9ce6693-0634-46c5-bf21-5ca7d026d44e:157893:1874663042966016:192.168.203.147@tcp:36:rm.0 00000100:00100000:1.0:1787826109.097199:0:157893:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c7da16c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826109.120253:0:157893:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job rm:a9ce6693-0634-46c5-bf21-5ca7d026d44e:157893:1874663042966016:192.168.203.147@tcp:36:rm.0 00010000:00010000:0.0:1787826110.310076:0:33329:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200008:1.0:1787826110.414903:0:157908:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1787826110.414922:0:157908:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826110.414954:0:157908:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826110.415682:0:157908:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5a00/0xbef42c0221d6f0ec lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 157908 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826110.415707:0:157908:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5a00/0xbef42c0221d6f0ec lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 157908 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826110.415720:0:157908:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5a00/0xbef42c0221d6f0ec lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 157908 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826110.415782:0:157908:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2300 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:157908:1874663042966144:192.168.203.147@tcp:101:lfs.0 00000100:00100000:1.0:1787826110.415886:0:157908:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c55daf00 going to sleep for 11 seconds 00000100:00100000:0.0:1787826110.425528:0:157908:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9181c4d2b000: committing for last_committed 339302416388 gen 11 00000100:00100000:0.0:1787826110.425552:0:157908:0:(client.c:3174:ptlrpc_free_committed()) @@@ keeping (FL_REPLAY) req@ffff9181c71f1880 x1874663042965504/t339302416387(339302416387) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826124 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00000100:00100000:0.0:1787826110.425582:0:157908:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2300 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:157908:1874663042966144:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787826110.425611:0:157908:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5a00/0xbef42c0221d6f0ec lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cecf14 expref: -99 pid: 157908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826110.425624:0:157908:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5a00/0xbef42c0221d6f0ec lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cecf14 expref: -99 pid: 157908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826110.425636:0:157908:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5a00/0xbef42c0221d6f0ec lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cecf14 expref: -99 pid: 157908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826110.425647:0:157908:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5a00/0xbef42c0221d6f0ec lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cecf14 expref: -99 pid: 157908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826110.425661:0:157908:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5a00/0xbef42c0221d6f0ec lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cecf14 expref: -99 pid: 157908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826110.425669:0:157908:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5a00/0xbef42c0221d6f0ec lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cecf14 expref: -99 pid: 157908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826110.425683:0:157908:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5a00/0xbef42c0221d6f0ec lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cecf14 expref: -99 pid: 157908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826110.425692:0:157908:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c35b2300 x1874663042966144/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826126 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787826110.425706:0:157908:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5a00/0xbef42c0221d6f0ec lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cecf14 expref: -99 pid: 157908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826110.425722:0:157908:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:0.0:1787826110.425731:0:157908:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1787826110.425782:0:157908:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 69632 00000080:00010000:0.0:1787826110.425791:0:157908:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6f0ec 00000080:00010000:0.0:1787826110.425797:0:157908:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826110.425800:0:157908:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f0ec from it ffffb975014fbcb8 00010000:00010000:0.0:1787826110.425804:0:157908:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5a00/0xbef42c0221d6f0ec lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cecf14 expref: -99 pid: 157908 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826110.425812:0:157908:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5a00/0xbef42c0221d6f0ec lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cecf14 expref: -99 pid: 157908 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:0.0:1787826110.426647:0:157908:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1787826110.426657:0:157908:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826110.426678:0:157908:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826110.426692:0:157908:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5a00/0xbef42c0221d6f0ec lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cecf14 expref: -99 pid: 157908 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826110.426711:0:157908:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5a00/0xbef42c0221d6f0ec lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cecf14 expref: -99 pid: 157908 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826110.426718:0:157908:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826110.426724:0:157908:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826110.426727:0:157908:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6f0ec 00000080:00010000:0.0:1787826110.426732:0:157908:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f0ec from it ffffb975014fbcb8 00010000:00010000:0.0:1787826110.426735:0:157908:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5a00/0xbef42c0221d6f0ec lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cecf14 expref: -99 pid: 157908 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826110.426769:0:157908:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5a00/0xbef42c0221d6f0ec lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cecf14 expref: -99 pid: 157908 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787826110.426913:0:157908:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826110.426923:0:157908:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826110.426931:0:157908:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5a00/0xbef42c0221d6f0ec lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cecf14 expref: -99 pid: 157908 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787826110.426940:0:157908:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5a00/0xbef42c0221d6f0ec lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cecf14 expref: -99 pid: 157908 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787826110.426947:0:157908:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826110.426952:0:157908:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826110.426954:0:157908:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6f0ec 00000080:00010000:0.0:1787826110.426958:0:157908:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f0ec from it ffffb975014fbbb0 00010000:00010000:0.0:1787826110.426961:0:157908:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5a00/0xbef42c0221d6f0ec lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cecf14 expref: -99 pid: 157908 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787826110.426970:0:157908:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5a00/0xbef42c0221d6f0ec lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cecf14 expref: -99 pid: 157908 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787826110.426979:0:157908:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 44 00000080:00200008:0.0:1787826110.426987:0:157908:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), flags 100000 00000080:00200080:0.0:1787826110.427112:0:157908:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffd45112790 00800000:00000080:0.0:1787826110.427131:0:157908:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c343d400 uarg=0000000000000000 00800000:00000004:0.0:1787826110.427139:0:157908:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9181c4d2b000: age 8173, max_age 8181 00000100:00100000:0.0:1787826110.427189:0:157908:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:157908:1874663042966272:192.168.203.147@tcp:41:lfs.0 00000100:00100000:0.0:1787826110.427453:0:157908:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c7da1480 going to sleep for 11 seconds 00000100:00100000:0.0:1787826110.430683:0:157908:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:157908:1874663042966272:192.168.203.147@tcp:41:lfs.0 00800000:00000004:0.0:1787826110.430719:0:157908:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0000-mdc-ffff9181c4d2b000: update ffff9181c87b0450 cache blocks 317234/353529 objects 1023646/1024000 00000080:00200080:0.0:1787826110.430826:0:157908:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffd45112790 00800000:00000080:0.0:1787826110.430847:0:157908:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c343d400 uarg=0000000000000000 00800000:00000004:0.0:1787826110.430856:0:157908:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9181c4d2b000: age 8173, max_age 8181 00000100:00100000:0.0:1787826110.430900:0:157908:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:157908:1874663042966400:192.168.203.147@tcp:41:lfs.0 00000100:00100000:0.0:1787826110.431418:0:157908:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c7da1480 going to sleep for 11 seconds 00000080:00200000:0.0:1787826110.591256:0:157915:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826110.591293:0:157915:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826110.591316:0:157915:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5a00/0xbef42c0221d6f0ec lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cecf14 expref: -99 pid: 157908 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787826110.591335:0:157915:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5a00/0xbef42c0221d6f0ec lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cecf14 expref: -99 pid: 157908 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787826110.591351:0:157915:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826110.591361:0:157915:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826110.591365:0:157915:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6f0ec 00000080:00010000:0.0:1787826110.591374:0:157915:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f0ec from it ffffb975017bbbd0 00010000:00010000:0.0:1787826110.591379:0:157915:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5a00/0xbef42c0221d6f0ec lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cecf14 expref: -99 pid: 157908 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787826110.591390:0:157915:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5a00/0xbef42c0221d6f0ec lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cecf14 expref: -99 pid: 157908 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00000080:00200000:0.0:1787826110.591419:0:157915:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826110.591437:0:157915:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=fsa-oleg347-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9181c41acb08), flags=1536 00000080:00200000:0.0:1787826110.591441:0:157915:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826110.591448:0:157915:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826110.591454:0:157915:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5a00/0xbef42c0221d6f0ec lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cecf14 expref: -99 pid: 157908 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:0.0:1787826110.591464:0:157915:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5a00/0xbef42c0221d6f0ec lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cecf14 expref: -99 pid: 157908 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00000080:00200000:0.0:1787826110.591471:0:157915:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826110.591475:0:157915:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826110.591478:0:157915:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6f0ec 00000080:00010000:0.0:1787826110.591482:0:157915:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f0ec from it ffffb975017bbcd0 00010000:00010000:0.0:1787826110.591485:0:157915:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5a00/0xbef42c0221d6f0ec lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cecf14 expref: -99 pid: 157908 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:0.0:1787826110.591499:0:157915:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5a00/0xbef42c0221d6f0ec lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cecf14 expref: -99 pid: 157908 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00000080:00200000:0.0:1787826110.591508:0:157915:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 3 00000080:00200000:0.0:1787826110.591513:0:157915:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826110.591518:0:157915:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826110.591525:0:157915:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5a00/0xbef42c0221d6f0ec lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cecf14 expref: -99 pid: 157908 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00010000:00010000:0.0:1787826110.591533:0:157915:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5a00/0xbef42c0221d6f0ec lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cecf14 expref: -99 pid: 157908 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00000080:00200000:0.0:1787826110.591540:0:157915:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826110.591544:0:157915:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826110.591546:0:157915:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6f0ec 00000080:00010000:0.0:1787826110.591550:0:157915:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f0ec from it ffffb975017bbdb0 00010000:00010000:0.0:1787826110.591553:0:157915:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5a00/0xbef42c0221d6f0ec lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cecf14 expref: -99 pid: 157908 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00010000:00010000:0.0:1787826110.591561:0:157915:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5a00/0xbef42c0221d6f0ec lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cecf14 expref: -99 pid: 157908 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00000080:00200000:0.0:1787826110.591570:0:157915:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 3 00000080:00200000:0.0:1787826110.591575:0:157915:0:(namei.c:2129:ll_create_nd()) VFS Op:name=fsa-oleg347-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9181c41acb08), flags=33188, excl=1 00000080:00200000:0.0:1787826110.591580:0:157915:0:(namei.c:2084:ll_mknod()) VFS Op:name=fsa-oleg347-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9181c41acb08) mode 100644 dev 0 00010000:00010000:0.0:1787826110.591609:0:157915:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5a00/0xbef42c0221d6f0ec lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00cecf14 expref: -99 pid: 157908 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00010000:00010000:0.0:1787826110.591676:0:157915:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5a00/0xbef42c0221d6f0ec lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cecf14 expref: -99 pid: 157908 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00010000:00010000:0.0:1787826110.591684:0:157915:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826110.591688:0:157915:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b5a00/0xbef42c0221d6f0ec lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cecf14 expref: -99 pid: 157908 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00000020:00100000:0.0:1787826110.591718:0:157915:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787826110.591735:0:157915:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c55a4380 x1874663042966912/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 528/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826110.591789:0:157915:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job mcreate:a9ce6693-0634-46c5-bf21-5ca7d026d44e:157915:1874663042966912:192.168.203.147@tcp:36:mcreate.0 00000100:00100000:0.0:1787826110.591897:0:157915:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c7da1240 going to sleep for 11 seconds 00000100:00100000:0.0:1787826110.599974:0:157915:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job mcreate:a9ce6693-0634-46c5-bf21-5ca7d026d44e:157915:1874663042966912:192.168.203.147@tcp:36:mcreate.0 00000080:00200000:0.0:1787826110.602573:0:157915:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x20001a9e3:0x1bd3:0x0](ffff9181c83f3248) 00000080:00200000:0.0:1787826110.602633:0:157915:0:(namei.c:2137:ll_create_nd()) VFS Op:name=fsa-oleg347-client.virtnet, unhashed 1 00000080:00200000:0.0:1787826110.625809:0:157918:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826110.625843:0:157918:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826110.625911:0:157918:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6f0f3 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 157918 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826110.625924:0:157918:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6f0f3 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 157918 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826110.625935:0:157918:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6f0f3 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 157918 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826110.625963:0:157918:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job rm:a9ce6693-0634-46c5-bf21-5ca7d026d44e:157918:1874663042967040:192.168.203.147@tcp:101:rm.0 00000100:00100000:0.0:1787826110.631584:0:157918:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c7da1240 going to sleep for 11 seconds 00000100:00100000:0.0:1787826110.652718:0:157918:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job rm:a9ce6693-0634-46c5-bf21-5ca7d026d44e:157918:1874663042967040:192.168.203.147@tcp:101:rm.0 00010000:00010000:0.0:1787826110.652780:0:157918:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6f0f3 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cecf30 expref: -99 pid: 157918 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826110.652794:0:157918:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6f0f3 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cecf30 expref: -99 pid: 157918 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826110.652806:0:157918:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6f0f3 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cecf30 expref: -99 pid: 157918 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826110.652815:0:157918:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6f0f3 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cecf30 expref: -99 pid: 157918 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826110.652827:0:157918:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6f0f3 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cecf30 expref: -99 pid: 157918 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826110.652836:0:157918:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6f0f3 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cecf30 expref: -99 pid: 157918 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826110.652844:0:157918:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6f0f3 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cecf30 expref: -99 pid: 157918 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826110.652855:0:157918:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c55a7100 x1874663042967040/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826126 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787826110.652869:0:157918:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6f0f3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cecf30 expref: -99 pid: 157918 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826110.652887:0:157918:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:0.0:1787826110.652893:0:157918:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1787826110.652918:0:157918:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 69632 00000080:00010000:0.0:1787826110.652926:0:157918:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6f0f3 00000080:00010000:0.0:1787826110.652931:0:157918:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826110.652933:0:157918:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f0f3 from it ffffb975019bbb58 00010000:00010000:0.0:1787826110.652936:0:157918:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6f0f3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cecf30 expref: -99 pid: 157918 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826110.652944:0:157918:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6f0f3 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cecf30 expref: -99 pid: 157918 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826110.652968:0:157918:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826110.652988:0:157918:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=fsa-oleg347-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9181c41acb08), flags=8 00000080:00200000:0.0:1787826110.652993:0:157918:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=fsa-oleg347-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9181c41acb08), intent=getattr 00000002:00010000:0.0:1787826110.653012:0:157918:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: fsa-oleg347-client.virtnet,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787826110.653050:0:157918:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6f0fa lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 157918 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826110.653056:0:157918:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6f0fa lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 157918 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826110.653065:0:157918:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6f0fa lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 157918 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826110.653085:0:157918:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job rm:a9ce6693-0634-46c5-bf21-5ca7d026d44e:157918:1874663042967168:192.168.203.147@tcp:101:rm.0 00000100:00100000:0.0:1787826110.653146:0:157918:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c7da1240 going to sleep for 11 seconds 00000100:00100000:0.0:1787826110.656263:0:157918:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job rm:a9ce6693-0634-46c5-bf21-5ca7d026d44e:157918:1874663042967168:192.168.203.147@tcp:101:rm.0 00010000:00010000:0.0:1787826110.656289:0:157918:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6f0fa lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cecf4c expref: -99 pid: 157918 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826110.656309:0:157918:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6f0fa lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1bd3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cecf4c expref: -99 pid: 157918 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826110.656317:0:157918:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6f0fa lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1bd3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cecf4c expref: -99 pid: 157918 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826110.656327:0:157918:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6f0fa lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bd3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cecf4c expref: -99 pid: 157918 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826110.656337:0:157918:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6f0fa lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bd3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cecf4c expref: -99 pid: 157918 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826110.656349:0:157918:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6f0fa lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1bd3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cecf4c expref: -99 pid: 157918 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826110.656357:0:157918:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6f0fa lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1bd3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cecf4c expref: -99 pid: 157918 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826110.656364:0:157918:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6f0fa lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bd3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cecf4c expref: -99 pid: 157918 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826110.656375:0:157918:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c55a7100 x1874663042967168/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 600/584 e 0 to 0 dl 1787826126 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787826110.656387:0:157918:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6f0fa lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1bd3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cecf4c expref: -99 pid: 157918 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826110.656398:0:157918:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/7123 (0 0) 00000080:00200000:0.0:1787826110.656413:0:157918:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:0.0:1787826110.656432:0:157918:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20001a9e3:0x1bd3:0x0], updating i_size 0 00000080:00200000:0.0:1787826110.656435:0:157918:0:(namei.c:147:ll_iget()) got inode: [0x20001a9e3:0x1bd3:0x0](ffff9181c83f3248): rc = 0 00000080:00200000:0.0:1787826110.656449:0:157918:0:(file.c:6843:ll_layout_conf()) [0x20001a9e3:0x1bd3:0x0]: layout version change: 4294967294 -> 4294967295 00000080:00010000:0.0:1787826110.656454:0:157918:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x1bd3:0x0](ffff9181c83f3248) for lock 0xbef42c0221d6f0fa 00000080:00010000:0.0:1787826110.656524:0:157918:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x1bd3:0x0](ffff9181c83f3248) 00000080:00010000:0.0:1787826110.656527:0:157918:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f0fa from it ffffb975019bbb20 00010000:00010000:0.0:1787826110.656531:0:157918:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6f0fa lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1bd3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00cecf4c expref: -99 pid: 157918 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826110.656539:0:157918:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6f0fa lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1bd3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00cecf4c expref: -99 pid: 157918 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:0.0:1787826110.656572:0:157918:0:(file.c:6296:ll_getattr_dentry()) START file fsa-oleg347-client.virtnet:[0x20001a9e3:0x1bd3:0x0](ffff9181c83f3248), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1787826110.656578:0:157918:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x20001a9e3:0x1bd3:0x0](ffff9181c83f3248),name=fsa-oleg347-client.virtnet 00000002:00010000:0.0:1787826110.656597:0:157918:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x20001a9e3:0x1bd3:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826110.656605:0:157918:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6f0fa lrc: 2/1,0 mode: PR/PR res: [0x20001a9e3:0x1bd3:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00cecf4c expref: -99 pid: 157918 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826110.656614:0:157918:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6f0fa lrc: 2/1,0 mode: PR/PR res: [0x20001a9e3:0x1bd3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00cecf4c expref: -99 pid: 157918 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826110.656621:0:157918:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826110.656625:0:157918:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x1bd3:0x0](ffff9181c83f3248) 00000080:00010000:0.0:1787826110.656629:0:157918:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x1bd3:0x0](ffff9181c83f3248) for lock 0xbef42c0221d6f0fa 00000080:00010000:0.0:1787826110.656632:0:157918:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f0fa from it ffffb975019bbcb8 00010000:00010000:0.0:1787826110.656635:0:157918:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6f0fa lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1bd3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00cecf4c expref: -99 pid: 157918 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826110.656641:0:157918:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6f0fa lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1bd3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00cecf4c expref: -99 pid: 157918 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787826110.656692:0:157918:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826110.656917:0:157918:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826110.656926:0:157918:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6f0f3 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cecf30 expref: -99 pid: 157918 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826110.656935:0:157918:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6f0f3 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cecf30 expref: -99 pid: 157918 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826110.656942:0:157918:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826110.656946:0:157918:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826110.656948:0:157918:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6f0f3 00000080:00010000:0.0:1787826110.656952:0:157918:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f0f3 from it ffffb975019bbc10 00010000:00010000:0.0:1787826110.656955:0:157918:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6f0f3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cecf30 expref: -99 pid: 157918 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826110.656963:0:157918:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6f0f3 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cecf30 expref: -99 pid: 157918 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787826110.656971:0:157918:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826110.656978:0:157918:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=fsa-oleg347-client.virtnet, flags=0 00000080:00200000:0.0:1787826110.656983:0:157918:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826110.656989:0:157918:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826110.656994:0:157918:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6f0f3 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cecf30 expref: -99 pid: 157918 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787826110.657002:0:157918:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6f0f3 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cecf30 expref: -99 pid: 157918 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787826110.657008:0:157918:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826110.657012:0:157918:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826110.657014:0:157918:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6f0f3 00000080:00010000:0.0:1787826110.657017:0:157918:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f0f3 from it ffffb975019bbd68 00010000:00010000:0.0:1787826110.657020:0:157918:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6f0f3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cecf30 expref: -99 pid: 157918 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787826110.657622:0:157918:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6f0f3 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cecf30 expref: -99 pid: 157918 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787826110.657635:0:157918:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 3 00000080:00200000:0.0:1787826110.657647:0:157918:0:(namei.c:2418:ll_unlink()) VFS Op:name=fsa-oleg347-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9181c41acb08) 00010000:00010000:0.0:1787826110.657688:0:157918:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6f0f3 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00cecf30 expref: -99 pid: 157918 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787826110.658222:0:157918:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6f0fa lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1bd3:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x878a5eea00cecf4c expref: -99 pid: 157918 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787826110.658235:0:157918:0:(vvp_object.c:117:vvp_conf_set()) [0x20001a9e3:0x1bd3:0x0]: losing layout lock 00010000:00010000:0.0:1787826110.658272:0:157918:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6f0fa lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x1bd3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cecf4c expref: -99 pid: 157918 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787826110.658284:0:157918:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6f0f3 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cecf30 expref: -99 pid: 157918 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787826110.658291:0:157918:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:0.0:1787826110.658294:0:157918:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513800/0xbef42c0221d6f0fa lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x1bd3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cecf4c expref: -99 pid: 157918 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787826110.658312:0:157918:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513400/0xbef42c0221d6f0f3 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cecf30 expref: -99 pid: 157918 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000020:00100000:0.0:1787826110.658330:0:157918:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787826110.658350:0:157918:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c55a7100 x1874663042967424/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 512/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826110.658373:0:157918:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job rm:a9ce6693-0634-46c5-bf21-5ca7d026d44e:157918:1874663042967424:192.168.203.147@tcp:36:rm.0 00000100:00100000:0.0:1787826110.658471:0:157918:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c7da1240 going to sleep for 11 seconds 00000100:00100000:0.0:1787826110.666630:0:157918:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job rm:a9ce6693-0634-46c5-bf21-5ca7d026d44e:157918:1874663042967424:192.168.203.147@tcp:36:rm.0 00000080:00200000:0.0:1787826110.666695:0:157918:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1bd3:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:0.0:1787826110.666705:0:157918:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1bd3:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826110.666713:0:157918:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1bd3:0x0](ffff9181c83f3248) 00000100:00100000:3.0:1787826111.010133:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7528148b-9bee-4d31-b597-8aae3b0b4982:2391:1874663042967552:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826111.017263:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7528148b-9bee-4d31-b597-8aae3b0b4982:2391:1874663042967552:192.168.203.147@tcp:400:kworker.0 00010000:00010000:3.0:1787826111.328958:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826112.352953:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826114.404879:0:33329:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826115.424976:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826116.137282:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7528148b-9bee-4d31-b597-8aae3b0b4982:2389:1874663042967680:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826116.137544:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042967936:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826116.137581:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042968192:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826116.137619:0:2389:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1787826116.137625:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042967808:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826116.137650:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042968064:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826116.159524:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042967936:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826116.160843:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042967808:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826116.162482:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7528148b-9bee-4d31-b597-8aae3b0b4982:2389:1874663042967680:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826116.175999:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042968192:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826116.196415:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042968064:192.168.203.147@tcp:400:kworker.0 00010000:00010000:0.0:1787826116.450611:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826117.475550:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826118.497235:0:33329:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:0.0:1787826118.678984:0:157990:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: local REPLAY BARRIER on lustre-MDT0000 00010000:00010000:3.0:1787826119.521821:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826120.555327:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826121.252120:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7528148b-9bee-4d31-b597-8aae3b0b4982:2392:1874663042968320:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826121.252239:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2392:1874663042968576:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826121.252269:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2392:1874663042968832:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826121.252303:0:2392:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1787826121.252309:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2392:1874663042968448:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826121.252346:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2392:1874663042968704:192.168.203.147@tcp:400:kworker.0 00010000:00010000:0.0:1787826121.569216:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826122.593094:0:33329:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826123.618387:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826124.641584:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826125.665074:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826126.369519:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7528148b-9bee-4d31-b597-8aae3b0b4982:2391:1874663042968960:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826126.369593:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2391:1874663042969216:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826126.369621:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2391:1874663042969472:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826126.379472:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2391:1874663042969216:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826126.379488:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2391:1874663042969472:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826126.384166:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042969600:192.168.203.147@tcp:38:ptlrpcd_01_01.0 00000100:00080000:1.0:1787826126.411665:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c39f6800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787826126.411683:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-19) 00000100:00100000:1.0:1787826126.411688:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042969600:192.168.203.147@tcp:38:ptlrpcd_01_01.0 00010000:00010000:0.0:1787826126.690524:0:33329:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826127.713262:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826128.736981:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826129.760985:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826130.785613:0:33329:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787826131.489520:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) a9ce6693-0634-46c5-bf21-5ca7d026d44e->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826131.489527:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787826131.489530:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c39f6800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826131.489541:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c4d2b000: connect to NID 192.168.203.147@tcp last attempt 8198 00000100:00080000:3.0:1787826131.489562:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c4d2b000: tried all connections, increasing latency to 5s 00000100:00080000:3.0:1787826131.489571:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c4d2b000: import ffff9181c39f6800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826131.489583:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826131.489601:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663042969856/t0(0) o38->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826131.489657:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181e0e59800 lustre-OST0001_UUID: changing import state from FULL to CONNECTING 00000100:00080000:3.0:1787826131.489660:0:111835:0:(import.c:2101:ptlrpc_disconnect_and_idle_import()) lustre-OST0001-osc-ffff9181c4d2b000: disconnect after 21s idle 00000100:00100000:3.0:1787826131.489686:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2391:1874663042969984:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826131.489808:0:2391:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1787826131.489815:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7528148b-9bee-4d31-b597-8aae3b0b4982:2391:1874663042969728:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826131.489848:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2391:1874663042970112:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826131.490014:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042969856:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787826131.490049:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042970240:192.168.203.147@tcp:9:kworker.0 00000100:00080000:1.0:1787826131.494530:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c39f6800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787826131.494546:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-19) 00000100:00100000:1.0:1787826131.494562:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042969856:192.168.203.147@tcp:38:kworker.0 00000100:00080000:1.0:1787826131.494587:0:2388:0:(import.c:2011:ptlrpc_disconnect_idle_interpret()) @@@ inflight=1, refcount=3: rc = 0 req@ffff9181c5634e00 x1874663042970240/t0(0) o9->lustre-OST0001-osc-ffff9181c4d2b000@192.168.203.147@tcp:28/4 lens 224/224 e 0 to 0 dl 1787826141 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787826131.494597:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181e0e59800 lustre-OST0001_UUID: changing import state from CONNECTING to IDLE 00000100:00100000:1.0:1787826131.494600:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042970240:192.168.203.147@tcp:9:kworker.0 00000100:00100000:3.0:1787826131.494801:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2391:1874663042969984:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826131.501214:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2391:1874663042970112:192.168.203.147@tcp:400:kworker.0 00010000:00010000:3.0:1787826131.808907:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826132.839970:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826133.857058:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826135.904948:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826136.610406:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042970496:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787826136.622721:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2392:1874663042970752:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826136.622766:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2392:1874663042970624:192.168.203.147@tcp:400:kworker.0 00000100:00080000:0.0:1787826136.624351:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c39f6800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826136.624366:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-19) 00000100:00100000:0.0:1787826136.624371:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042970496:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1787826136.930262:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826137.953170:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826140.001046:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826141.025903:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826141.731060:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7528148b-9bee-4d31-b597-8aae3b0b4982:2390:1874663042970880:192.168.203.147@tcp:400:kworker.0 00000100:00080000:0.0:1787826141.731266:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) a9ce6693-0634-46c5-bf21-5ca7d026d44e->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826141.731273:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787826141.731277:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c39f6800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826141.731284:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c4d2b000: connect to NID 192.168.203.147@tcp last attempt 8208 00000100:00080000:0.0:1787826141.731293:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c4d2b000: tried all connections, increasing latency to 15s 00000100:00080000:0.0:1787826141.731302:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c4d2b000: import ffff9181c39f6800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787826141.731315:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826141.731337:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7480 x1874663042971008/t0(0) o38->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826141.731704:0:2390:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1787826141.731713:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042971136:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826141.732871:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042971264:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826141.738415:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042971264:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826141.738430:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042971136:192.168.203.147@tcp:400:kworker.0 00010000:00010000:0.0:1787826142.050127:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787826142.691065:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7528148b-9bee-4d31-b597-8aae3b0b4982->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826142.691071:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00100000:3.0:1787826142.691122:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7528148b-9bee-4d31-b597-8aae3b0b4982:2392:1874663042970368:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826142.691176:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7528148b-9bee-4d31-b597-8aae3b0b4982:2390:1874663042970880:192.168.203.147@tcp:400:kworker.0 00000100:00080000:1.0:1787826142.691201:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5a6c800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826142.704223:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8146 00000100:00080000:1.0:1787826142.704313:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c5a6c800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787826142.704347:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3100 x1874663042971392/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787826142.704374:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) a9ce6693-0634-46c5-bf21-5ca7d026d44e->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826142.704379:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787826142.704382:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c39f6800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826142.704592:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c4d2b000: connect to NID 192.168.203.147@tcp last attempt 8213 00000100:00080000:1.0:1787826142.704597:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c4d2b000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787826142.704602:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c4d2b000: import ffff9181c39f6800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787826142.704611:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787826142.704624:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1180 x1874663042971520/t0(0) o38->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826142.704686:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042971648:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826142.704770:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042971776:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826142.711427:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042971648:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826142.711646:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042971776:192.168.203.147@tcp:400:kworker.0 00000100:00080000:3.0:1787826142.711867:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c39f6800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826142.711875:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-19) 00000100:00100000:3.0:1787826142.711880:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042971520:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787826144.096944:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826145.121137:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826146.144999:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787826147.874801:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7528148b-9bee-4d31-b597-8aae3b0b4982->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826147.874812:0:99984:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7528148b-9bee-4d31-b597-8aae3b0b4982->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1787826147.874815:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) a9ce6693-0634-46c5-bf21-5ca7d026d44e->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826147.874820:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787826147.874825:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c39f6800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826147.874838:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c4d2b000: connect to NID 192.168.203.147@tcp last attempt 8214 00000100:00080000:1.0:1787826147.874853:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c4d2b000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787826147.874863:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c4d2b000: import ffff9181c39f6800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787826147.874879:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787826147.874930:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3b80 x1874663042971904/t0(0) o38->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826147.875078:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042972032:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826147.875195:0:2389:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1787826147.875202:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042972160:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826147.879291:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042972160:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826147.892588:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042972032:192.168.203.147@tcp:400:kworker.0 00010000:00010000:3.0:1787826148.193000:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826149.217294:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826150.240979:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826152.290430:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826152.997327:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787826142/real 1787826142] req@ffff9181c35b3100 x1874663042971392/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826152 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826152.997363:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3100 x1874663042971392/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826152 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826152.997391:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042972288:192.168.203.147@tcp:38:kworker.0 00000100:00080000:3.0:1787826152.997638:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5a6c800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826152.997658:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826152.997663:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7528148b-9bee-4d31-b597-8aae3b0b4982:2388:1874663042971392:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787826152.999054:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7528148b-9bee-4d31-b597-8aae3b0b4982:2388:1874663042972672:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787826153.020639:0:2388:0:(import.c:1132:ptlrpc_connect_interpret()) MGC192.168.203.147@tcp: connect to target with instance 0 10000000:01000000:3.0:1787826153.020661:0:2388:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808005 00000100:02000400:3.0:1787826153.020673:0:2388:0:(import.c:1284:ptlrpc_connect_interpret()) Evicted from MGS (at 192.168.203.147@tcp) after server handle changed from 0x878a5eea00cecb63 to 0x878a5eea00cecf8b 00000100:00080000:3.0:1787826153.020679:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5a6c800 MGS: changing import state from CONNECTING to EVICTED 00000100:00080000:3.0:1787826153.020687:0:2388:0:(import.c:1674:ptlrpc_import_recovery_state_machine()) evicted from MGS@192.168.203.147@tcp; invalidating 00000100:00080000:3.0:1787826153.022371:0:158221:0:(import.c:1598:ptlrpc_invalidate_import_thread()) thread invalidate import MGC192.168.203.147@tcp to MGS@192.168.203.147@tcp 00000100:00100000:3.0:1787826153.022376:0:158221:0:(import.c:323:ptlrpc_invalidate_import()) Sleeping 20s for inflight to error out 10000000:01000000:3.0:1787826153.022380:0:158221:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808003 00010000:00010000:3.0:1787826153.022412:0:158221:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f0a6 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x110e400000000 nid: local remote: 0x878a5eea00cecb71 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1787826153.022426:0:158221:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f0a6 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x110f400000000 nid: local remote: 0x878a5eea00cecb71 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787826153.022432:0:158221:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:3.0:1787826153.022452:0:158221:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826153.022464:0:158221:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f0a6 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x590f400000000 nid: local remote: 0x878a5eea00cecb71 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826153.022482:0:158221:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b5e00/0xbef42c0221d6f0b4 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0x878a5eea00cecbe1 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1787826153.022489:0:158221:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5e00/0xbef42c0221d6f0b4 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0x878a5eea00cecbe1 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787826153.022494:0:158221:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:3.0:1787826153.022498:0:158221:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826153.022502:0:158221:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b5e00/0xbef42c0221d6f0b4 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0x878a5eea00cecbe1 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826153.022515:0:158221:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f09f lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0x878a5eea00cecb6a expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1787826153.022522:0:158221:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f09f lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0x878a5eea00cecb6a expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787826153.022532:0:158221:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:3.0:1787826153.022535:0:158221:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826153.022544:0:158221:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f09f lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0x878a5eea00cecb6a expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826153.022558:0:158221:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e0512e00/0xbef42c0221d6f0ad lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0x878a5eea00cecb7f expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1787826153.022564:0:158221:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512e00/0xbef42c0221d6f0ad lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0x878a5eea00cecb7f expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787826153.022569:0:158221:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:3.0:1787826153.022572:0:158221:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826153.022576:0:158221:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e0512e00/0xbef42c0221d6f0ad lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0x878a5eea00cecb7f expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787826153.022593:0:158221:0:(import.c:65:import_set_state_nolock()) ffff9181c5a6c800 MGS: changing import state from EVICTED to RECOVER 00000100:00080000:3.0:1787826153.022598:0:158221:0:(import.c:65:import_set_state_nolock()) ffff9181c5a6c800 MGS: changing import state from RECOVER to FULL 10000000:01000000:3.0:1787826153.022601:0:158221:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808004 00000100:00080000:3.0:1787826153.022604:0:158221:0:(pinger.c:186:ptlrpc_pinger_ir_up()) IR up 00000100:02000000:3.0:1787826153.022608:0:158221:0:(import.c:1743:ptlrpc_import_recovery_state_machine()) MGC192.168.203.147@tcp: Connection restored to 192.168.203.147@tcp (at 192.168.203.147@tcp) 00010000:00010000:1.0:1787826153.313218:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826154.337054:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:0.0:1787826154.720809:0:15163:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:0.0:1787826154.721118:0:15163:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff9181c4d2b000 from 328 10000000:01000000:0.0:1787826154.721126:0:15163:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:0.0:1787826154.721199:0:15163:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0513c00/0xbef42c0221d6f101 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826154.721213:0:15163:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e0513c00/0xbef42c0221d6f101 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826154.721229:0:15163:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e0513c00/0xbef42c0221d6f101 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826154.721265:0:15163:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6a00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:7528148b-9bee-4d31-b597-8aae3b0b4982:15163:1874663042972800:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00100000:0.0:1787826154.721398:0:15163:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c7da1cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826154.741186:0:15163:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f2680 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:7528148b-9bee-4d31-b597-8aae3b0b4982:15163:1874663042972928:192.168.203.147@tcp:256:ll_cfg_requeue.0 00000100:00080000:1.0:1787826154.741274:0:15163:0:(import.c:214:ptlrpc_set_import_discon()) mdc: import ffff9181c39f6800 already not connected (conn 597, was 0): DISCONN 00010000:00080000:1.0:1787826154.741289:0:15163:0:(ldlm_lib.c:85:import_set_conn()) imp ffff9181c39f6800@lustre-MDT0000-mdc-ffff9181c4d2b000: found existing conn 192.168.203.147@tcp, moved to head 00000100:00080000:1.0:1787826154.741296:0:15163:0:(import.c:65:import_set_state_nolock()) ffff9181c39f6800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826154.741302:0:15163:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c4d2b000: connect to NID 192.168.203.147@tcp last attempt 0 00000100:00080000:1.0:1787826154.741312:0:15163:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c4d2b000: import ffff9181c39f6800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787826154.741327:0:15163:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787826154.741363:0:15163:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1c00 x1874663042973056/t0(0) o38->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 10000000:01000000:1.0:1787826154.741430:0:15163:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-cliir' succeeded (0). 00010000:00010000:1.0:1787826154.741443:0:15163:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0513c00/0xbef42c0221d6f101 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00ced175 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826154.741456:0:15163:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.147@tcp lock: ffff9181e0513c00/0xbef42c0221d6f101 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00ced175 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826154.741468:0:15163:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:1.0:1787826154.741471:0:15163:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff9181c4d2b000 from 52 10000000:01000000:1.0:1787826154.741475:0:15163:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:1.0:1787826154.741508:0:15163:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f108 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826154.741517:0:15163:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f108 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826154.741531:0:15163:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f108 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826154.741555:0:15163:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:7528148b-9bee-4d31-b597-8aae3b0b4982:15163:1874663042973184:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00100000:0.0:1787826154.741678:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042973056:192.168.203.147@tcp:38:ll_cfg_requeue.0 00000100:00100000:1.0:1787826154.743027:0:15163:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6e79cc0 going to sleep for 11 seconds 00010000:00080000:0.0:1787826154.747192:0:2388:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffff9181c35b1c00 x1874663042973056/t0(0) o38->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 520/416 e 0 to 0 dl 1787826164 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826154.747224:0:2388:0:(import.c:1132:ptlrpc_connect_interpret()) lustre-MDT0000-mdc-ffff9181c4d2b000: connect to target with instance 80 00000100:00080000:0.0:1787826154.747249:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c39f6800 lustre-MDT0000_UUID: changing import state from CONNECTING to REPLAY 00000100:00080000:0.0:1787826154.747258:0:2388:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1787826154.747261:0:2388:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9181c4d2b000: committing for last_committed 339302416388 gen 11 00000100:00100000:0.0:1787826154.747269:0:2388:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff9181c55a4380 x1874663042966912/t339302416389(339302416389) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 528/448 e 0 to 0 dl 1787826126 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826154.747278:0:2388:0:(recover.c:53:ptlrpc_replay_next()) import ffff9181c39f6800 from lustre-MDT0000_UUID committed 339302416388 last 0 00000100:00080000:0.0:1787826154.747287:0:2388:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9181c3573480 x1874663041393408/t330712482189(330712482189) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 0 dl 1787826107 ref 1 fl New:RPQU/604/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:0 00000100:00100000:0.0:1787826154.747302:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042973056:192.168.203.147@tcp:38:ll_cfg_requeue.0 00000100:00100000:0.0:1787826154.747338:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c3573480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663041393408:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787826154.749739:0:15163:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:7528148b-9bee-4d31-b597-8aae3b0b4982:15163:1874663042973184:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:0.0:1787826154.749814:0:15163:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f108 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00ced17c expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826154.749829:0:15163:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f108 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x878a5eea00ced17c expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826154.749945:0:15163:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:7528148b-9bee-4d31-b597-8aae3b0b4982:15163:1874663042973312:192.168.203.147@tcp:501:ll_cfg_requeue.0 00000100:00100000:0.0:1787826154.750028:0:15163:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c7da16c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826154.779044:0:15163:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f1180 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:7528148b-9bee-4d31-b597-8aae3b0b4982:15163:1874663042973440:192.168.203.147@tcp:503:ll_cfg_requeue.0 00000040:00080000:1.0:1787826154.789720:0:158245:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x3:0x0] index 64768 count 51 00000020:01000000:0.0:1787826154.790118:0:15163:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-client gen 52-51 (rc=0) 10000000:01000000:0.0:1787826154.790156:0:15163:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-client' succeeded (0). 00010000:00010000:0.0:1787826154.790173:0:15163:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f108 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00ced17c expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826154.790187:0:15163:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f108 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00ced17c expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787826154.790200:0:15163:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:0.0:1787826154.790204:0:15163:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff9181c4d2b000 from 7 10000000:01000000:0.0:1787826154.790207:0:15163:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00010000:00010000:0.0:1787826154.790262:0:15163:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512600/0xbef42c0221d6f10f lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826154.790271:0:15163:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e0512600/0xbef42c0221d6f10f lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826154.790285:0:15163:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e0512600/0xbef42c0221d6f10f lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826154.790321:0:15163:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:7528148b-9bee-4d31-b597-8aae3b0b4982:15163:1874663042973568:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00100000:0.0:1787826154.790434:0:15163:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c7da1c00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826154.811512:0:15163:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f2300 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:7528148b-9bee-4d31-b597-8aae3b0b4982:15163:1874663042973696:192.168.203.147@tcp:501:ll_cfg_requeue.0 00000100:00100000:1.0:1787826154.811607:0:15163:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:7528148b-9bee-4d31-b597-8aae3b0b4982:15163:1874663042973824:192.168.203.147@tcp:503:ll_cfg_requeue.0 00000100:00100000:1.0:1787826154.811711:0:15163:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6e79cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826154.828867:0:15163:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:7528148b-9bee-4d31-b597-8aae3b0b4982:15163:1874663042973824:192.168.203.147@tcp:503:ll_cfg_requeue.0 00000040:00080000:3.0:1787826154.835946:0:158252:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x2:0x0] index 64768 count 7 00000020:01000000:1.0:1787826154.836258:0:15163:0:(obd_config.c:2145:class_config_parse_llog()) Processed log params gen 7-6 (rc=0) 10000000:01000000:1.0:1787826154.836286:0:15163:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'params' succeeded (0). 00010000:00010000:1.0:1787826154.836301:0:15163:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512600/0xbef42c0221d6f10f lrc: 3/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00ced18a expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826154.836314:0:15163:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.147@tcp lock: ffff9181e0512600/0xbef42c0221d6f10f lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00ced18a expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826154.836326:0:15163:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:1.0:1787826154.836329:0:15163:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff9181c3d3cd28 from 1 10000000:01000000:1.0:1787826154.836332:0:15163:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00010000:00010000:1.0:1787826154.836392:0:15163:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f116 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826154.836407:0:15163:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f116 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826154.836420:0:15163:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f116 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826154.836456:0:15163:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:7528148b-9bee-4d31-b597-8aae3b0b4982:15163:1874663042973952:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00100000:1.0:1787826154.836586:0:15163:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d8141480 going to sleep for 11 seconds 00010000:00010000:0.0:1787826155.361535:0:33329:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826156.384912:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826157.421155:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787826158.113171:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) a9ce6693-0634-46c5-bf21-5ca7d026d44e->lustre-MDT0000_UUID: level REPLAY/5 force 0 force_next 1 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826158.113179:0:99984:0:(pinger.c:247:ptlrpc_pinger_process_import()) a9ce6693-0634-46c5-bf21-5ca7d026d44e->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: REPLAY) 00000100:00100000:1.0:1787826158.113227:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042974336:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826158.113242:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7528148b-9bee-4d31-b597-8aae3b0b4982:2389:1874663042974208:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826158.113322:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042974464:192.168.203.147@tcp:400:kworker.0 00010000:00010000:1.0:1787826158.118130:0:2389:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 0/0 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1787826158.118153:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042974464:192.168.203.147@tcp:400:kworker.0 00010000:00010000:0.0:1787826158.118214:0:15161:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 0 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826158.118229:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826158.118231:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826158.118233:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826158.119223:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7528148b-9bee-4d31-b597-8aae3b0b4982:2389:1874663042974208:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826158.119319:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042974336:192.168.203.147@tcp:400:kworker.0 00010000:00010000:0.0:1787826158.433271:0:33329:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826159.458579:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787826160.160919:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) a9ce6693-0634-46c5-bf21-5ca7d026d44e->lustre-MDT0000_UUID: level REPLAY/5 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826160.160928:0:99984:0:(pinger.c:247:ptlrpc_pinger_process_import()) a9ce6693-0634-46c5-bf21-5ca7d026d44e->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: REPLAY) 00000100:00100000:1.0:1787826160.160989:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7528148b-9bee-4d31-b597-8aae3b0b4982:2390:1874663042974592:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826160.161130:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042974848:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826160.161170:0:2390:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1787826160.161177:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042974720:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826160.178378:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042974720:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826160.178402:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7528148b-9bee-4d31-b597-8aae3b0b4982:2390:1874663042974592:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826160.178416:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042974848:192.168.203.147@tcp:400:kworker.0 00010000:00010000:1.0:1787826160.481243:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826161.505803:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826162.534174:0:33329:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200008:1.0:1787826163.039316:0:158321:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1787826163.039332:0:158321:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:1.0:1787826163.039360:0:158321:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826163.039444:0:158321:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6f11d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 158321 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826163.039550:0:158321:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6f11d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 158321 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826163.039568:0:158321:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6f11d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 158321 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787826163.039600:0:158321:0:(client.c:1845:ptlrpc_send_new_req()) @@@ req waiting for recovery: (FULL != REPLAY) req@ffff9181c35b3100 x1874663042974976/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/1152 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/600/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:0 00000100:00100000:1.0:1787826163.039613:0:158321:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c134f840 going to sleep for 0 seconds 00010000:00010000:3.0:1787826163.553514:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826164.064980:0:158321:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c134f840 going to sleep for 0 seconds 00010000:00010000:1.0:1787826164.577066:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826165.088934:0:158321:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c134f840 going to sleep for 0 seconds 00000100:00100000:0.0:1787826165.281341:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7528148b-9bee-4d31-b597-8aae3b0b4982:2389:1874663042975104:192.168.203.147@tcp:400:kworker.0 00000100:00080000:1.0:1787826165.281572:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) a9ce6693-0634-46c5-bf21-5ca7d026d44e->lustre-MDT0000_UUID: level REPLAY/5 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826165.281578:0:99984:0:(pinger.c:247:ptlrpc_pinger_process_import()) a9ce6693-0634-46c5-bf21-5ca7d026d44e->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: REPLAY) 00000100:00100000:0.0:1787826165.283845:0:2389:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1787826165.283876:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042975232:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826165.287780:0:2390:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1787826165.287809:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042975360:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826165.292130:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042975232:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826165.292987:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7528148b-9bee-4d31-b597-8aae3b0b4982:2389:1874663042975104:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826165.298560:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042975360:192.168.203.147@tcp:400:kworker.0 00010000:00010000:0.0:1787826165.607053:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826166.113118:0:158321:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c134f840 going to sleep for 0 seconds 00010000:00010000:0.0:1787826166.625454:0:33329:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826167.136923:0:158321:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c134f840 going to sleep for 0 seconds 00010000:00010000:3.0:1787826167.648963:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826168.160973:0:158321:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c134f840 going to sleep for 0 seconds 00010000:00010000:1.0:1787826168.673472:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826169.186258:0:158321:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c134f840 going to sleep for 0 seconds 00010000:00010000:0.0:1787826169.696978:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826170.208929:0:158321:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c134f840 going to sleep for 0 seconds 00010000:00010000:0.0:1787826170.721093:0:33329:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000400:0.0:1787826170.914643:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787826154/real 1787826154] req@ffff9181c3573480 x1874663041393408/t330712482189(330712482189) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 1 dl 1787826170 ref 2 fl Rpc:XPQr/604/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:0 00000100:00100000:0.0:1787826170.914680:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=REPLAY (now=REPLAY) req@ffff9181c3573480 x1874663041393408/t330712482189(330712482189) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 1 dl 1787826170 ref 2 fl Rpc:XPQU/604/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:0 00000100:00020000:0.0:1787826170.914700:0:2388:0:(client.c:3398:ptlrpc_replay_interpret()) @@@ request replay timed out req@ffff9181c3573480 x1874663041393408/t330712482189(330712482189) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 1 dl 1787826170 ref 2 fl Interpret:EXPQU/604/ffffffff rc -110/-1 job:'lfs.0' uid:0 gid:0 projid:0 00000100:00080000:0.0:1787826170.935905:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c39f6800 lustre-MDT0000_UUID: changing import state from REPLAY to CONNECTING 00000100:00080000:0.0:1787826170.935923:0:2388:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c4d2b000: connect to NID 192.168.203.147@tcp last attempt 8226 00000100:00080000:0.0:1787826170.935938:0:2388:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c4d2b000: import ffff9181c39f6800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787826170.935950:0:2388:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826170.935983:0:2388:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5c00 x1874663042975872/t0(0) o38->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826170.938212:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c3573480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663041393408:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787826170.938236:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042975872:192.168.203.147@tcp:38:ptlrpcd_rcv.0 00010000:00080000:1.0:1787826170.953968:0:2388:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffff9181c55a5c00 x1874663042975872/t0(0) o38->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 520/416 e 0 to 0 dl 1787826180 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'ptlrpcd_rcv.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787826170.954005:0:2388:0:(import.c:1132:ptlrpc_connect_interpret()) lustre-MDT0000-mdc-ffff9181c4d2b000: connect to target with instance 80 00000100:00080000:1.0:1787826170.954026:0:2388:0:(import.c:1297:ptlrpc_connect_interpret()) reconnected to lustre-MDT0000_UUID@192.168.203.147@tcp after partition 00000100:00080000:1.0:1787826170.954029:0:2388:0:(import.c:1307:ptlrpc_connect_interpret()) lustre-MDT0000-mdc-ffff9181c4d2b000: reconnected to lustre-MDT0000_UUID during replay 00000100:00080000:1.0:1787826170.954031:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c39f6800 lustre-MDT0000_UUID: changing import state from CONNECTING to REPLAY 00000100:00080000:1.0:1787826170.954036:0:2388:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1787826170.954038:0:2388:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9181c4d2b000: committing for last_committed 339302416388 gen 11 00000100:00100000:1.0:1787826170.954045:0:2388:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff9181c55a4380 x1874663042966912/t339302416389(339302416389) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 528/448 e 0 to 0 dl 1787826126 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787826170.954053:0:2388:0:(recover.c:53:ptlrpc_replay_next()) import ffff9181c39f6800 from lustre-MDT0000_UUID committed 339302416388 last 0 00000100:00080000:1.0:1787826170.954059:0:2388:0:(recover.c:118:ptlrpc_replay_next()) @@@ resend_replay=1, last_transno=0 req@ffff9181c3573480 x1874663041393408/t330712482189(330712482189) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 1 dl 1787826170 ref 1 fl Complete:EXPQU/606/ffffffff rc -110/-1 job:'lfs.0' uid:0 gid:0 projid:0 00000100:00080000:1.0:1787826170.954070:0:2388:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9181c3573480 x1874663041393408/t330712482189(330712482189) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 1 dl 1787826170 ref 1 fl New:EXPQU/606/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:0 00000100:00100000:1.0:1787826170.954081:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042975872:192.168.203.147@tcp:38:ptlrpcd_rcv.0 00000100:00100000:1.0:1787826170.954108:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c3573480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663041393408:192.168.203.147@tcp:101:lfs.0 00000100:00080000:1.0:1787826170.997779:0:2388:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9181c3573480 x1874663041393408/t330712482189(330712482189) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 520/608 e 0 to 0 dl 1787826186 ref 2 fl Interpret:RPQU/604/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00080000:1.0:1787826170.997800:0:2388:0:(mdc_request.c:696:mdc_replay_open()) updating handle from 0x878a5eea00cecc6d to 0x878a5eea00ced191 00000100:00020000:1.0:1787826170.997806:0:2388:0:(client.c:3448:ptlrpc_replay_interpret()) @@@ status 301, old was 0 req@ffff9181c3573480 x1874663041393408/t330712482189(330712482189) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 520/608 e 0 to 0 dl 1787826186 ref 2 fl Interpret:RPQU/604/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000100:00080000:1.0:1787826170.997828:0:2388:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1787826170.997830:0:2388:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9181c4d2b000: committing for last_committed 339302416388 gen 11 00000100:00100000:1.0:1787826170.997836:0:2388:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff9181c55a4380 x1874663042966912/t339302416389(339302416389) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 528/448 e 0 to 0 dl 1787826126 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787826170.997845:0:2388:0:(recover.c:53:ptlrpc_replay_next()) import ffff9181c39f6800 from lustre-MDT0000_UUID committed 339302416388 last 330712482189 00000100:00080000:1.0:1787826170.997853:0:2388:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9181c71f1880 x1874663042965504/t339302416387(339302416387) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826124 ref 1 fl New:RPQU/604/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00000100:00100000:1.0:1787826170.997866:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c3573480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663041393408:192.168.203.147@tcp:101:lfs.0 00000100:00100000:1.0:1787826170.997890:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042965504:192.168.203.147@tcp:101:multiop.0 00000100:00080000:1.0:1787826171.001377:0:2388:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9181c71f1880 x1874663042965504/t339302416387(339302416387) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 592/608 e 0 to 0 dl 1787826186 ref 2 fl Interpret:RPQU/604/0 rc 301/301 job:'multiop.0' uid:0 gid:0 projid:0 00000002:00080000:1.0:1787826171.001405:0:2388:0:(mdc_request.c:696:mdc_replay_open()) updating handle from 0x878a5eea00cecedc to 0x878a5eea00ced1c9 00000100:00020000:1.0:1787826171.001410:0:2388:0:(client.c:3448:ptlrpc_replay_interpret()) @@@ status 301, old was 0 req@ffff9181c71f1880 x1874663042965504/t339302416387(339302416387) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 592/608 e 0 to 0 dl 1787826186 ref 2 fl Interpret:RPQU/604/0 rc 301/301 job:'multiop.0' uid:0 gid:0 projid:0 00000100:00080000:1.0:1787826171.001418:0:2388:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1787826171.001420:0:2388:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9181c4d2b000: committing for last_committed 339302416388 gen 11 00000100:00100000:1.0:1787826171.001425:0:2388:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff9181c55a4380 x1874663042966912/t339302416389(339302416389) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 528/448 e 0 to 0 dl 1787826126 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787826171.001434:0:2388:0:(recover.c:53:ptlrpc_replay_next()) import ffff9181c39f6800 from lustre-MDT0000_UUID committed 339302416388 last 339302416387 00000100:00080000:1.0:1787826171.001440:0:2388:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9181c55a4380 x1874663042966912/t339302416389(339302416389) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 528/448 e 0 to 0 dl 1787826126 ref 1 fl New:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787826171.001451:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042965504:192.168.203.147@tcp:101:multiop.0 00000100:00100000:1.0:1787826171.001474:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042966912:192.168.203.147@tcp:36:mcreate.0 00000100:00080000:1.0:1787826171.005326:0:2388:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9181c55a4380 x1874663042966912/t339302416389(339302416389) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 528/448 e 0 to 0 dl 1787826187 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787826171.005341:0:2388:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1787826171.005344:0:2388:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9181c4d2b000: committing for last_committed 339302416388 gen 11 00000100:00100000:1.0:1787826171.005349:0:2388:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff9181c55a4380 x1874663042966912/t339302416389(339302416389) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 528/448 e 0 to 0 dl 1787826187 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787826171.005357:0:2388:0:(recover.c:53:ptlrpc_replay_next()) import ffff9181c39f6800 from lustre-MDT0000_UUID committed 339302416388 last 339302416389 00000100:00080000:1.0:1787826171.005364:0:2388:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9181c55a7100 x1874663042967424/t339302416390(339302416390) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 512/456 e 0 to 0 dl 1787826126 ref 1 fl New:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787826171.005376:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042966912:192.168.203.147@tcp:36:mcreate.0 00000100:00100000:1.0:1787826171.005410:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042967424:192.168.203.147@tcp:36:rm.0 00000100:00080000:1.0:1787826171.009811:0:2388:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9181c55a7100 x1874663042967424/t339302416390(339302416390) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 512/456 e 0 to 0 dl 1787826187 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787826171.009826:0:2388:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1787826171.009829:0:2388:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9181c4d2b000: committing for last_committed 339302416388 gen 11 00000100:00100000:1.0:1787826171.009835:0:2388:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff9181c55a4380 x1874663042966912/t339302416389(339302416389) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 528/448 e 0 to 0 dl 1787826187 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787826171.009844:0:2388:0:(recover.c:53:ptlrpc_replay_next()) import ffff9181c39f6800 from lustre-MDT0000_UUID committed 339302416388 last 339302416390 00000100:00080000:1.0:1787826171.009849:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c39f6800 lustre-MDT0000_UUID: changing import state from REPLAY to REPLAY_LOCKS 00000100:00100000:1.0:1787826171.010132:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042967424:192.168.203.147@tcp:36:rm.0 00000100:00100000:1.0:1787826171.011117:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042976000:192.168.203.147@tcp:101:ldlm_lock_repla.0 00000100:00100000:1.0:1787826171.011202:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042976128:192.168.203.147@tcp:101:ldlm_lock_repla.0 00010000:00010000:1.0:1787826171.015183:0:2388:0:(ldlm_request.c:2777:replay_lock_interpret()) ### replayed lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513000/0xbef42c0221d64bee lrc: 2/0,0 mode: CR/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced1f3 expref: -99 pid: 152210 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000100:00100000:1.0:1787826171.015210:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042976000:192.168.203.147@tcp:101:ldlm_lock_repla.0 00010000:00010000:1.0:1787826171.015225:0:2388:0:(ldlm_request.c:2777:replay_lock_interpret()) ### replayed lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6800/0xbef42c0221d64b3f lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced1fa expref: -99 pid: 152202 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00000100:00080000:1.0:1787826171.015233:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c39f6800 lustre-MDT0000_UUID: changing import state from REPLAY_LOCKS to REPLAY_WAIT 00000100:00100000:1.0:1787826171.015259:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042976128:192.168.203.147@tcp:101:ldlm_lock_repla.0 00000100:00100000:1.0:1787826171.015286:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042976256:192.168.203.147@tcp:400:ptlrpcd_rcv.0 00000100:00100000:1.0:1787826171.025419:0:2388:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9181c4d2b000: committing for last_committed 339302416390 gen 11 00000100:00080000:1.0:1787826171.025452:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c39f6800 lustre-MDT0000_UUID: changing import state from REPLAY_WAIT to RECOVER 00000100:00080000:1.0:1787826171.025465:0:2388:0:(client.c:3247:ptlrpc_resend_req()) @@@ going to resend req@ffff9181c35b0700 x1874663042976256/t0(0) o400->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 224/224 e 0 to 0 dl 1787826187 ref 1 fl Interpret:RQU/2c0/0 rc 0/0 job:'ptlrpcd_rcv.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787826171.025475:0:2388:0:(client.c:3256:ptlrpc_resend_req()) @@@ it has reply, so skip it req@ffff9181c35b0700 x1874663042976256/t0(0) o400->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 224/224 e 0 to 0 dl 1787826187 ref 1 fl Interpret:RQU/2c0/0 rc 0/0 job:'ptlrpcd_rcv.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787826171.025482:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c39f6800 lustre-MDT0000_UUID: changing import state from RECOVER to FULL 00000100:02000000:1.0:1787826171.026060:0:2388:0:(import.c:1743:ptlrpc_import_recovery_state_machine()) lustre-MDT0000-mdc-ffff9181c4d2b000: Connection restored to 192.168.203.147@tcp (at 192.168.203.147@tcp) 00000100:00080000:1.0:1787826171.026067:0:2388:0:(recover.c:191:ptlrpc_wake_delayed()) @@@ waking (set 00000000b5fba576): req@ffff9181c35b3100 x1874663042974976/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/1152 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/600/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:0 00000100:00100000:1.0:1787826171.026098:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042976256:192.168.203.147@tcp:400:ptlrpcd_rcv.0 00000100:00100000:3.0:1787826171.026264:0:158321:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c134f840 going to sleep for 11 seconds 00000100:00100000:3.0:1787826171.030134:0:158321:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:158321:1874663042974976:192.168.203.147@tcp:101:lfs.0 00010000:00010000:3.0:1787826171.030161:0:158321:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6f11d lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced208 expref: -99 pid: 158321 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826171.030174:0:158321:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6f11d lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced208 expref: -99 pid: 158321 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826171.030186:0:158321:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6f11d lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced208 expref: -99 pid: 158321 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826171.030197:0:158321:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6f11d lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced208 expref: -99 pid: 158321 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826171.030210:0:158321:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6f11d lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced208 expref: -99 pid: 158321 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826171.030218:0:158321:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6f11d lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced208 expref: -99 pid: 158321 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826171.030226:0:158321:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6f11d lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced208 expref: -99 pid: 158321 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826171.030238:0:158321:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c35b3100 x1874663042974976/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826187 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826171.030253:0:158321:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6f11d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced208 expref: -99 pid: 158321 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826171.030267:0:158321:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:3.0:1787826171.030273:0:158321:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:3.0:1787826171.030300:0:158321:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 69632 00000080:00010000:3.0:1787826171.030306:0:158321:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6f11d 00000080:00010000:3.0:1787826171.030312:0:158321:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:3.0:1787826171.030315:0:158321:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f11d from it ffffb97500c87cb8 00010000:00010000:3.0:1787826171.030319:0:158321:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6f11d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced208 expref: -99 pid: 158321 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826171.030328:0:158321:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6f11d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced208 expref: -99 pid: 158321 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:3.0:1787826171.031084:0:158321:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), request_mask 0, flags 0, foreign 0 00000080:00200000:3.0:1787826171.031091:0:158321:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:3.0:1787826171.031109:0:158321:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826171.031119:0:158321:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6f11d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced208 expref: -99 pid: 158321 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826171.031129:0:158321:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6f11d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced208 expref: -99 pid: 158321 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826171.031136:0:158321:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826171.031141:0:158321:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:3.0:1787826171.031144:0:158321:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6f11d 00000080:00010000:3.0:1787826171.031148:0:158321:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f11d from it ffffb97500c87cb8 00010000:00010000:3.0:1787826171.031151:0:158321:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6f11d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced208 expref: -99 pid: 158321 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826171.031159:0:158321:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6f11d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced208 expref: -99 pid: 158321 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:3.0:1787826171.031268:0:158321:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:3.0:1787826171.031277:0:158321:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826171.031285:0:158321:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6f11d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced208 expref: -99 pid: 158321 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826171.031294:0:158321:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6f11d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced208 expref: -99 pid: 158321 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:3.0:1787826171.031300:0:158321:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826171.031304:0:158321:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:3.0:1787826171.031306:0:158321:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6f11d 00000080:00010000:3.0:1787826171.031310:0:158321:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f11d from it ffffb97500c87bb0 00010000:00010000:3.0:1787826171.031313:0:158321:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6f11d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced208 expref: -99 pid: 158321 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826171.031320:0:158321:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6f11d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced208 expref: -99 pid: 158321 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:3.0:1787826171.031329:0:158321:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 44 00000080:00200008:3.0:1787826171.031336:0:158321:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), flags 100000 00000080:00200080:3.0:1787826171.031433:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.031448:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00800000:00000004:3.0:1787826171.031455:0:158321:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9181c4d2b000: age 8182, max_age 8241 00000100:00100000:3.0:1787826171.031490:0:158321:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:158321:1874663042976384:192.168.203.147@tcp:41:lfs.0 00000100:00100000:3.0:1787826171.031575:0:158321:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d800c3c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826171.038258:0:158321:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:158321:1874663042976384:192.168.203.147@tcp:41:lfs.0 00800000:00000004:3.0:1787826171.038298:0:158321:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0000-mdc-ffff9181c4d2b000: update ffff9181c87b0450 cache blocks 317234/353529 objects 1023645/1024000 00000080:00200080:3.0:1787826171.038801:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.038821:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00800000:00000004:3.0:1787826171.038831:0:158321:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9181c4d2b000: age 8182, max_age 8241 00000100:00100000:3.0:1787826171.038916:0:158321:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:158321:1874663042976512:192.168.203.147@tcp:41:lfs.0 00000100:00100000:3.0:1787826171.038981:0:158321:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d800c3c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826171.043256:0:158321:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:158321:1874663042976512:192.168.203.147@tcp:41:lfs.0 00800000:00000004:3.0:1787826171.043284:0:158321:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0001-mdc-ffff9181c4d2b000: update ffff9181c87b1e08 cache blocks 317336/353529 objects 1023704/1024000 00000080:00200080:3.0:1787826171.043352:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043369:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043377:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043381:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043385:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043388:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043392:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043396:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043399:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043403:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043407:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043410:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043414:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043418:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043422:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043425:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043429:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043433:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043437:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043440:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043444:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043446:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043450:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043453:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043456:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043459:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043462:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043465:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043468:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043471:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043474:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043477:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043480:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043483:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043486:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043489:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043492:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043495:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043498:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043500:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043504:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043507:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043510:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043513:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043516:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043519:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043522:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043524:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043528:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043530:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043534:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043536:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043539:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043542:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043544:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043547:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043550:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043553:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043556:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043559:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043562:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043564:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043567:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043570:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043573:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043575:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043578:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043581:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043584:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043587:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043590:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043592:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043595:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043598:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043601:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043603:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043606:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043620:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043623:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043626:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043630:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043633:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043636:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043639:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043643:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043646:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043650:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043652:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043656:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043658:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043661:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043665:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043668:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043670:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043673:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043676:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043679:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043681:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043684:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043687:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043690:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043693:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043696:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043699:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043701:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043704:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043707:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043711:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043714:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043717:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043720:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043722:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043725:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043728:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043730:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043733:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043737:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043740:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043765:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043769:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043772:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043775:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043778:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043781:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043784:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00800000:00000080:3.0:1787826171.043786:0:158321:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.043790:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.043795:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00020000:00000004:3.0:1787826171.043802:0:158321:0:(obd_class.h:1090:obd_statfs()) lustre-OST0000-osc-ffff9181c4d2b000: age 8182, max_age 8241 00000100:00100000:3.0:1787826171.043833:0:158321:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:158321:1874663042976640:192.168.203.147@tcp:13:lfs.0 00000100:00100000:3.0:1787826171.043898:0:158321:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d800c3c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826171.050454:0:158321:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:158321:1874663042976640:192.168.203.147@tcp:13:lfs.0 00020000:00000004:3.0:1787826171.050544:0:158321:0:(obd_class.h:1116:obd_statfs()) lustre-OST0000-osc-ffff9181c4d2b000: update ffff9181c87b37c0 cache blocks 901356/958279 objects 261710/262144 00000080:00200080:3.0:1787826171.050647:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.050663:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00020000:00000004:3.0:1787826171.050670:0:158321:0:(obd_class.h:1090:obd_statfs()) lustre-OST0001-osc-ffff9181c4d2b000: age 8182, max_age 8241 00000100:00080000:3.0:1787826171.050682:0:158321:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9181c4d2b000: connect to NID 192.168.203.147@tcp last attempt 8163 00000100:00080000:3.0:1787826171.050691:0:158321:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9181c4d2b000: import ffff9181e0e59800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826171.050701:0:158321:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683216 ocd_grant: 8437760, lost: 0. 00000100:00100000:3.0:1787826171.050723:0:158321:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635180 x1874663042976768/t0(0) o8->lustre-OST0001-osc-ffff9181c4d2b000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826171.050785:0:158321:0:(client.c:1845:ptlrpc_send_new_req()) @@@ req waiting for recovery: (FULL != CONNECTING) req@ffff9181c5634000 x1874663042976896/t0(0) o13->lustre-OST0001-osc-ffff9181c4d2b000@192.168.203.147@tcp:7/4 lens 224/368 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826171.050793:0:158321:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d800c3c0 going to sleep for 0 seconds 00000100:00100000:1.0:1787826171.050938:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042976768:192.168.203.147@tcp:8:lfs.0 00000100:00100000:1.0:1787826171.055258:0:2388:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-ffff9181c4d2b000: committing for last_committed 4294975530 gen 34 00000100:00080000:1.0:1787826171.055281:0:2388:0:(import.c:1132:ptlrpc_connect_interpret()) lustre-OST0001-osc-ffff9181c4d2b000: connect to target with instance 1 00000100:00080000:1.0:1787826171.055290:0:2388:0:(import.c:956:ptlrpc_connect_set_flags()) lustre-OST0001-osc-ffff9181c4d2b000: Resetting ns_connect_flags to server flags: 0xa0427af2e3440038 00000100:00080000:1.0:1787826171.055301:0:2388:0:(import.c:1213:ptlrpc_connect_interpret()) connected to replayable target: lustre-OST0001_UUID 00000100:00080000:1.0:1787826171.055305:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181e0e59800 lustre-OST0001_UUID: changing import state from CONNECTING to FULL 00000080:00000004:1.0:1787826171.055315:0:2388:0:(lcommon_misc.c:94:cl_ocd_update()) Changing connect_flags: 0xa0427af2e3440038 -> 0xa0427af2e3440038 00000080:00080000:1.0:1787826171.055320:0:2388:0:(lcommon_misc.c:56:cl_init_ea_size()) updating def/max_easize: 72/65536 00000100:00080000:1.0:1787826171.055334:0:2388:0:(recover.c:191:ptlrpc_wake_delayed()) @@@ waking (set 000000004af1ddfc): req@ffff9181c5634000 x1874663042976896/t0(0) o13->lustre-OST0001-osc-ffff9181c4d2b000@192.168.203.147@tcp:7/4 lens 224/368 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787826171.055361:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042976768:192.168.203.147@tcp:8:lfs.0 00000100:00100000:3.0:1787826171.055510:0:158321:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d800c3c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826171.063551:0:158321:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:158321:1874663042976896:192.168.203.147@tcp:13:lfs.0 00020000:00000004:3.0:1787826171.063580:0:158321:0:(obd_class.h:1116:obd_statfs()) lustre-OST0001-osc-ffff9181c4d2b000: update ffff9181c87b5178 cache blocks 901360/958279 objects 261727/262144 00000080:00200080:3.0:1787826171.063649:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.063667:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.063674:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.063677:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.063681:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.063685:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.063689:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.063691:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.063695:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.063699:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.063702:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.063705:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.063708:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.063711:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.063715:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.063719:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.063723:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.063725:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.063729:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.063733:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.063736:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.063739:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.063776:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.063782:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.063785:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.063788:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.063791:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.063794:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.063797:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.063800:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.063804:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.063807:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.063810:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.063813:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.063816:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.063819:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.063822:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.063825:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.063828:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.063831:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.063834:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.063837:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.063840:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.063844:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.063847:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.063850:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.063853:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.063856:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.063860:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.063862:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.063866:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.063869:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.063872:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.063875:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.063878:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.063881:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.063884:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.063886:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.063890:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.063892:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.063896:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.063898:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.063902:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.063904:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.063908:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.063911:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.063914:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.063917:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.063920:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.063923:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.063926:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.063929:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.063933:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.063935:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.063938:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.063941:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.063945:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.063947:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.063951:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.063953:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.063957:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.063959:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.063962:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.063965:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.063968:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.063971:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.063974:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.063977:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.063981:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.063984:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.064001:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.064005:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.064009:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.064013:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.064016:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.064019:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.064022:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.064025:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.064028:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.064031:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.064034:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.064036:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.064039:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.064042:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.064045:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.064047:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.064050:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.064053:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.064056:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.064059:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.064062:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.064065:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.064068:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.064071:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.064074:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.064077:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.064080:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.064083:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.064087:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.064090:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.064093:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.064096:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.064099:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.064102:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200080:3.0:1787826171.064105:0:158321:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffcbdeb90b0 00020000:00000080:3.0:1787826171.064108:0:158321:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c442cc00 uarg=0000000000000000 00000080:00200008:3.0:1787826171.064121:0:158321:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), flags 100000 00010000:00010000:1.0:1787826172.771838:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826174.829934:0:33329:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826175.521420:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2392:1874663042977152:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826175.525811:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2392:1874663042977408:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826175.534584:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2392:1874663042977152:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826175.534896:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2391:1874663042977280:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826175.536490:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7528148b-9bee-4d31-b597-8aae3b0b4982:2391:1874663042977024:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826175.536516:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2391:1874663042977536:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826175.536571:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2392:1874663042977408:192.168.203.147@tcp:400:kworker.0 00010000:00010000:0.0:1787826175.842563:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826176.868091:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:1.0:1787826177.950270:0:158605:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: oleg347-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 00000001:02000400:3.0:1787826179.743244:0:158753:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec 00010000:00010000:3.0:1787826179.942404:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826180.642550:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7528148b-9bee-4d31-b597-8aae3b0b4982:2390:1874663042977664:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826180.642652:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042977920:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826180.642680:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042978176:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826180.642709:0:2390:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1787826180.642715:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042977792:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826180.642780:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042978048:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826180.651012:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042977920:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826180.651058:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042977792:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826180.652272:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042978048:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826180.652285:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7528148b-9bee-4d31-b597-8aae3b0b4982:2390:1874663042977664:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826180.652297:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2390:1874663042978176:192.168.203.147@tcp:400:kworker.0 00010000:00010000:1.0:1787826180.961057:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200008:0.0:1787826181.606185:0:158791:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1787826181.606202:0:158791:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826181.606224:0:158791:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826181.606242:0:158791:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6f11d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced208 expref: -99 pid: 158321 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787826181.606257:0:158791:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6f11d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced208 expref: -99 pid: 158321 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787826181.606265:0:158791:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826181.606273:0:158791:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826181.606276:0:158791:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6f11d 00000080:00010000:0.0:1787826181.606283:0:158791:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f11d from it ffffb97500c97cb8 00010000:00010000:0.0:1787826181.606287:0:158791:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6f11d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced208 expref: -99 pid: 158321 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787826181.606296:0:158791:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6f11d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced208 expref: -99 pid: 158321 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00000080:00200008:0.0:1787826181.607706:0:158791:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1787826181.607715:0:158791:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826181.607733:0:158791:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826181.608011:0:158791:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6f11d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced208 expref: -99 pid: 158321 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:0.0:1787826181.608026:0:158791:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6f11d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced208 expref: -99 pid: 158321 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00000080:00200000:0.0:1787826181.608155:0:158791:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826181.608162:0:158791:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826181.608165:0:158791:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6f11d 00000080:00010000:0.0:1787826181.608172:0:158791:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f11d from it ffffb97500c97cb8 00010000:00010000:0.0:1787826181.608176:0:158791:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6f11d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced208 expref: -99 pid: 158321 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:0.0:1787826181.608185:0:158791:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6f11d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced208 expref: -99 pid: 158321 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00000080:00200000:0.0:1787826181.608312:0:158791:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826181.608322:0:158791:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826181.608330:0:158791:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6f11d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced208 expref: -99 pid: 158321 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00010000:00010000:0.0:1787826181.608339:0:158791:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6f11d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced208 expref: -99 pid: 158321 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00000080:00200000:0.0:1787826181.608346:0:158791:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826181.608350:0:158791:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826181.608352:0:158791:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6f11d 00000080:00010000:0.0:1787826181.608357:0:158791:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f11d from it ffffb97500c97bb0 00010000:00010000:0.0:1787826181.608360:0:158791:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6f11d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced208 expref: -99 pid: 158321 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00010000:00010000:0.0:1787826181.608367:0:158791:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6f11d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced208 expref: -99 pid: 158321 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00000080:00200000:0.0:1787826181.608376:0:158791:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 44 00000080:00200008:0.0:1787826181.608384:0:158791:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), flags 100000 00000080:00200080:0.0:1787826181.608559:0:158791:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) cmd=c00866a4 arg=7ffe01604ea0 00800000:00000080:0.0:1787826181.608576:0:158791:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c4d2b000: cmd=c00866a4 len=592 karg=ffff9181c343d400 uarg=0000000000000000 00800000:00000004:0.0:1787826181.608584:0:158791:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9181c4d2b000: age 8242, max_age 8252 00000100:00100000:0.0:1787826181.608630:0:158791:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job lfs:a9ce6693-0634-46c5-bf21-5ca7d026d44e:158791:1874663042978304:192.168.203.147@tcp:41:lfs.0 00000100:00100000:0.0:1787826181.608713:0:158791:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c7749000 going to sleep for 11 seconds 00000080:00200008:3.0:1787826181.667571:0:157888:0:(llite_lib.c:2427:ll_setattr_raw()) START file f73b.replay-single:[0x20001a9e3:0x1bd2:0x0](ffff9181d080cb08) current size 0, valid attrs 41, mode 8000, uid 1, gid 0, new size 18446744071748348645, atime 1787826181.000000000, mtime 1787826181.000000000, ctime 1787826181.000000000 00000020:00100000:3.0:1787826181.667644:0:157888:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:3.0:1787826181.667659:0:157888:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c5635880 x1874663042978944/t0(0) o36->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 488/512 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826181.667685:0:157888:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635880 pname:cluuid:pid:xid:nid:opc:job multiop:a9ce6693-0634-46c5-bf21-5ca7d026d44e:157888:1874663042978944:192.168.203.147@tcp:36:multiop.0 00000100:00100000:3.0:1787826181.667785:0:157888:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c55daf00 going to sleep for 11 seconds 00000100:00100000:0.0:1787826181.680650:0:157888:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635880 pname:cluuid:pid:xid:nid:opc:job multiop:a9ce6693-0634-46c5-bf21-5ca7d026d44e:157888:1874663042978944:192.168.203.147@tcp:36:multiop.0 00000080:00200000:0.0:1787826181.680706:0:157888:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1bd2:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00010000:0.0:1787826181.680730:0:157888:0:(file.c:7065:ll_layout_intent()) ### lustre: requeue layout lock for file [0x20001a9e3:0x1bd2:0x0](ffff9181d080cb08) 00000002:00010000:0.0:1787826181.680740:0:157888:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x20001a9e3:0x1bd2:0x0]) in obj [0x20001a9e3:0x1bd2:0x0], intent: layout flags 0 00010000:00010000:0.0:1787826181.680807:0:157888:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6f124 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x1bd2:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 157888 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826181.680817:0:157888:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6f124 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x1bd2:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x0 pid: 157888 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826181.680827:0:157888:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6f124 lrc: 3/1,0 mode: --/CR res: [0x20001a9e3:0x1bd2:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x0 pid: 157888 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826181.680847:0:157888:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job multiop:a9ce6693-0634-46c5-bf21-5ca7d026d44e:157888:1874663042979072:192.168.203.147@tcp:101:multiop.0 00000100:00100000:0.0:1787826181.680943:0:157888:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c7749900 going to sleep for 11 seconds 00000100:00100000:1.0:1787826181.691278:0:157888:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job multiop:a9ce6693-0634-46c5-bf21-5ca7d026d44e:157888:1874663042979072:192.168.203.147@tcp:101:multiop.0 00010000:00010000:1.0:1787826181.691313:0:157888:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6f124 lrc: 4/1,0 mode: CR/CR res: [0x20001a9e3:0x1bd2:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced3d6 expref: -99 pid: 157888 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826181.691329:0:157888:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6f124 lrc: 4/1,0 mode: CR/CR res: [0x20001a9e3:0x1bd2:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced3d6 expref: -99 pid: 157888 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826181.691344:0:157888:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x400 disposition=0x1, status=0 req@ffff9181c55a7b80 x1874663042979072/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 376/400 e 0 to 0 dl 1787826197 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'multiop.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826181.691363:0:157888:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: layout, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6f124 lrc: 3/1,0 mode: CR/CR res: [0x20001a9e3:0x1bd2:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced3d6 expref: -99 pid: 157888 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00000002:00010000:1.0:1787826181.691375:0:157888:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6f124 lrc: 3/1,0 mode: CR/CR res: [0x20001a9e3:0x1bd2:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced3d6 expref: -99 pid: 157888 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826181.691389:0:157888:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 16 res 8590043619/7122 (0 0) 00000080:00200000:1.0:1787826181.691396:0:157888:0:(namei.c:952:ll_intent_lock()) intent lock 1024 on i1 [0x20001a9e3:0x1bd2:0x0] suppgids 0 0: rc 0 00000080:00010000:1.0:1787826181.691416:0:157888:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x1bd2:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f124 00000080:00010000:1.0:1787826181.691425:0:157888:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f124 from it ffffb97501dfbaf8 00010000:00010000:1.0:1787826181.691429:0:157888:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6f124 lrc: 3/1,0 mode: CR/CR res: [0x20001a9e3:0x1bd2:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced3d6 expref: -99 pid: 157888 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826181.691439:0:157888:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6f124 lrc: 2/0,0 mode: CR/CR res: [0x20001a9e3:0x1bd2:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced3d6 expref: -99 pid: 157888 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826181.691466:0:157888:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6f124 lrc: 2/1,0 mode: CR/CR res: [0x20001a9e3:0x1bd2:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced3d6 expref: -99 pid: 157888 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826181.691476:0:157888:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6f124 lrc: 2/1,0 mode: CR/CR res: [0x20001a9e3:0x1bd2:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced3d6 expref: -99 pid: 157888 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00010000:1.0:1787826181.691484:0:157888:0:(file.c:6948:ll_layout_lock_set()) ### file [0x20001a9e3:0x1bd2:0x0](00000000239e286a) being reconfigured ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6f124 lrc: 3/1,0 mode: CR/CR res: [0x20001a9e3:0x1bd2:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced3d6 expref: -99 pid: 157888 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826181.691522:0:157888:0:(file.c:6843:ll_layout_conf()) [0x20001a9e3:0x1bd2:0x0]: layout version change: 4294967294 -> 0 00010000:00010000:1.0:1787826181.691528:0:157888:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6f124 lrc: 3/1,0 mode: CR/CR res: [0x20001a9e3:0x1bd2:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00ced3d6 expref: -99 pid: 157888 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826181.691549:0:157888:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6f124 lrc: 2/0,0 mode: CR/CR res: [0x20001a9e3:0x1bd2:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00ced3d6 expref: -99 pid: 157888 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00020000:00200000:1.0:1787826181.691572:0:157888:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000100:00100000:1.0:1787826181.691655:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042979200:192.168.203.147@tcp:2:multiop.0 00000100:00100000:0.0:1787826181.701224:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042979200:192.168.203.147@tcp:2:multiop.0 00000100:00100000:3.0:1787826181.711060:0:157888:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f1500 pname:cluuid:pid:xid:nid:opc:job multiop:a9ce6693-0634-46c5-bf21-5ca7d026d44e:157888:1874663042979328:192.168.203.147@tcp:101:multiop.0 00010000:00010000:3.0:1787826181.711092:0:157888:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7200/0xbef42c0221d6f12b lrc: 4/1,0 mode: --/CR res: [0x20001a9e3:0x1bd2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced3e4 expref: -99 pid: 157888 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826181.711106:0:157888:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7200/0xbef42c0221d6f12b lrc: 4/1,0 mode: --/PR res: [0x20001a9e3:0x1bd2:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced3e4 expref: -99 pid: 157888 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826181.711120:0:157888:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7200/0xbef42c0221d6f12b lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bd2:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced3e4 expref: -99 pid: 157888 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826181.711131:0:157888:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7200/0xbef42c0221d6f12b lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bd2:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced3e4 expref: -99 pid: 157888 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826181.711145:0:157888:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7200/0xbef42c0221d6f12b lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1bd2:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced3e4 expref: -99 pid: 157888 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826181.711153:0:157888:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7200/0xbef42c0221d6f12b lrc: 5/2,0 mode: PR/PR res: [0x20001a9e3:0x1bd2:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced3e4 expref: -99 pid: 157888 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826181.711162:0:157888:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7200/0xbef42c0221d6f12b lrc: 4/1,0 mode: PR/PR res: [0x20001a9e3:0x1bd2:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced3e4 expref: -99 pid: 157888 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826181.711174:0:157888:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c71f1500 x1874663042979328/t0(0) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 576/640 e 0 to 0 dl 1787826197 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'multiop.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826181.711185:0:157888:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: getattr, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7200/0xbef42c0221d6f12b lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1bd2:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced3e4 expref: -99 pid: 157888 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:3.0:1787826181.711199:0:157888:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7200/0xbef42c0221d6f12b lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1bd2:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced3e4 expref: -99 pid: 157888 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826181.711213:0:157888:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c59cc800 type 13 mode 4 res 8590043619/7122 (0 0) 00000080:00200000:3.0:1787826181.711220:0:157888:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x20001a9e3:0x1bd2:0x0] suppgids 0 0: rc 0 00000080:00200000:3.0:1787826181.711267:0:157888:0:(file.c:6843:ll_layout_conf()) [0x20001a9e3:0x1bd2:0x0]: layout version change: 0 -> 0 00000080:00010000:3.0:1787826181.711272:0:157888:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20001a9e3:0x1bd2:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f12b 00000080:00010000:3.0:1787826181.711278:0:157888:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20001a9e3:0x1bd2:0x0](ffff9181d080cb08) 00000080:00010000:3.0:1787826181.711281:0:157888:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f12b from it ffffb97501dfbcd8 00010000:00010000:3.0:1787826181.711284:0:157888:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7200/0xbef42c0221d6f12b lrc: 3/1,0 mode: PR/PR res: [0x20001a9e3:0x1bd2:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00ced3e4 expref: -99 pid: 157888 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826181.711292:0:157888:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7200/0xbef42c0221d6f12b lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1bd2:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00ced3e4 expref: -99 pid: 157888 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826181.711328:0:157888:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1bd2:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:3.0:1787826181.711347:0:157888:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x20001a9e3:0x1bd2:0x0] 00000008:00010000:3.0:1787826181.711392:0:157888:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: a 00000008:00010000:3.0:1787826181.711395:0:157888:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff9181c5c6f9b0, osc lock ffff9181c7d405f0, flags 0x400001002 00010000:00010000:3.0:1787826181.711412:0:157888:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff9181c4d2b000 lock: 00000000ffd0d697/0xbef42c0221d6f0e5 lrc: 2/1,0 mode: PR/PR res: [0x280000401:0x1862:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x878a5eea00cecef1 expref: -99 pid: 157893 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826181.711427:0:157888:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 18446744073709551615) ns: lustre-OST0000-osc-ffff9181c4d2b000 lock: 00000000ffd0d697/0xbef42c0221d6f0e5 lrc: 2/1,0 mode: PR/PR res: [0x280000401:0x1862:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x878a5eea00cecef1 expref: -99 pid: 157893 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000020:00200000:3.0:1787826181.711444:0:157888:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00010000:00010000:3.0:1787826181.711448:0:157888:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9181c4d2b000 lock: 00000000ffd0d697/0xbef42c0221d6f0e5 lrc: 5/1,0 mode: PR/PR res: [0x280000401:0x1862:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x878a5eea00cecef1 expref: -99 pid: 157893 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826181.711456:0:157888:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff9181c4d2b000 lock: 00000000ffd0d697/0xbef42c0221d6f0e5 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x1862:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x878a5eea00cecef1 expref: -99 pid: 157893 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000020:00200000:3.0:1787826181.711466:0:157888:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:3.0:1787826181.711475:0:157888:0:(file.c:1775:ll_merge_attr_nolock()) [0x20001a9e3:0x1bd2:0x0] updating i_size 0 i_blocks 0 00000020:00010000:3.0:1787826181.711478:0:157888:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9181c563ebf0 (ffff9181c4405128/0) at cl_lock_release():245 00000020:00010000:3.0:1787826181.711481:0:157888:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9181c563ebf0 (ffff9181c4405128/0) at cl_lock_cancel():142 00000020:00010000:3.0:1787826181.711483:0:157888:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9181c5c6f9b0 (ffff9181c4405758/1) at cl_lock_cancel():142 00000020:00010000:3.0:1787826181.711487:0:157888:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9181c563ebf0 (ffff9181c4405128/0) at cl_lock_fini():72 00000020:00010000:3.0:1787826181.711490:0:157888:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9181c5c6f9b0 (ffff9181c4405128/1) at cl_lock_fini():72 00000080:00200000:3.0:1787826181.711498:0:157888:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1bd2:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200008:3.0:1787826181.711521:0:157888:0:(file.c:479:ll_file_release()) START file f73b.replay-single:[0x20001a9e3:0x1bd2:0x0](ffff9181d080cb08), flags 100002 00000002:00100000:3.0:1787826181.711569:0:157888:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181c71f1880 x1874663042965504/t339302416387(339302416387) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 592/608 e 0 to 0 dl 1787826186 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00000020:00100000:3.0:1787826181.711579:0:157888:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:3.0:1787826181.711584:0:157888:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c5635180 x1874663042979584/t0(0) o35->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:3.0:1787826181.711603:0:157888:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635180 pname:cluuid:pid:xid:nid:opc:job multiop:a9ce6693-0634-46c5-bf21-5ca7d026d44e:157888:1874663042979584:192.168.203.147@tcp:35:multiop.0 00000100:00100000:3.0:1787826181.711678:0:157888:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c55daa80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826181.720833:0:157888:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635180 pname:cluuid:pid:xid:nid:opc:job multiop:a9ce6693-0634-46c5-bf21-5ca7d026d44e:157888:1874663042979584:192.168.203.147@tcp:35:multiop.0 00000002:00100000:3.0:1787826181.720870:0:157888:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181c71f1880 x1874663042965504/t339302416387(339302416387) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 592/608 e 0 to 0 dl 1787826186 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00000080:00200000:3.0:1787826181.720927:0:157888:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1bd2:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826181.720940:0:157888:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826181.720966:0:157888:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1bd2:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826181.720984:0:157888:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1bd2:0x0](ffff9181d080cb08) 00000008:00010000:3.0:1787826181.721014:0:157888:0:(osc_object.c:202:osc_object_ast_clear()) ### update lock size 0 blocks 0 [cma]time: 0 0 0 by oinfo size 0 blocks 0 [cma]time 1787826181 0 0 ns: lustre-OST0000-osc-ffff9181c4d2b000 lock: 00000000ffd0d697/0xbef42c0221d6f0e5 lrc: 1/0,0 mode: PR/PR res: [0x280000401:0x1862:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x878a5eea00cecef1 expref: -99 pid: 157893 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787826181.722186:0:157751:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:0.0:1787826181.722210:0:157751:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826181.722225:0:157751:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6f11d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced208 expref: -99 pid: 158321 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00010000:00010000:0.0:1787826181.722240:0:157751:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6f11d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced208 expref: -99 pid: 158321 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00000080:00200000:0.0:1787826181.722254:0:157751:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826181.722261:0:157751:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:0.0:1787826181.722264:0:157751:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6f11d 00000080:00010000:0.0:1787826181.722271:0:157751:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f11d from it ffffb97501813b58 00010000:00010000:0.0:1787826181.722275:0:157751:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6f11d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced208 expref: -99 pid: 158321 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00010000:00010000:0.0:1787826181.722284:0:157751:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6f11d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced208 expref: -99 pid: 158321 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00000080:00200000:0.0:1787826181.722299:0:157751:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826181.722313:0:157751:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f73b.replay-single, dir=[0x200000007:0x1:0x0](ffff9181c41acb08), flags=9 00000080:00200000:0.0:1787826181.722317:0:157751:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f73b.replay-single, dir=[0x200000007:0x1:0x0](ffff9181c41acb08), intent=getattr 00000002:00010000:0.0:1787826181.722329:0:157751:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f73b.replay-single,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787826181.722371:0:157751:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6f132 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 157751 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826181.722379:0:157751:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6f132 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 157751 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826181.722389:0:157751:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513e00/0xbef42c0221d6f132 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 157751 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826181.722412:0:157751:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job bash:a9ce6693-0634-46c5-bf21-5ca7d026d44e:157751:1874663042979712:192.168.203.147@tcp:101:bash.0 00000100:00100000:0.0:1787826181.722491:0:157751:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c7749f00 going to sleep for 11 seconds 00010000:00010000:1.0:1787826181.985168:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826184.037071:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826185.060290:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826185.761090:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042979968:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826185.761180:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042980224:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826185.761213:0:2389:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:0.0:1787826185.761220:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7528148b-9bee-4d31-b597-8aae3b0b4982:2389:1874663042979840:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826185.761254:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042980096:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826185.761278:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042980352:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826185.770218:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7528148b-9bee-4d31-b597-8aae3b0b4982:2389:1874663042979840:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826185.770243:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042980352:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826185.770256:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042980224:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826185.770713:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042980096:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826185.770725:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2389:1874663042979968:192.168.203.147@tcp:400:kworker.0 00010000:00010000:1.0:1787826186.089281:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826187.106106:0:33329:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826187.396229:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826187.396273:0:33333:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049309184 00000100:00100000:1.0:1787826187.396292:0:33333:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049309184 00000100:00100000:1.0:1787826187.396340:0:33333:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 9268 00000100:00100000:1.0:1787826187.396350:0:33333:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b1880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:LOV_OSC_UUID+4:177402:x1874663049309184:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826187.396367:0:33333:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-OST0000-osc-ffff9181c4d2b000 lock: 00000000ffd0d697/0xbef42c0221d6f0e5 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x1862:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x420000010000 nid: local remote: 0x878a5eea00cecef1 expref: -99 pid: 157893 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826187.396445:0:33333:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826187.396453:0:33333:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b1880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:LOV_OSC_UUID+4:177402:x1874663049309184:12345-192.168.203.147@tcp:104: Request processed in 104us (245us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826187.396460:0:33333:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 9268 00010000:00010000:3.0:1787826187.396906:0:15160:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826187.396914:0:15160:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0000-osc-ffff9181c4d2b000 lock: 00000000ffd0d697/0xbef42c0221d6f0e5 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x1862:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x420000010000 nid: local remote: 0x878a5eea00cecef1 expref: -99 pid: 157893 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826187.396934:0:15160:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3480e00 already unused, calling callback (ffffffffc10ec300) 00010000:00010000:3.0:1787826187.396940:0:15160:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0000-osc-ffff9181c4d2b000 lock: 00000000ffd0d697/0xbef42c0221d6f0e5 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x1862:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x428400010000 nid: local remote: 0x878a5eea00cecef1 expref: -99 pid: 157893 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826187.396985:0:15160:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0000-osc-ffff9181c4d2b000 lock: 00000000ffd0d697/0xbef42c0221d6f0e5 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x1862:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4c29400010020 nid: local remote: 0x878a5eea00cecef1 expref: -99 pid: 157893 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826187.396992:0:15160:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826187.397023:0:15160:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0000-osc-ffff9181c4d2b000 lock: 00000000ffd0d697/0xbef42c0221d6f0e5 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x1862:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4c29400010020 nid: local remote: 0x878a5eea00cecef1 expref: -99 pid: 157893 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826187.397036:0:15160:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0000-osc-ffff9181c4d2b000 lock: 00000000ffd0d697/0xbef42c0221d6f0e5 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x1862:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4c29400010020 nid: local remote: 0x878a5eea00cecef1 expref: -99 pid: 157893 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826187.397058:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826187.397060:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826187.397062:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826187.397081:0:2391:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787826187.397095:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2391:1874663042980480:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1787826187.417534:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2391:1874663042980480:192.168.203.147@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826188.142957:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:0.0:1787826188.635663:0:159380:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == replay-single test 74: Ensure applications don't fail waiting for OST recovery ========================================================== 06:23:07 (1787826187) 00000080:00200008:3.0:1787826188.806474:0:159397:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181c41acb08), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1787826188.806503:0:159397:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08),name=/ 00000002:00010000:3.0:1787826188.806530:0:159397:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826188.806548:0:159397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6f11d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced208 expref: -99 pid: 158321 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00010000:00010000:3.0:1787826188.806563:0:159397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6f11d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced208 expref: -99 pid: 158321 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00000080:00200000:3.0:1787826188.806570:0:159397:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826188.806579:0:159397:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00010000:3.0:1787826188.806582:0:159397:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181c41acb08) for lock 0xbef42c0221d6f11d 00000080:00010000:3.0:1787826188.806590:0:159397:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f11d from it ffffb97501ce3cb8 00010000:00010000:3.0:1787826188.806595:0:159397:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6f11d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced208 expref: -99 pid: 158321 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00010000:00010000:3.0:1787826188.806607:0:159397:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6f11d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced208 expref: -99 pid: 158321 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00010000:00010000:1.0:1787826189.155446:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200000:1.0:1787826191.192614:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181c41acb08) 00000080:00200000:1.0:1787826191.192675:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x200001b71:0x9:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.192689:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.192727:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b71:0x9:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.192733:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200001b71:0x9:0x0](ffff9181c41ab248) 00000080:00200000:1.0:1787826191.192821:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200001b71:0xb:0x0](ffff9181c418e3c8) 00000080:00200000:1.0:1787826191.192832:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200003ab0:0x1:0x0](ffff9181c41b7448) 00000080:00200000:1.0:1787826191.193922:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000402:0x1:0x0](ffff9181c41b21c8) 00000080:00200000:1.0:1787826191.193962:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x200001b71:0xc:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.193971:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b71:0xc:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.193976:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200001b71:0xc:0x0](ffff9181c4119148) 00000080:00200000:1.0:1787826191.193993:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x200001b71:0x15:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.194000:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.194016:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b71:0x15:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.194020:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200001b71:0x15:0x0](ffff9181c41aa1c8) 00000080:00200000:1.0:1787826191.194048:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x200001b71:0x16:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.194054:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.194061:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b71:0x16:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.194065:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200001b71:0x16:0x0](ffff9181c418f448) 00000080:00200000:1.0:1787826191.194089:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x200001b71:0x17:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.194094:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.194100:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b71:0x17:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.194104:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200001b71:0x17:0x0](ffff9181c418a1c8) 00000080:00200000:1.0:1787826191.194123:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x200001b71:0x18:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.194128:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.194134:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b71:0x18:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.194138:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200001b71:0x18:0x0](ffff9181c41af448) 00000080:00200000:1.0:1787826191.194157:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x200001b71:0x19:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.194162:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.194169:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b71:0x19:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.194173:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200001b71:0x19:0x0](ffff9181c41a80c8) 00000080:00200000:1.0:1787826191.194197:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x200001b71:0x1a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.194201:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.194207:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b71:0x1a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.194212:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200001b71:0x1a:0x0](ffff9181c4189988) 00000080:00200000:1.0:1787826191.194232:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x200001b71:0x1b:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.194236:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.194242:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b71:0x1b:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.194246:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200001b71:0x1b:0x0](ffff9181c411f448) 00000080:00200000:1.0:1787826191.194266:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x200001b71:0x1c:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.194271:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.194277:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b71:0x1c:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.194281:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200001b71:0x1c:0x0](ffff9181c41aec08) 00000080:00200000:1.0:1787826191.194301:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x200001b71:0x1d:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.194306:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.194312:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b71:0x1d:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.194316:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200001b71:0x1d:0x0](ffff9181c418db88) 00000080:00200000:1.0:1787826191.194334:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x200001b71:0x1e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.194339:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.194345:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b71:0x1e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.194349:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200001b71:0x1e:0x0](ffff9181c418ba88) 00000080:00200000:1.0:1787826191.194367:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200001b71:0x107:0x0](ffff9181c428ba88) 00000080:00200000:1.0:1787826191.194380:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x200001b71:0x109:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.194385:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b71:0x109:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.194388:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200001b71:0x109:0x0](ffff9181c426b248) 00000080:00200000:1.0:1787826191.194402:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x200001b71:0x115:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.194406:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.194412:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b71:0x115:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.194416:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200001b71:0x115:0x0](ffff9181c4268908) 00000080:00200000:1.0:1787826191.194441:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x200001b71:0x118:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.194446:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.194452:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b71:0x118:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.194456:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200001b71:0x118:0x0](ffff9181c428cb08) 00000080:00200000:1.0:1787826191.194477:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x200001b71:0x122:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.194481:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.194488:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b71:0x122:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.194492:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200001b71:0x122:0x0](ffff9181c41b6c08) 00000080:00200000:1.0:1787826191.194515:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x200001b71:0x123:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.194520:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.194525:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b71:0x123:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.194529:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200001b71:0x123:0x0](ffff9181c426d348) 00000080:00200000:1.0:1787826191.194552:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x200001b71:0x12e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.194556:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.194563:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b71:0x12e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.194566:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200001b71:0x12e:0x0](ffff9181c42600c8) 00000080:00200000:1.0:1787826191.194588:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20000fe01:0x1:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.194592:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.194600:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20000fe01:0x1:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.194603:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20000fe01:0x1:0x0](ffff9181c428ec08) 00000080:00200000:1.0:1787826191.194627:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20000fe02:0x3:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.194632:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.194639:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20000fe02:0x3:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.194643:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20000fe02:0x3:0x0](ffff9181c4289148) 00000080:00200000:1.0:1787826191.194666:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20000fe02:0x4:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.194671:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.194678:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20000fe02:0x4:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.194682:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20000fe02:0x4:0x0](ffff9181c4269148) 00000080:00200000:1.0:1787826191.194880:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20000fe02:0x9:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.194887:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.194898:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20000fe02:0x9:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.194904:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20000fe02:0x9:0x0](ffff9181c41b5348) 00000080:00200000:1.0:1787826191.194966:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20000fe02:0xa:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.194971:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.194979:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20000fe02:0xa:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.194983:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20000fe02:0xa:0x0](ffff9181c42680c8) 00000080:00200000:1.0:1787826191.195008:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20000fe02:0x21:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.195013:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.195021:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20000fe02:0x21:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.195025:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20000fe02:0x21:0x0](ffff9181c4289988) 00000080:00200000:1.0:1787826191.195049:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000153f1:0x1:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.195053:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.195061:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000153f1:0x1:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.195065:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000153f1:0x1:0x0](ffff9181c428a1c8) 00000080:00200000:1.0:1787826191.195088:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000153f1:0x2:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.195092:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.195100:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000153f1:0x2:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.195104:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000153f1:0x2:0x0](ffff9181c428e3c8) 00000080:00200000:1.0:1787826191.195127:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000153f1:0x3:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.195132:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.195138:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000153f1:0x3:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.195142:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000153f1:0x3:0x0](ffff9181c426a1c8) 00000080:00200000:1.0:1787826191.195162:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000153f1:0x4:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.195168:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.195174:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000153f1:0x4:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.195179:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000153f1:0x4:0x0](ffff9181c41b5b88) 00000080:00200000:1.0:1787826191.196945:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000153f1:0x5:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.196960:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.196981:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000153f1:0x5:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.196987:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000153f1:0x5:0x0](ffff9181c41b42c8) 00000080:00200000:1.0:1787826191.197017:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000153f1:0x6:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.197022:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.197030:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000153f1:0x6:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.197034:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000153f1:0x6:0x0](ffff9181c426f448) 00000080:00200000:1.0:1787826191.197062:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000153f1:0x7:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.197067:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.197075:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000153f1:0x7:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.197079:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000153f1:0x7:0x0](ffff9181c426e3c8) 00000080:00200000:1.0:1787826191.197103:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000153f1:0x8:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.197108:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.197117:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000153f1:0x8:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.197120:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000153f1:0x8:0x0](ffff9181c428f448) 00000080:00200000:1.0:1787826191.197147:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000153f1:0x9:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.197151:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.197157:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000153f1:0x9:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.197160:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000153f1:0x9:0x0](ffff9181c42880c8) 00000080:00200000:1.0:1787826191.197179:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000153f1:0xa:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.197183:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.197189:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000153f1:0xa:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.197192:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000153f1:0xa:0x0](ffff9181c428d348) 00000080:00200000:1.0:1787826191.197212:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x200015bc1:0xb:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.197217:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.197223:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x200015bc1:0xb:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.197227:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200015bc1:0xb:0x0](ffff9181c426aa08) 00000080:00200000:1.0:1787826191.197248:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x200015bc1:0xc:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.197253:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.197259:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x200015bc1:0xc:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.197263:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200015bc1:0xc:0x0](ffff9181c4239148) 00000080:00200000:1.0:1787826191.197286:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x200015bc1:0xd:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.197291:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.197298:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x200015bc1:0xd:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.197302:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200015bc1:0xd:0x0](ffff9181c42380c8) 00000080:00200000:1.0:1787826191.197324:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x200015bc1:0xe:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.197329:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.197336:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x200015bc1:0xe:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.197339:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200015bc1:0xe:0x0](ffff9181c4264b08) 00000080:00200000:1.0:1787826191.197366:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x200015bc1:0xf:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.197371:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.197377:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x200015bc1:0xf:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.197382:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200015bc1:0xf:0x0](ffff9181c4262a08) 00000080:00200000:1.0:1787826191.197403:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x200015bc1:0x10:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.197408:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.197415:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x200015bc1:0x10:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.197419:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200015bc1:0x10:0x0](ffff9181c426c2c8) 00000080:00200000:1.0:1787826191.197441:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x200015bc1:0x11:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.197446:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.197452:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x200015bc1:0x11:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.197456:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200015bc1:0x11:0x0](ffff9181c423ec08) 00000080:00200000:1.0:1787826191.197478:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x200015bc1:0x12:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.197483:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.197489:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x200015bc1:0x12:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.197493:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200015bc1:0x12:0x0](ffff9181c423d348) 00000080:00200000:1.0:1787826191.197516:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x200015bc1:0x13:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.197521:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.197528:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x200015bc1:0x13:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.197532:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200015bc1:0x13:0x0](ffff9181c426cb08) 00000080:00200000:1.0:1787826191.197556:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x200015bc1:0x14:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.197560:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.197567:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x200015bc1:0x14:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.197571:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200015bc1:0x14:0x0](ffff9181c423c2c8) 00000080:00200000:1.0:1787826191.197594:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x200017331:0x1:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.197598:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.197605:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x200017331:0x1:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.197609:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200017331:0x1:0x0](ffff9181c423cb08) 00000080:00200000:1.0:1787826191.197631:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200017b01:0x1:0x0](ffff9181c426ec08) 00000080:00200000:1.0:1787826191.197645:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x643:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.197650:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.197657:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x643:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.197661:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x643:0x0](ffff9181c6978908) 00000080:00200000:1.0:1787826191.197686:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x968:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.197690:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x968:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.197693:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x968:0x0](ffff9181c4279148) 00000080:00200000:1.0:1787826191.200360:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x969:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.200375:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x969:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.200383:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x969:0x0](ffff9181c427e3c8) 00000080:00200000:1.0:1787826191.200412:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x96a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.200417:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x96a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.200420:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x96a:0x0](ffff9181c427f448) 00000080:00200000:1.0:1787826191.200434:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x96b:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.200438:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x96b:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.200441:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x96b:0x0](ffff9181c6ad8908) 00000080:00200000:1.0:1787826191.200453:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x96c:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.200457:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x96c:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.200460:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x96c:0x0](ffff9181c6adc2c8) 00000080:00200000:1.0:1787826191.200472:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x96d:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.200475:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x96d:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.200478:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x96d:0x0](ffff9181c6a65348) 00000080:00200000:1.0:1787826191.200492:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x96e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.200495:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x96e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.200499:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x96e:0x0](ffff9181c6a642c8) 00000080:00200000:1.0:1787826191.200513:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x96f:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.200517:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x96f:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.200520:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x96f:0x0](ffff9181c6add348) 00000080:00200000:1.0:1787826191.200534:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x970:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.200538:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x970:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.200542:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x970:0x0](ffff9181c6a60908) 00000080:00200000:1.0:1787826191.200556:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x971:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.200560:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x971:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.200566:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x971:0x0](ffff9181c6a64b08) 00000080:00200000:1.0:1787826191.200580:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x972:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.200583:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x972:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.200587:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x972:0x0](ffff9181c6a62a08) 00000080:00200000:1.0:1787826191.200599:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x973:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.200603:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x973:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.200607:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x973:0x0](ffff9181c6a63248) 00000080:00200000:1.0:1787826191.200620:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x974:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.200624:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x974:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.200627:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x974:0x0](ffff9181c6a63a88) 00000080:00200000:1.0:1787826191.200640:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x975:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.200643:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x975:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.200647:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x975:0x0](ffff9181c6adec08) 00000080:00200000:1.0:1787826191.200659:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x976:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.200663:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x976:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.200666:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x976:0x0](ffff9181c6ad80c8) 00000080:00200000:1.0:1787826191.200678:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x977:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.200681:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x977:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.200685:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x977:0x0](ffff9181c6ad9148) 00000080:00200000:1.0:1787826191.200696:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x978:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.200700:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x978:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.200703:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x978:0x0](ffff9181c6a66c08) 00000080:00200000:1.0:1787826191.200714:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x979:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.200718:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x979:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.200721:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x979:0x0](ffff9181c6a600c8) 00000080:00200000:1.0:1787826191.200734:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x97a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.200737:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x97a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.200740:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x97a:0x0](ffff9181c6a61148) 00000080:00200000:1.0:1787826191.200777:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x97b:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.200780:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x97b:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.200783:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x97b:0x0](ffff9181c6a663c8) 00000080:00200000:1.0:1787826191.200794:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x97c:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.200798:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x97c:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.200801:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x97c:0x0](ffff9181c6a67448) 00000080:00200000:1.0:1787826191.200812:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x97d:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.200815:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x97d:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.200819:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x97d:0x0](ffff9181c6a621c8) 00000080:00200000:1.0:1787826191.200830:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x97e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.200833:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x97e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.200837:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x97e:0x0](ffff9181c6a61988) 00000080:00200000:1.0:1787826191.200858:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x97f:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.200863:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x97f:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.200866:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x97f:0x0](ffff9181c6a65b88) 00000080:00200000:1.0:1787826191.200880:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x980:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.200883:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x980:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.200886:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x980:0x0](ffff9181c6ade3c8) 00000080:00200000:1.0:1787826191.200897:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x981:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.200900:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x981:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.200903:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x981:0x0](ffff9181c6adf448) 00000080:00200000:1.0:1787826191.200914:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x982:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.200917:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x982:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.200921:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x982:0x0](ffff9181c6ad2a08) 00000080:00200000:1.0:1787826191.200932:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x983:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.200935:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x983:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.200939:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x983:0x0](ffff9181c6ad3248) 00000080:00200000:1.0:1787826191.200952:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x984:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.200955:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x984:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.200959:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x984:0x0](ffff9181c6ad3a88) 00000080:00200000:1.0:1787826191.200971:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x985:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.200974:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x985:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.200978:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x985:0x0](ffff9181c6ad5b88) 00000080:00200000:1.0:1787826191.200991:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x986:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.200995:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x986:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.200998:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x986:0x0](ffff9181c6ad1988) 00000080:00200000:1.0:1787826191.201012:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x987:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.201015:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x987:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.201019:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x987:0x0](ffff9181c6ad21c8) 00000080:00200000:1.0:1787826191.201031:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x988:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.201035:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x988:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.201038:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x988:0x0](ffff9181c6ad7448) 00000080:00200000:1.0:1787826191.201053:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x989:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.201057:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x989:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.201061:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x989:0x0](ffff9181c6ad63c8) 00000080:00200000:1.0:1787826191.201074:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x98a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.201077:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x98a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.201080:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x98a:0x0](ffff9181c6ad1148) 00000080:00200000:1.0:1787826191.201093:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x98b:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.201097:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x98b:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.201100:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x98b:0x0](ffff9181c6ad00c8) 00000080:00200000:1.0:1787826191.201112:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x98c:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.201115:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x98c:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.201117:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x98c:0x0](ffff9181c6ad6c08) 00000080:00200000:1.0:1787826191.201127:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x98d:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.201130:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x98d:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.201133:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x98d:0x0](ffff9181c6ad5348) 00000080:00200000:1.0:1787826191.201144:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x98e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.201148:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x98e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.201151:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x98e:0x0](ffff9181c6ada1c8) 00000080:00200000:1.0:1787826191.201164:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x98f:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.201168:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x98f:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.201171:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x98f:0x0](ffff9181c6ad42c8) 00000080:00200000:1.0:1787826191.201183:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x990:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.201186:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x990:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.201190:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x990:0x0](ffff9181c6ad9988) 00000080:00200000:1.0:1787826191.201200:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x991:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.201203:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x991:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.201206:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x991:0x0](ffff9181c6addb88) 00000080:00200000:1.0:1787826191.201218:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x992:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.201221:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x992:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.201224:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x992:0x0](ffff9181c6ad0908) 00000080:00200000:1.0:1787826191.201236:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x993:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.201239:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x993:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.201243:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x993:0x0](ffff9181c6ad4b08) 00000080:00200000:1.0:1787826191.201253:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x994:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.201256:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x994:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.201259:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x994:0x0](ffff9181c6a0cb08) 00000080:00200000:1.0:1787826191.201269:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x995:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.201273:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x995:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.201276:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x995:0x0](ffff9181c6a0aa08) 00000080:00200000:1.0:1787826191.201287:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x996:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.201290:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x996:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.201293:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x996:0x0](ffff9181c6a0b248) 00000080:00200000:1.0:1787826191.201305:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x997:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.201308:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x997:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.201311:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x997:0x0](ffff9181c6a0ba88) 00000080:00200000:1.0:1787826191.201324:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x998:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.201327:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x998:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.201331:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x998:0x0](ffff9181c6a0db88) 00000080:00200000:1.0:1787826191.201342:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x999:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.201346:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x999:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.201349:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x999:0x0](ffff9181c6a09988) 00000080:00200000:1.0:1787826191.201361:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x99a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.201364:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x99a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.201367:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x99a:0x0](ffff9181c6adba88) 00000080:00200000:1.0:1787826191.201378:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x99b:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.201382:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x99b:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.201385:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x99b:0x0](ffff9181c428c2c8) 00000080:00200000:1.0:1787826191.201398:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x99c:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.201401:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x99c:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.201404:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x99c:0x0](ffff9181c4288908) 00000080:00200000:1.0:1787826191.201413:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x99d:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.201416:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x99d:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.201420:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x99d:0x0](ffff9181c6a0a1c8) 00000080:00200000:1.0:1787826191.201431:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x99e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.201435:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x99e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.201438:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x99e:0x0](ffff9181c6a0f448) 00000080:00200000:1.0:1787826191.201449:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x99f:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.201452:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x99f:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.201455:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x99f:0x0](ffff9181c6a0e3c8) 00000080:00200000:1.0:1787826191.201468:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x9a0:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.201472:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x9a0:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.201475:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x9a0:0x0](ffff9181c6adb248) 00000080:00200000:1.0:1787826191.201487:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x9a1:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.201491:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x9a1:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.201495:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x9a1:0x0](ffff9181c6adaa08) 00000080:00200000:1.0:1787826191.201507:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x9a2:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.201510:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x9a2:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.201513:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x9a2:0x0](ffff9181c6adcb08) 00000080:00200000:1.0:1787826191.201525:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x9a3:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.201529:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x9a3:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.201533:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x9a3:0x0](ffff9181c428aa08) 00000080:00200000:1.0:1787826191.201546:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x9a4:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.201550:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x9a4:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.201554:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x9a4:0x0](ffff9181c6a09148) 00000080:00200000:1.0:1787826191.201568:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x9a5:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.201571:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x9a5:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.201575:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x9a5:0x0](ffff9181c6a080c8) 00000080:00200000:1.0:1787826191.201588:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x9a6:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.201591:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x9a6:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.201595:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x9a6:0x0](ffff9181c6a0ec08) 00000080:00200000:1.0:1787826191.201608:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x9a7:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.201611:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x9a7:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.201615:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x9a7:0x0](ffff9181c6a0d348) 00000080:00200000:1.0:1787826191.201628:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x9a8:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.201631:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x9a8:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.201635:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x9a8:0x0](ffff9181c6a0c2c8) 00000080:00200000:1.0:1787826191.201649:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x9a9:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.201652:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x9a9:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.201656:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x9a9:0x0](ffff9181c6a08908) 00000080:00200000:1.0:1787826191.201671:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x9aa:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.201675:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x9aa:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.201678:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x9aa:0x0](ffff9181c6a042c8) 00000080:00200000:1.0:1787826191.201688:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x9ab:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.201692:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x9ab:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.201695:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x9ab:0x0](ffff9181c6a00908) 00000080:00200000:1.0:1787826191.201706:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x9ac:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.201709:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x9ac:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.201712:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x9ac:0x0](ffff9181c6a04b08) 00000080:00200000:1.0:1787826191.201722:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x9ad:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.201725:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x9ad:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.201728:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x9ad:0x0](ffff9181c6a02a08) 00000080:00200000:1.0:1787826191.201738:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x9ae:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.201769:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x9ae:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.201773:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x9ae:0x0](ffff9181c6a03248) 00000080:00200000:1.0:1787826191.201788:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x9af:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.201791:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x9af:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.201794:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x9af:0x0](ffff9181c6a03a88) 00000080:00200000:1.0:1787826191.201805:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x9b0:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.201809:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x9b0:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.201813:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x9b0:0x0](ffff9181c427a1c8) 00000080:00200000:1.0:1787826191.201825:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x9b1:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.201828:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x9b1:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.201831:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x9b1:0x0](ffff9181c42780c8) 00000080:00200000:1.0:1787826191.201842:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x9b2:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00010000:00010000:3.0:1787826191.202039:0:15160:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200000:1.0:1787826191.205839:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x9b2:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.205868:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x9b2:0x0](ffff9181c427ec08) 00000080:00200000:1.0:1787826191.205926:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x9b3:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.205933:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x9b3:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.205938:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x9b3:0x0](ffff9181c427d348) 00000080:00200000:1.0:1787826191.205954:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x9b4:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.205958:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x9b4:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.205962:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x9b4:0x0](ffff9181c427c2c8) 00000080:00200000:1.0:1787826191.205974:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x9b5:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.205978:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x9b5:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.205982:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x9b5:0x0](ffff9181c428b248) 00000080:00200000:1.0:1787826191.205999:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x9b6:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.206003:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x9b6:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.206006:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x9b6:0x0](ffff9181c4278908) 00000080:00200000:1.0:1787826191.206020:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x9b7:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.206024:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x9b7:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.206027:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x9b7:0x0](ffff9181c427cb08) 00000080:00200000:1.0:1787826191.206040:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x9b8:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.206044:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x9b8:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.206048:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x9b8:0x0](ffff9181c428db88) 00000080:00200000:1.0:1787826191.206061:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x9b9:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.206066:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x9b9:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.206070:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x9b9:0x0](ffff9181c6a05b88) 00000080:00200000:1.0:1787826191.206088:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x9ba:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.206092:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x9ba:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.206096:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x9ba:0x0](ffff9181c6a01988) 00000080:00200000:1.0:1787826191.206110:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x9bb:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.206113:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x9bb:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.206116:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x9bb:0x0](ffff9181c6a021c8) 00000080:00200000:1.0:1787826191.206128:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x9bc:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.206131:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x9bc:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.206134:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x9bc:0x0](ffff9181c6a07448) 00000080:00200000:1.0:1787826191.206147:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x9bd:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.206150:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x9bd:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.206153:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x9bd:0x0](ffff9181c6a063c8) 00000080:00200000:1.0:1787826191.206166:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x9be:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.206170:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x9be:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.206174:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x9be:0x0](ffff9181c6a01148) 00000080:00200000:1.0:1787826191.206187:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x9bf:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.206190:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x9bf:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.206194:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x9bf:0x0](ffff9181c6a000c8) 00000080:00200000:1.0:1787826191.206206:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x9c0:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.206210:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x9c0:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.206214:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x9c0:0x0](ffff9181c6a06c08) 00000080:00200000:1.0:1787826191.206226:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x9c1:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.206230:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x9c1:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.206233:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x9c1:0x0](ffff9181c6a05348) 00000080:00200000:1.0:1787826191.206245:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x9c2:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.206249:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x9c2:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.206253:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x9c2:0x0](ffff9181c69ed348) 00000080:00200000:1.0:1787826191.206265:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x9c3:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.206269:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x9c3:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.206272:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x9c3:0x0](ffff9181c69ec2c8) 00000080:00200000:1.0:1787826191.206284:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x9c4:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.206288:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x9c4:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.206291:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x9c4:0x0](ffff9181c427aa08) 00000080:00200000:1.0:1787826191.206305:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x9c5:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.206308:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x9c5:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.206312:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x9c5:0x0](ffff9181c69e1148) 00000080:00200000:1.0:1787826191.206324:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x9c6:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.206327:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x9c6:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.206331:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x9c6:0x0](ffff9181c69e00c8) 00000080:00200000:1.0:1787826191.206342:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x9c7:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.206346:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x9c7:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.206349:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x9c7:0x0](ffff9181c69e6c08) 00000080:00200000:1.0:1787826191.206362:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x9c8:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.206365:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x9c8:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.206369:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x9c8:0x0](ffff9181c69e5348) 00000080:00200000:1.0:1787826191.206381:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x9c9:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.206384:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x9c9:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.206388:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x9c9:0x0](ffff9181c69e42c8) 00000080:00200000:1.0:1787826191.206401:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x9ca:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.206405:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x9ca:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.206408:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x9ca:0x0](ffff9181c69e0908) 00000080:00200000:1.0:1787826191.206419:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x9cb:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1787826191.206423:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x9cb:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.206427:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x9cb:0x0](ffff9181c69e4b08) 00000080:00200000:1.0:1787826191.206439:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e1:0x1:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.206447:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.206473:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e1:0x1:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.206477:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e1:0x1:0x0](ffff9181c69e2a08) 00000080:00200000:1.0:1787826191.206516:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.206521:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.206529:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.206533:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1:0x0](ffff9181c69e3248) 00000080:00200000:1.0:1787826191.206557:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x61:0x0](ffff9181c421cb08) 00000080:00200000:1.0:1787826191.206566:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0xa30:0x0](ffff9181c7af5348) 00000080:00200000:1.0:1787826191.206579:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0xafb:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.206584:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.206592:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0xafb:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.206597:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0xafb:0x0](ffff9181c6a4e3c8) 00000080:00200000:1.0:1787826191.206653:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0xafd:0x0](ffff9181c69942c8) 00000080:00200000:1.0:1787826191.206668:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0xb24:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.206674:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.206683:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0xb24:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.206687:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0xb24:0x0](ffff9181c6bfec08) 00000080:00200000:1.0:1787826191.207955:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0xb62:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.207968:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.207985:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0xb62:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.207991:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0xb62:0x0](ffff9181c6bf9148) 00000080:00200000:1.0:1787826191.208026:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0xb77:0x0](ffff9181c6a7f448) 00000080:00200000:1.0:1787826191.208036:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2400007ed:0x322:0x0](ffff9181c6a7aa08) 00000080:00200000:1.0:1787826191.208046:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2400007ed:0x323:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.208051:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.208058:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2400007ed:0x323:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.208063:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2400007ed:0x323:0x0](ffff9181c6bfc2c8) 00000080:00200000:1.0:1787826191.208087:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2400007ed:0x324:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.208091:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.208099:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2400007ed:0x324:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.208103:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2400007ed:0x324:0x0](ffff9181c6bf8908) 00000080:00200000:1.0:1787826191.208127:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2400007ed:0x325:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.208132:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.208139:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2400007ed:0x325:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.208143:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2400007ed:0x325:0x0](ffff9181c6a48908) 00000080:00200000:1.0:1787826191.208165:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2400007ed:0x326:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.208170:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.208176:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2400007ed:0x326:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.208180:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2400007ed:0x326:0x0](ffff9181c6a7b248) 00000080:00200000:1.0:1787826191.208200:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2400007ed:0x327:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.208204:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.208210:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2400007ed:0x327:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.208213:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2400007ed:0x327:0x0](ffff9181c6a79988) 00000080:00200000:1.0:1787826191.208231:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2400007ed:0x328:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.208235:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.208241:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2400007ed:0x328:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.208244:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2400007ed:0x328:0x0](ffff9181c6a7db88) 00000080:00200000:1.0:1787826191.208261:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2400007ed:0x329:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.208265:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.208271:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2400007ed:0x329:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.208274:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2400007ed:0x329:0x0](ffff9181c6a7ba88) 00000080:00200000:1.0:1787826191.208293:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2400007ed:0x32a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.208297:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.208303:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2400007ed:0x32a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.208307:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2400007ed:0x32a:0x0](ffff9181c6a7e3c8) 00000080:00200000:1.0:1787826191.208328:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2400007ed:0x32b:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.208332:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.208339:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2400007ed:0x32b:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.208342:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2400007ed:0x32b:0x0](ffff9181c6a7c2c8) 00000080:00200000:1.0:1787826191.208364:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2400007ed:0x32c:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.208368:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.208376:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2400007ed:0x32c:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.208380:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2400007ed:0x32c:0x0](ffff9181c6a7d348) 00000080:00200000:1.0:1787826191.208401:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2400007ed:0x32d:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.208406:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.208412:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2400007ed:0x32d:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.208416:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2400007ed:0x32d:0x0](ffff9181c6a4cb08) 00000080:00200000:1.0:1787826191.208435:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2400007ed:0x32e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.208439:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.208444:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2400007ed:0x32e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.208448:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2400007ed:0x32e:0x0](ffff9181c6a4aa08) 00000080:00200000:1.0:1787826191.208470:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2400007ed:0x32f:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.208474:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.208479:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2400007ed:0x32f:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.208484:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2400007ed:0x32f:0x0](ffff9181c6a7cb08) 00000080:00200000:1.0:1787826191.208506:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2400007ed:0x330:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.208510:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.208516:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2400007ed:0x330:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.208521:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2400007ed:0x330:0x0](ffff9181c6a7a1c8) 00000080:00200000:1.0:1787826191.208543:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2400007ed:0x331:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.208547:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.208553:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2400007ed:0x331:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.208557:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2400007ed:0x331:0x0](ffff9181c6a79148) 00000080:00200000:1.0:1787826191.208583:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2400007ed:0x332:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.208587:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.208594:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2400007ed:0x332:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.208598:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2400007ed:0x332:0x0](ffff9181c6a780c8) 00000080:00200000:1.0:1787826191.208622:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x2400007ed:0x333:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.208627:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.208634:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x2400007ed:0x333:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.208638:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2400007ed:0x333:0x0](ffff9181c6a78908) 00000080:00200000:1.0:1787826191.208660:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0xb7a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.208666:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.208673:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0xb7a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.208677:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0xb7a:0x0](ffff9181c7845b88) 00000080:00200000:1.0:1787826191.208714:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0xb80:0x0](ffff9181c6a7ec08) 00000080:00200000:1.0:1787826191.208724:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024620:0x1:0x0](ffff9181c6a4f448) 00000080:00200000:1.0:1787826191.208733:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2400007ee:0x1:0x0](ffff9181c6a4ba88) 00000080:00200000:1.0:1787826191.211487:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0xb81:0x0](ffff9181c6a4db88) 00000080:00200000:1.0:1787826191.211515:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024620:0x4:0x0](ffff9181c693b248) 00000080:00200000:1.0:1787826191.211524:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2400007ee:0x4:0x0](ffff9181c693ba88) 00000080:00200000:1.0:1787826191.211542:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000400:0x4:0x0](ffff9181c6939988) 00000080:00200000:1.0:1787826191.211551:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200000401:0x4:0x0](ffff9181c693a1c8) 00000080:00200000:1.0:1787826191.211560:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024620:0x5:0x0](ffff9181c693e3c8) 00000080:00200000:1.0:1787826191.211567:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2400007ee:0x5:0x0](ffff9181c6939148) 00000080:00200000:1.0:1787826191.211575:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x3:0x0](ffff9181c69363c8) 00000080:00200000:1.0:1787826191.211585:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x3:0x0](ffff9181c6937448) 00000080:00200000:1.0:1787826191.211593:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x3:0x0](ffff9181c7855b88) 00000080:00200000:1.0:1787826191.211604:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x3:0x0](ffff9181c78563c8) 00000080:00200000:1.0:1787826191.211617:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x4:0x0](ffff9181c78500c8) 00000080:00200000:1.0:1787826191.211625:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x4:0x0](ffff9181c78521c8) 00000080:00200000:1.0:1787826191.211634:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x5:0x0](ffff9181c6ae21c8) 00000080:00200000:1.0:1787826191.211643:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x5:0x0](ffff9181c6ae5348) 00000080:00200000:1.0:1787826191.211651:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x6:0x0](ffff9181c788a1c8) 00000080:00200000:1.0:1787826191.211663:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x6:0x0](ffff9181c788f448) 00000080:00200000:1.0:1787826191.211672:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x7:0x0](ffff9181c79a9148) 00000080:00200000:1.0:1787826191.211680:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x7:0x0](ffff9181c79ae3c8) 00000080:00200000:1.0:1787826191.211689:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x4:0x0](ffff9181c69ee3c8) 00000080:00200000:1.0:1787826191.211698:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x4:0x0](ffff9181c69e9148) 00000080:00200000:1.0:1787826191.211706:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x8:0x0](ffff9181c69ecb08) 00000080:00200000:1.0:1787826191.211715:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x8:0x0](ffff9181c69eaa08) 00000080:00200000:1.0:1787826191.211723:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x5:0x0](ffff9181c69eba88) 00000080:00200000:1.0:1787826191.211731:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x5:0x0](ffff9181c69e80c8) 00000080:00200000:1.0:1787826191.211739:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x6:0x0](ffff9181c6acdb88) 00000080:00200000:1.0:1787826191.213412:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x6:0x0](ffff9181c6ac9988) 00000080:00200000:1.0:1787826191.213429:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x7:0x0](ffff9181c6acf448) 00000080:00200000:1.0:1787826191.213437:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x7:0x0](ffff9181c6ace3c8) 00000080:00200000:1.0:1787826191.213448:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x9:0x0](ffff9181c6ac80c8) 00000080:00200000:1.0:1787826191.213456:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x9:0x0](ffff9181c6acec08) 00000080:00200000:1.0:1787826191.213465:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xa:0x0](ffff9181c6acaa08) 00000080:00200000:1.0:1787826191.213474:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xa:0x0](ffff9181c6accb08) 00000080:00200000:1.0:1787826191.213483:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x8:0x0](ffff9181c7a7cb08) 00000080:00200000:1.0:1787826191.213492:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x8:0x0](ffff9181c7a7aa08) 00000080:00200000:1.0:1787826191.213500:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xb:0x0](ffff9181c7846c08) 00000080:00200000:1.0:1787826191.213513:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xb:0x0](ffff9181c7845348) 00000080:00200000:1.0:1787826191.213523:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x9:0x0](ffff9181c693d348) 00000080:00200000:1.0:1787826191.213535:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x9:0x0](ffff9181c69380c8) 00000080:00200000:1.0:1787826191.213546:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0xa:0x0](ffff9181c7a7e3c8) 00000080:00200000:1.0:1787826191.213556:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0xa:0x0](ffff9181c7a79148) 00000080:00200000:1.0:1787826191.213564:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xc:0x0](ffff9181c7a7ec08) 00000080:00200000:1.0:1787826191.213572:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xc:0x0](ffff9181c7a7d348) 00000080:00200000:1.0:1787826191.213580:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xd:0x0](ffff9181c7847448) 00000080:00200000:1.0:1787826191.213589:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xd:0x0](ffff9181c78421c8) 00000080:00200000:1.0:1787826191.213598:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xe:0x0](ffff9181c7aa1988) 00000080:00200000:1.0:1787826191.213606:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xe:0x0](ffff9181c7aa21c8) 00000080:00200000:1.0:1787826191.213615:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xf:0x0](ffff9181c7aa63c8) 00000080:00200000:1.0:1787826191.213623:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xf:0x0](ffff9181c7aa1148) 00000080:00200000:1.0:1787826191.213632:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x10:0x0](ffff9181c41b4b08) 00000080:00200000:1.0:1787826191.213641:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x10:0x0](ffff9181c69e3a88) 00000080:00200000:1.0:1787826191.213651:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0xb:0x0](ffff9181c6a8aa08) 00000080:00200000:1.0:1787826191.213660:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0xb:0x0](ffff9181c6a89988) 00000080:00200000:1.0:1787826191.213667:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x11:0x0](ffff9181c421f448) 00000080:00200000:1.0:1787826191.213676:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x11:0x0](ffff9181c421e3c8) 00000080:00200000:1.0:1787826191.213685:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0xc:0x0](ffff9181c697aa08) 00000080:00200000:1.0:1787826191.213694:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0xc:0x0](ffff9181c697cb08) 00000080:00200000:1.0:1787826191.213716:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0xd:0x0](ffff9181c793ba88) 00000080:00200000:1.0:1787826191.213725:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0xd:0x0](ffff9181c793db88) 00000080:00200000:1.0:1787826191.213734:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x12:0x0](ffff9181c697ba88) 00000080:00200000:1.0:1787826191.213761:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x12:0x0](ffff9181c697db88) 00000080:00200000:1.0:1787826191.213774:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0xe:0x0](ffff9181c697a1c8) 00000080:00200000:1.0:1787826191.213786:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0xe:0x0](ffff9181c697f448) 00000080:00200000:1.0:1787826191.213793:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x13:0x0](ffff9181c6979148) 00000080:00200000:1.0:1787826191.213801:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x13:0x0](ffff9181c69780c8) 00000080:00200000:1.0:1787826191.213810:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x14:0x0](ffff9181c697c2c8) 00000080:00200000:1.0:1787826191.213817:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x14:0x0](ffff9181c4269988) 00000080:00200000:1.0:1787826191.213826:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x15:0x0](ffff9181c6a8d348) 00000080:00200000:1.0:1787826191.213834:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x15:0x0](ffff9181c6a8ec08) 00000080:00200000:1.0:1787826191.213842:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x16:0x0](ffff9181c7af00c8) 00000080:00200000:1.0:1787826191.213861:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x16:0x0](ffff9181c7af1148) 00000080:00200000:1.0:1787826191.213870:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0xf:0x0](ffff9181c793a1c8) 00000080:00200000:1.0:1787826191.213879:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0xf:0x0](ffff9181c793f448) 00000080:00200000:1.0:1787826191.213888:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x17:0x0](ffff9181c7af42c8) 00000080:00200000:1.0:1787826191.213896:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x17:0x0](ffff9181c7af0908) 00000080:00200000:1.0:1787826191.213904:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x18:0x0](ffff9181c7af3a88) 00000080:00200000:1.0:1787826191.213913:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x18:0x0](ffff9181c7af5b88) 00000080:00200000:1.0:1787826191.213923:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x10:0x0](ffff9181c7a8ec08) 00000080:00200000:1.0:1787826191.213931:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x10:0x0](ffff9181c7a880c8) 00000080:00200000:1.0:1787826191.213940:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x11:0x0](ffff9181c42621c8) 00000080:00200000:1.0:1787826191.213948:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x11:0x0](ffff9181c4261988) 00000080:00200000:1.0:1787826191.213956:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x19:0x0](ffff9181c7b25b88) 00000080:00200000:1.0:1787826191.213965:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x19:0x0](ffff9181c7b21988) 00000080:00200000:1.0:1787826191.213975:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x12:0x0](ffff9181c7b27448) 00000080:00200000:1.0:1787826191.213983:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x12:0x0](ffff9181c7b263c8) 00000080:00200000:1.0:1787826191.213990:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x13:0x0](ffff9181c7ae63c8) 00000080:00200000:1.0:1787826191.213997:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x13:0x0](ffff9181c7ae7448) 00000080:00200000:1.0:1787826191.214005:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x14:0x0](ffff9181c7b200c8) 00000080:00200000:1.0:1787826191.214014:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x14:0x0](ffff9181c7b26c08) 00000080:00200000:1.0:1787826191.214022:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x1a:0x0](ffff9181c411e3c8) 00000080:00200000:1.0:1787826191.214031:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x1a:0x0](ffff9181c41180c8) 00000080:00200000:1.0:1787826191.214044:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x15:0x0](ffff9181c4119988) 00000080:00200000:1.0:1787826191.214052:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x15:0x0](ffff9181c411db88) 00000080:00200000:1.0:1787826191.214060:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x1b:0x0](ffff9181c411cb08) 00000080:00200000:1.0:1787826191.214070:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x1b:0x0](ffff9181c411c2c8) 00000080:00200000:1.0:1787826191.214079:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x1c:0x0](ffff9181c7aeec08) 00000080:00200000:1.0:1787826191.214087:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x1c:0x0](ffff9181c7ae80c8) 00000080:00200000:1.0:1787826191.214095:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x16:0x0](ffff9181c7aee3c8) 00000080:00200000:1.0:1787826191.214102:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x16:0x0](ffff9181c7aef448) 00000080:00200000:1.0:1787826191.214109:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x17:0x0](ffff9181c7aeb248) 00000080:00200000:1.0:1787826191.214117:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x17:0x0](ffff9181c4251148) 00000080:00200000:1.0:1787826191.214125:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x1d:0x0](ffff9181c42521c8) 00000080:00200000:1.0:1787826191.214132:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x1d:0x0](ffff9181c4251988) 00000080:00200000:1.0:1787826191.214141:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x1e:0x0](ffff9181c4253a88) 00000080:00200000:1.0:1787826191.214149:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x1e:0x0](ffff9181c4253248) 00000080:00200000:1.0:1787826191.214157:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x1f:0x0](ffff9181c425ec08) 00000080:00200000:1.0:1787826191.214165:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x1f:0x0](ffff9181c42580c8) 00000080:00200000:1.0:1787826191.214173:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x20:0x0](ffff9181c42542c8) 00000080:00200000:1.0:1787826191.214181:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x20:0x0](ffff9181c4255348) 00000080:00200000:1.0:1787826191.214189:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x21:0x0](ffff9181c4263a88) 00000080:00200000:1.0:1787826191.214198:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x21:0x0](ffff9181c6b01988) 00000080:00200000:1.0:1787826191.214205:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x22:0x0](ffff9181c6b0a1c8) 00000080:00200000:1.0:1787826191.214212:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x22:0x0](ffff9181c6b09988) 00000080:00200000:1.0:1787826191.214219:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x18:0x0](ffff9181c6b0cb08) 00000080:00200000:1.0:1787826191.214227:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x18:0x0](ffff9181c6b08908) 00000080:00200000:1.0:1787826191.214234:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x19:0x0](ffff9181c425f448) 00000080:00200000:1.0:1787826191.214240:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x19:0x0](ffff9181c425a1c8) 00000080:00200000:1.0:1787826191.214247:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x23:0x0](ffff9181c6b080c8) 00000080:00200000:1.0:1787826191.214256:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x23:0x0](ffff9181c6b09148) 00000080:00200000:1.0:1787826191.214263:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x24:0x0](ffff9181c6b0f448) 00000080:00200000:1.0:1787826191.214270:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x24:0x0](ffff9181c6ba2a08) 00000080:00200000:1.0:1787826191.214278:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x1a:0x0](ffff9181c6ba0908) 00000080:00200000:1.0:1787826191.214285:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x1a:0x0](ffff9181c6ba42c8) 00000080:00200000:1.0:1787826191.214292:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x1b:0x0](ffff9181c6ba6c08) 00000080:00200000:1.0:1787826191.214300:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x1b:0x0](ffff9181c6ba00c8) 00000080:00200000:1.0:1787826191.214308:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x1c:0x0](ffff9181c6ba63c8) 00000080:00200000:1.0:1787826191.214315:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x1c:0x0](ffff9181c6ba7448) 00000080:00200000:1.0:1787826191.214321:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x25:0x0](ffff9181c6ba1988) 00000080:00200000:1.0:1787826191.214329:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x25:0x0](ffff9181c6ba5b88) 00000080:00200000:1.0:1787826191.214336:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x1d:0x0](ffff9181c6ba3248) 00000080:00200000:1.0:1787826191.214344:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x1d:0x0](ffff9181c6bad348) 00000080:00200000:1.0:1787826191.214351:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x26:0x0](ffff9181c6ae8908) 00000080:00200000:1.0:1787826191.214359:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x26:0x0](ffff9181c6aeaa08) 00000080:00200000:1.0:1787826191.214368:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x27:0x0](ffff9181c6b042c8) 00000080:00200000:1.0:1787826191.214376:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x27:0x0](ffff9181c6b05348) 00000080:00200000:1.0:1787826191.214384:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x1e:0x0](ffff9181c6b000c8) 00000080:00200000:1.0:1787826191.214392:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x1e:0x0](ffff9181c6b01148) 00000080:00200000:1.0:1787826191.214400:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x28:0x0](ffff9181c425b248) 00000080:00200000:1.0:1787826191.214407:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x28:0x0](ffff9181c425aa08) 00000080:00200000:1.0:1787826191.214413:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x1f:0x0](ffff9181c7a02a08) 00000080:00200000:1.0:1787826191.214421:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x1f:0x0](ffff9181c7a04b08) 00000080:00200000:1.0:1787826191.214430:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x20:0x0](ffff9181c7a0e3c8) 00000080:00200000:1.0:1787826191.214436:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x20:0x0](ffff9181c7a0f448) 00000080:00200000:1.0:1787826191.214443:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x21:0x0](ffff9181c4258908) 00000080:00200000:1.0:1787826191.214450:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x21:0x0](ffff9181c425c2c8) 00000080:00200000:1.0:1787826191.214457:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x22:0x0](ffff9181c7a09988) 00000080:00200000:1.0:1787826191.214463:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x22:0x0](ffff9181c7a0db88) 00000080:00200000:1.0:1787826191.214470:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x23:0x0](ffff9181c6aee3c8) 00000080:00200000:1.0:1787826191.214478:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x23:0x0](ffff9181c6ae9148) 00000080:00200000:1.0:1787826191.214486:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x29:0x0](ffff9181c7a900c8) 00000080:00200000:1.0:1787826191.214492:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x29:0x0](ffff9181c7a91148) 00000080:00200000:1.0:1787826191.214499:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x2a:0x0](ffff9181c6b021c8) 00000080:00200000:1.0:1787826191.214506:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x2a:0x0](ffff9181c7a100c8) 00000080:00200000:1.0:1787826191.214515:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x2b:0x0](ffff9181c7a163c8) 00000080:00200000:1.0:1787826191.214523:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x2b:0x0](ffff9181c7a17448) 00000080:00200000:1.0:1787826191.214530:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x2c:0x0](ffff9181c7a11988) 00000080:00200000:1.0:1787826191.214538:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x2c:0x0](ffff9181c7a15b88) 00000080:00200000:1.0:1787826191.214546:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x24:0x0](ffff9181c7a0aa08) 00000080:00200000:1.0:1787826191.214554:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x24:0x0](ffff9181c7a0cb08) 00000080:00200000:1.0:1787826191.214563:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x25:0x0](ffff9181c7a97448) 00000080:00200000:1.0:1787826191.214575:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x25:0x0](ffff9181c7a921c8) 00000080:00200000:1.0:1787826191.214583:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x26:0x0](ffff9181c7a95b88) 00000080:00200000:1.0:1787826191.214595:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x26:0x0](ffff9181c7a93a88) 00000080:00200000:1.0:1787826191.214603:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x2d:0x0](ffff9181c7a13248) 00000080:00200000:1.0:1787826191.214610:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x2d:0x0](ffff9181c7a12a08) 00000080:00200000:1.0:1787826191.214617:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x27:0x0](ffff9181c7a1ba88) 00000080:00200000:1.0:1787826191.214625:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x27:0x0](ffff9181c7a1b248) 00000080:00200000:1.0:1787826191.214633:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x28:0x0](ffff9181c7a1cb08) 00000080:00200000:1.0:1787826191.214641:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x28:0x0](ffff9181c7a18908) 00000080:00200000:1.0:1787826191.214649:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x29:0x0](ffff9181c6bcba88) 00000080:00200000:1.0:1787826191.214658:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x29:0x0](ffff9181c6bcb248) 00000080:00200000:1.0:1787826191.214666:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x2a:0x0](ffff9181c6bcc2c8) 00000080:00200000:1.0:1787826191.214674:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x2a:0x0](ffff9181c6bcd348) 00000080:00200000:1.0:1787826191.214682:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x2b:0x0](ffff9181c6bc80c8) 00000080:00200000:1.0:1787826191.214690:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x2b:0x0](ffff9181c6bc9148) 00000080:00200000:1.0:1787826191.214696:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x2c:0x0](ffff9181c6bd00c8) 00000080:00200000:1.0:1787826191.214704:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x2c:0x0](ffff9181c6bd1148) 00000080:00200000:1.0:1787826191.214711:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x2d:0x0](ffff9181c6bd7448) 00000080:00200000:1.0:1787826191.214719:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x2d:0x0](ffff9181c6bd21c8) 00000080:00200000:1.0:1787826191.214728:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x2e:0x0](ffff9181c6bd3248) 00000080:00200000:1.0:1787826191.214737:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x2e:0x0](ffff9181c6bd2a08) 00000080:00200000:1.0:1787826191.214779:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x2e:0x0](ffff9181c6bd5348) 00000080:00200000:1.0:1787826191.214788:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x2e:0x0](ffff9181c6bdba88) 00000080:00200000:1.0:1787826191.214797:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x2f:0x0](ffff9181c6bc0908) 00000080:00200000:1.0:1787826191.214806:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x2f:0x0](ffff9181c6bc42c8) 00000080:00200000:1.0:1787826191.214814:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x30:0x0](ffff9181c6bc6c08) 00000080:00200000:1.0:1787826191.214821:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x30:0x0](ffff9181c6bc00c8) 00000080:00200000:1.0:1787826191.214828:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x2f:0x0](ffff9181c7988908) 00000080:00200000:1.0:1787826191.214835:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x2f:0x0](ffff9181c798c2c8) 00000080:00200000:1.0:1787826191.214842:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x31:0x0](ffff9181c6bc21c8) 00000080:00200000:1.0:1787826191.215833:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x31:0x0](ffff9181c6bc1988) 00000080:00200000:1.0:1787826191.215863:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x32:0x0](ffff9181c7998908) 00000080:00200000:1.0:1787826191.215874:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x32:0x0](ffff9181c799c2c8) 00000080:00200000:1.0:1787826191.215882:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x30:0x0](ffff9181c79963c8) 00000080:00200000:1.0:1787826191.215889:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x30:0x0](ffff9181c7997448) 00000080:00200000:1.0:1787826191.215897:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x33:0x0](ffff9181c7983248) 00000080:00200000:1.0:1787826191.215906:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x33:0x0](ffff9181c7982a08) 00000080:00200000:1.0:1787826191.215916:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x31:0x0](ffff9181c7995b88) 00000080:00200000:1.0:1787826191.215923:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x31:0x0](ffff9181c7993a88) 00000080:00200000:1.0:1787826191.215931:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x32:0x0](ffff9181c78642c8) 00000080:00200000:1.0:1787826191.215941:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x32:0x0](ffff9181c7865348) 00000080:00200000:1.0:1787826191.215951:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x33:0x0](ffff9181c78600c8) 00000080:00200000:1.0:1787826191.215959:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x33:0x0](ffff9181c7861148) 00000080:00200000:1.0:1787826191.215967:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x34:0x0](ffff9181c7981148) 00000080:00200000:1.0:1787826191.215975:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x34:0x0](ffff9181c79863c8) 00000080:00200000:1.0:1787826191.215983:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x34:0x0](ffff9181c786d348) 00000080:00200000:1.0:1787826191.215994:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x34:0x0](ffff9181c786ec08) 00000080:00200000:1.0:1787826191.216002:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x35:0x0](ffff9181c7867448) 00000080:00200000:1.0:1787826191.216010:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x35:0x0](ffff9181c78621c8) 00000080:00200000:1.0:1787826191.216019:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x36:0x0](ffff9181c7999148) 00000080:00200000:1.0:1787826191.216027:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x36:0x0](ffff9181c799e3c8) 00000080:00200000:1.0:1787826191.216034:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x37:0x0](ffff9181c799f448) 00000080:00200000:1.0:1787826191.216042:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x37:0x0](ffff9181c799a1c8) 00000080:00200000:1.0:1787826191.216049:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x35:0x0](ffff9181c799db88) 00000080:00200000:1.0:1787826191.216057:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x35:0x0](ffff9181c799ba88) 00000080:00200000:1.0:1787826191.216065:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x36:0x0](ffff9181c7875b88) 00000080:00200000:1.0:1787826191.216072:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x36:0x0](ffff9181c7873a88) 00000080:00200000:1.0:1787826191.216079:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x37:0x0](ffff9181c7872a08) 00000080:00200000:1.0:1787826191.216087:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x37:0x0](ffff9181c7874b08) 00000080:00200000:1.0:1787826191.216096:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x38:0x0](ffff9181c78742c8) 00000080:00200000:1.0:1787826191.216103:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x38:0x0](ffff9181c7875348) 00000080:00200000:1.0:1787826191.216111:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x39:0x0](ffff9181c7871148) 00000080:00200000:1.0:1787826191.216118:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x39:0x0](ffff9181c7879988) 00000080:00200000:1.0:1787826191.216127:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x3a:0x0](ffff9181c7825b88) 00000080:00200000:1.0:1787826191.216136:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x3a:0x0](ffff9181c7821988) 00000080:00200000:1.0:1787826191.216158:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x3b:0x0](ffff9181c787f448) 00000080:00200000:1.0:1787826191.216167:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x3b:0x0](ffff9181c787a1c8) 00000080:00200000:1.0:1787826191.216174:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x3c:0x0](ffff9181c780d348) 00000080:00200000:1.0:1787826191.216182:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x3c:0x0](ffff9181c780ec08) 00000080:00200000:1.0:1787826191.216190:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x38:0x0](ffff9181c780e3c8) 00000080:00200000:1.0:1787826191.216198:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x38:0x0](ffff9181c780f448) 00000080:00200000:1.0:1787826191.216206:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x39:0x0](ffff9181c78263c8) 00000080:00200000:1.0:1787826191.216213:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x39:0x0](ffff9181c7821148) 00000080:00200000:1.0:1787826191.216220:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x3a:0x0](ffff9181c7801148) 00000080:00200000:1.0:1787826191.216227:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x3a:0x0](ffff9181c78063c8) 00000080:00200000:1.0:1787826191.216235:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x3b:0x0](ffff9181c780db88) 00000080:00200000:1.0:1787826191.216242:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x3b:0x0](ffff9181c780ba88) 00000080:00200000:1.0:1787826191.216250:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x3c:0x0](ffff9181c780aa08) 00000080:00200000:1.0:1787826191.216258:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x3c:0x0](ffff9181c780cb08) 00000080:00200000:1.0:1787826191.216266:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x3d:0x0](ffff9181c78100c8) 00000080:00200000:1.0:1787826191.216273:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x3d:0x0](ffff9181c7811148) 00000080:00200000:1.0:1787826191.216281:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x3e:0x0](ffff9181c7817448) 00000080:00200000:1.0:1787826191.216289:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x3e:0x0](ffff9181c78121c8) 00000080:00200000:1.0:1787826191.216297:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x3f:0x0](ffff9181c78021c8) 00000080:00200000:1.0:1787826191.216304:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x3f:0x0](ffff9181c7801988) 00000080:00200000:1.0:1787826191.216312:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x40:0x0](ffff9181c7803a88) 00000080:00200000:1.0:1787826191.216320:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x40:0x0](ffff9181c7803248) 00000080:00200000:1.0:1787826191.216327:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x41:0x0](ffff9181c7815b88) 00000080:00200000:1.0:1787826191.216336:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x41:0x0](ffff9181c7813a88) 00000080:00200000:1.0:1787826191.216343:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x42:0x0](ffff9181c7812a08) 00000080:00200000:1.0:1787826191.216351:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x42:0x0](ffff9181c7814b08) 00000080:00200000:1.0:1787826191.216359:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x43:0x0](ffff9181c7804b08) 00000080:00200000:1.0:1787826191.216367:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x43:0x0](ffff9181c7800908) 00000080:00200000:1.0:1787826191.216375:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x3d:0x0](ffff9181c6b863c8) 00000080:00200000:1.0:1787826191.216382:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x3d:0x0](ffff9181c6b87448) 00000080:00200000:1.0:1787826191.216389:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x3e:0x0](ffff9181c6b8cb08) 00000080:00200000:1.0:1787826191.216397:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x3e:0x0](ffff9181c6b88908) 00000080:00200000:1.0:1787826191.216405:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x44:0x0](ffff9181c6b89148) 00000080:00200000:1.0:1787826191.216413:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x44:0x0](ffff9181c6b8e3c8) 00000080:00200000:1.0:1787826191.216421:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x45:0x0](ffff9181c6b921c8) 00000080:00200000:1.0:1787826191.216429:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x45:0x0](ffff9181c6b91988) 00000080:00200000:1.0:1787826191.216436:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x46:0x0](ffff9181c6b81988) 00000080:00200000:1.0:1787826191.216444:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x46:0x0](ffff9181c6b85b88) 00000080:00200000:1.0:1787826191.216452:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x3f:0x0](ffff9181c786aa08) 00000080:00200000:1.0:1787826191.216461:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x3f:0x0](ffff9181c786cb08) 00000080:00200000:1.0:1787826191.216470:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x47:0x0](ffff9181c6b9d348) 00000080:00200000:1.0:1787826191.216479:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x47:0x0](ffff9181c6b9ec08) 00000080:00200000:1.0:1787826191.216486:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x40:0x0](ffff9181c6b97448) 00000080:00200000:1.0:1787826191.216494:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x40:0x0](ffff9181c69c5348) 00000080:00200000:1.0:1787826191.216500:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x41:0x0](ffff9181c69c00c8) 00000080:00200000:1.0:1787826191.216507:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x41:0x0](ffff9181c69c1148) 00000080:00200000:1.0:1787826191.216515:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x48:0x0](ffff9181c69c7448) 00000080:00200000:1.0:1787826191.216523:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x48:0x0](ffff9181c69c21c8) 00000080:00200000:1.0:1787826191.216530:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x49:0x0](ffff9181c6b9e3c8) 00000080:00200000:1.0:1787826191.216541:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x49:0x0](ffff9181c6b9f448) 00000080:00200000:1.0:1787826191.216548:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x4a:0x0](ffff9181c69c3a88) 00000080:00200000:1.0:1787826191.216554:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x4a:0x0](ffff9181c69c3248) 00000080:00200000:1.0:1787826191.216562:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x42:0x0](ffff9181c69c0908) 00000080:00200000:1.0:1787826191.216569:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x42:0x0](ffff9181c69c42c8) 00000080:00200000:1.0:1787826191.216577:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x43:0x0](ffff9181c69d2a08) 00000080:00200000:1.0:1787826191.216586:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x43:0x0](ffff9181c69d4b08) 00000080:00200000:1.0:1787826191.216594:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x4b:0x0](ffff9181c69ccb08) 00000080:00200000:1.0:1787826191.216602:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x4b:0x0](ffff9181c69c8908) 00000080:00200000:1.0:1787826191.216611:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x44:0x0](ffff9181c69db248) 00000080:00200000:1.0:1787826191.216619:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x44:0x0](ffff9181c69daa08) 00000080:00200000:1.0:1787826191.216626:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x4c:0x0](ffff9181c69d8908) 00000080:00200000:1.0:1787826191.216635:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x4c:0x0](ffff9181c69dc2c8) 00000080:00200000:1.0:1787826191.216643:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x4d:0x0](ffff9181c69dec08) 00000080:00200000:1.0:1787826191.216650:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x4d:0x0](ffff9181c69d80c8) 00000080:00200000:1.0:1787826191.216658:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x4e:0x0](ffff9181c69de3c8) 00000080:00200000:1.0:1787826191.216666:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x4e:0x0](ffff9181c69df448) 00000080:00200000:1.0:1787826191.216673:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x4f:0x0](ffff9181c69d9988) 00000080:00200000:1.0:1787826191.216681:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x4f:0x0](ffff9181c69ddb88) 00000080:00200000:1.0:1787826191.216688:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x50:0x0](ffff9181c6a22a08) 00000080:00200000:1.0:1787826191.216696:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x50:0x0](ffff9181c6a24b08) 00000080:00200000:1.0:1787826191.216717:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x51:0x0](ffff9181c6a242c8) 00000080:00200000:1.0:1787826191.216726:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x51:0x0](ffff9181c6a25348) 00000080:00200000:1.0:1787826191.216733:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x52:0x0](ffff9181c6a21148) 00000080:00200000:1.0:1787826191.216740:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x52:0x0](ffff9181c6a263c8) 00000080:00200000:1.0:1787826191.217945:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x53:0x0](ffff9181c6a221c8) 00000080:00200000:1.0:1787826191.217956:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x53:0x0](ffff9181c6a21988) 00000080:00200000:1.0:1787826191.217964:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x45:0x0](ffff9181c69d5348) 00000080:00200000:1.0:1787826191.217973:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x45:0x0](ffff9181c69d6c08) 00000080:00200000:1.0:1787826191.217982:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x46:0x0](ffff9181c69d1148) 00000080:00200000:1.0:1787826191.217992:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x46:0x0](ffff9181c69d63c8) 00000080:00200000:1.0:1787826191.218000:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x47:0x0](ffff9181c6a2a1c8) 00000080:00200000:1.0:1787826191.218008:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x47:0x0](ffff9181c6a29988) 00000080:00200000:1.0:1787826191.218016:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x54:0x0](ffff9181c6a2ba88) 00000080:00200000:1.0:1787826191.218024:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x54:0x0](ffff9181c6a2b248) 00000080:00200000:1.0:1787826191.218031:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x55:0x0](ffff9181c6a342c8) 00000080:00200000:1.0:1787826191.218039:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x55:0x0](ffff9181c6a35348) 00000080:00200000:1.0:1787826191.218048:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x56:0x0](ffff9181c6a36c08) 00000080:00200000:1.0:1787826191.218056:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x56:0x0](ffff9181c6a300c8) 00000080:00200000:1.0:1787826191.218064:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x57:0x0](ffff9181c6a363c8) 00000080:00200000:1.0:1787826191.218073:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x57:0x0](ffff9181c6a37448) 00000080:00200000:1.0:1787826191.218081:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x48:0x0](ffff9181c69d5b88) 00000080:00200000:1.0:1787826191.218090:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x48:0x0](ffff9181c6a39988) 00000080:00200000:1.0:1787826191.218099:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x49:0x0](ffff9181c6b23248) 00000080:00200000:1.0:1787826191.218107:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x49:0x0](ffff9181c6b22a08) 00000080:00200000:1.0:1787826191.218116:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x58:0x0](ffff9181c6b242c8) 00000080:00200000:1.0:1787826191.218124:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x58:0x0](ffff9181c6b25348) 00000080:00200000:1.0:1787826191.218132:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x4a:0x0](ffff9181c6b263c8) 00000080:00200000:1.0:1787826191.218140:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x4a:0x0](ffff9181c6b27448) 00000080:00200000:1.0:1787826191.218951:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x4b:0x0](ffff9181c6b2aa08) 00000080:00200000:1.0:1787826191.218961:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x4b:0x0](ffff9181c6b2cb08) 00000080:00200000:1.0:1787826191.218972:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x59:0x0](ffff9181c6b2c2c8) 00000080:00200000:1.0:1787826191.218981:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x59:0x0](ffff9181c6b2d348) 00000080:00200000:1.0:1787826191.218990:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x5a:0x0](ffff9181c6b29148) 00000080:00200000:1.0:1787826191.218998:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x5a:0x0](ffff9181c6b2e3c8) 00000080:00200000:1.0:1787826191.219009:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x4c:0x0](ffff9181c6b2db88) 00000080:00200000:1.0:1787826191.219018:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x4c:0x0](ffff9181c6b2ba88) 00000080:00200000:1.0:1787826191.219026:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x4d:0x0](ffff9181c6b33a88) 00000080:00200000:1.0:1787826191.219034:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x4d:0x0](ffff9181c6b33248) 00000080:00200000:1.0:1787826191.219042:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x5b:0x0](ffff9181c6b32a08) 00000080:00200000:1.0:1787826191.219051:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x5b:0x0](ffff9181c6b34b08) 00000080:00200000:1.0:1787826191.219060:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x5c:0x0](ffff9181c6b321c8) 00000080:00200000:1.0:1787826191.219069:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x5c:0x0](ffff9181c6b31988) 00000080:00200000:1.0:1787826191.219077:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x5d:0x0](ffff9181c6b39148) 00000080:00200000:1.0:1787826191.219086:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x5d:0x0](ffff9181c6b3e3c8) 00000080:00200000:1.0:1787826191.219093:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x5e:0x0](ffff9181c6b3a1c8) 00000080:00200000:1.0:1787826191.219101:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x5e:0x0](ffff9181c6b39988) 00000080:00200000:1.0:1787826191.219108:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x5f:0x0](ffff9181c6b3b248) 00000080:00200000:1.0:1787826191.219116:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x5f:0x0](ffff9181c6b3aa08) 00000080:00200000:1.0:1787826191.219124:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x4e:0x0](ffff9181c6b3c2c8) 00000080:00200000:1.0:1787826191.219132:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x4e:0x0](ffff9181c6b3d348) 00000080:00200000:1.0:1787826191.219140:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x4f:0x0](ffff9181c6944b08) 00000080:00200000:1.0:1787826191.219954:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x4f:0x0](ffff9181c6940908) 00000080:00200000:1.0:1787826191.219965:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x50:0x0](ffff9181c6949148) 00000080:00200000:1.0:1787826191.219978:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x50:0x0](ffff9181c694e3c8) 00000080:00200000:1.0:1787826191.219987:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x51:0x0](ffff9181c6a3d348) 00000080:00200000:1.0:1787826191.219997:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x51:0x0](ffff9181c6a3ec08) 00000080:00200000:1.0:1787826191.220005:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x52:0x0](ffff9181c69463c8) 00000080:00200000:1.0:1787826191.220014:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x52:0x0](ffff9181c6947448) 00000080:00200000:1.0:1787826191.220022:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x53:0x0](ffff9181c6bbd348) 00000080:00200000:1.0:1787826191.220031:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x53:0x0](ffff9181c6bbc2c8) 00000080:00200000:1.0:1787826191.220039:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x60:0x0](ffff9181c6a3e3c8) 00000080:00200000:1.0:1787826191.220048:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x60:0x0](ffff9181c6a3f448) 00000080:00200000:1.0:1787826191.220056:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x61:0x0](ffff9181c6943248) 00000080:00200000:1.0:1787826191.220065:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x61:0x0](ffff9181c6942a08) 00000080:00200000:1.0:1787826191.220073:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x54:0x0](ffff9181c6957448) 00000080:00200000:1.0:1787826191.220081:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x54:0x0](ffff9181c69521c8) 00000080:00200000:1.0:1787826191.220089:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x62:0x0](ffff9181c6bbcb08) 00000080:00200000:1.0:1787826191.220098:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x62:0x0](ffff9181c6bbaa08) 00000080:00200000:1.0:1787826191.220106:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x63:0x0](ffff9181c790e3c8) 00000080:00200000:1.0:1787826191.220114:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x63:0x0](ffff9181c790f448) 00000080:00200000:1.0:1787826191.220121:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x55:0x0](ffff9181c7909988) 00000080:00200000:1.0:1787826191.220131:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x55:0x0](ffff9181c790db88) 00000080:00200000:1.0:1787826191.220139:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x64:0x0](ffff9181c79042c8) 00000080:00200000:1.0:1787826191.220981:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x64:0x0](ffff9181c7905348) 00000080:00200000:1.0:1787826191.220995:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x65:0x0](ffff9181c791b248) 00000080:00200000:1.0:1787826191.221003:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x65:0x0](ffff9181c791aa08) 00000080:00200000:1.0:1787826191.221012:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x56:0x0](ffff9181c7910908) 00000080:00200000:1.0:1787826191.221023:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x56:0x0](ffff9181c79142c8) 00000080:00200000:1.0:1787826191.221031:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x66:0x0](ffff9181c79c4b08) 00000080:00200000:1.0:1787826191.221040:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x66:0x0](ffff9181c79c0908) 00000080:00200000:1.0:1787826191.221049:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x57:0x0](ffff9181c79c6c08) 00000080:00200000:1.0:1787826191.221060:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x57:0x0](ffff9181c79c00c8) 00000080:00200000:1.0:1787826191.221068:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x58:0x0](ffff9181c79c63c8) 00000080:00200000:1.0:1787826191.221075:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x58:0x0](ffff9181c79c7448) 00000080:00200000:1.0:1787826191.221083:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x59:0x0](ffff9181c6bbdb88) 00000080:00200000:1.0:1787826191.221092:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x59:0x0](ffff9181c6bb9988) 00000080:00200000:1.0:1787826191.221101:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x5a:0x0](ffff9181c6bbf448) 00000080:00200000:1.0:1787826191.221108:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x5a:0x0](ffff9181c6bbe3c8) 00000080:00200000:1.0:1787826191.221116:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x5b:0x0](ffff9181c7918908) 00000080:00200000:1.0:1787826191.221124:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x5b:0x0](ffff9181c791c2c8) 00000080:00200000:1.0:1787826191.221131:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x5c:0x0](ffff9181c79c3248) 00000080:00200000:1.0:1787826191.221139:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x5c:0x0](ffff9181c79c2a08) 00000080:00200000:1.0:1787826191.221426:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb8:0x67:0x0](ffff9181c79cec08) 00000080:00200000:1.0:1787826191.221435:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df1:0x67:0x0](ffff9181c79c80c8) 00000080:00200000:1.0:1787826191.221443:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2400007ed:0x857:0x0](ffff9181c7908908) 00000080:00200000:1.0:1787826191.221454:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x5d:0x0](ffff9181c790a1c8) 00000080:00200000:1.0:1787826191.221462:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x5d:0x0](ffff9181c79080c8) 00000080:00200000:1.0:1787826191.221469:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x3:0x0](ffff9181c7a780c8) 00000080:00200000:1.0:1787826191.221479:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x3:0x0](ffff9181c7a78908) 00000080:00200000:1.0:1787826191.221486:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x5e:0x0](ffff9181c7a79988) 00000080:00200000:1.0:1787826191.221495:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x5e:0x0](ffff9181c7a7db88) 00000080:00200000:1.0:1787826191.221502:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x4:0x0](ffff9181c7a7b248) 00000080:00200000:1.0:1787826191.221511:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x4:0x0](ffff9181c7a7ba88) 00000080:00200000:1.0:1787826191.221518:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x5f:0x0](ffff9181c788b248) 00000080:00200000:1.0:1787826191.221528:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x5f:0x0](ffff9181c788ec08) 00000080:00200000:1.0:1787826191.221538:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x5:0x0](ffff9181c6b93248) 00000080:00200000:1.0:1787826191.221546:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x5:0x0](ffff9181c6b93a88) 00000080:00200000:1.0:1787826191.221554:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x60:0x0](ffff9181c790aa08) 00000080:00200000:1.0:1787826191.221562:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x60:0x0](ffff9181c790c2c8) 00000080:00200000:1.0:1787826191.221571:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x6:0x0](ffff9181c7a7a1c8) 00000080:00200000:1.0:1787826191.221579:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x6:0x0](ffff9181c7a7c2c8) 00000080:00200000:1.0:1787826191.221589:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x61:0x0](ffff9181c6b94b08) 00000080:00200000:1.0:1787826191.221597:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x61:0x0](ffff9181c6b90908) 00000080:00200000:1.0:1787826191.221605:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x7:0x0](ffff9181c7a85348) 00000080:00200000:1.0:1787826191.221612:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x7:0x0](ffff9181c7a842c8) 00000080:00200000:1.0:1787826191.221621:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x62:0x0](ffff9181c7909148) 00000080:00200000:1.0:1787826191.221629:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x62:0x0](ffff9181c790ba88) 00000080:00200000:1.0:1787826191.221637:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x8:0x0](ffff9181c6b95348) 00000080:00200000:1.0:1787826191.221644:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x8:0x0](ffff9181c6b91148) 00000080:00200000:1.0:1787826191.221652:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x63:0x0](ffff9181c790ec08) 00000080:00200000:1.0:1787826191.221659:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x63:0x0](ffff9181c790d348) 00000080:00200000:1.0:1787826191.221667:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x9:0x0](ffff9181c7a83a88) 00000080:00200000:1.0:1787826191.221674:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x9:0x0](ffff9181c7a81988) 00000080:00200000:1.0:1787826191.221682:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x64:0x0](ffff9181c6b942c8) 00000080:00200000:1.0:1787826191.221690:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x64:0x0](ffff9181c6b963c8) 00000080:00200000:1.0:1787826191.221697:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0xa:0x0](ffff9181c788ba88) 00000080:00200000:1.0:1787826191.221719:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0xa:0x0](ffff9181c788cb08) 00000080:00200000:1.0:1787826191.221727:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x65:0x0](ffff9181c7889148) 00000080:00200000:1.0:1787826191.221734:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x65:0x0](ffff9181c788e3c8) 00000080:00200000:1.0:1787826191.221881:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0xb:0x0](ffff9181c7a82a08) 00000080:00200000:1.0:1787826191.221896:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0xb:0x0](ffff9181c7a84b08) 00000080:00200000:1.0:1787826191.221905:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x66:0x0](ffff9181c7a83248) 00000080:00200000:1.0:1787826191.221912:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x66:0x0](ffff9181c7a86c08) 00000080:00200000:1.0:1787826191.221921:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0xc:0x0](ffff9181c7a81148) 00000080:00200000:1.0:1787826191.221929:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0xc:0x0](ffff9181c7a85b88) 00000080:00200000:1.0:1787826191.221938:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x67:0x0](ffff9181c6b96c08) 00000080:00200000:1.0:1787826191.221946:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x67:0x0](ffff9181c6b92a08) 00000080:00200000:1.0:1787826191.221953:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0xd:0x0](ffff9181c7a863c8) 00000080:00200000:1.0:1787826191.221961:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0xd:0x0](ffff9181c7a821c8) 00000080:00200000:1.0:1787826191.221969:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x68:0x0](ffff9181c790b248) 00000080:00200000:1.0:1787826191.221977:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x68:0x0](ffff9181c790cb08) 00000080:00200000:1.0:1787826191.221985:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0xe:0x0](ffff9181c6ba1148) 00000080:00200000:1.0:1787826191.221993:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0xe:0x0](ffff9181c6ba3a88) 00000080:00200000:1.0:1787826191.222000:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x69:0x0](ffff9181c7a80908) 00000080:00200000:1.0:1787826191.222008:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x69:0x0](ffff9181c7a7f448) 00000080:00200000:1.0:1787826191.222018:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0xf:0x0](ffff9181c7a96c08) 00000080:00200000:1.0:1787826191.222026:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0xf:0x0](ffff9181c7a95348) 00000080:00200000:1.0:1787826191.222035:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x6a:0x0](ffff9181c6ba21c8) 00000080:00200000:1.0:1787826191.222043:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x6a:0x0](ffff9181c6ba5348) 00000080:00200000:1.0:1787826191.222050:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x10:0x0](ffff9181c6ba4b08) 00000080:00200000:1.0:1787826191.222057:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x10:0x0](ffff9181c7a800c8) 00000080:00200000:1.0:1787826191.222066:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x6b:0x0](ffff9181c788aa08) 00000080:00200000:1.0:1787826191.222074:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x6b:0x0](ffff9181c788d348) 00000080:00200000:1.0:1787826191.222082:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x11:0x0](ffff9181c788c2c8) 00000080:00200000:1.0:1787826191.222090:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x11:0x0](ffff9181c7888908) 00000080:00200000:1.0:1787826191.222098:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x6c:0x0](ffff9181c7a91988) 00000080:00200000:1.0:1787826191.222105:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x6c:0x0](ffff9181c7a92a08) 00000080:00200000:1.0:1787826191.222113:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x12:0x0](ffff9181c78880c8) 00000080:00200000:1.0:1787826191.222120:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x12:0x0](ffff9181c788db88) 00000080:00200000:1.0:1787826191.222129:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x6d:0x0](ffff9181c69542c8) 00000080:00200000:1.0:1787826191.222138:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x6d:0x0](ffff9181c6956c08) 00000080:00200000:1.0:1787826191.223008:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x13:0x0](ffff9181c7a942c8) 00000080:00200000:1.0:1787826191.223025:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x13:0x0](ffff9181c7a93248) 00000080:00200000:1.0:1787826191.223035:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x6e:0x0](ffff9181c6953a88) 00000080:00200000:1.0:1787826191.223043:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x6e:0x0](ffff9181c69500c8) 00000080:00200000:1.0:1787826191.223052:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x14:0x0](ffff9181c6b900c8) 00000080:00200000:1.0:1787826191.223060:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x14:0x0](ffff9181c6b95b88) 00000080:00200000:1.0:1787826191.223069:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x6f:0x0](ffff9181c7a90908) 00000080:00200000:1.0:1787826191.223078:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x6f:0x0](ffff9181c7a94b08) 00000080:00200000:1.0:1787826191.223086:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x15:0x0](ffff9181c7a87448) 00000080:00200000:1.0:1787826191.223095:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x15:0x0](ffff9181c425cb08) 00000080:00200000:1.0:1787826191.223105:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x70:0x0](ffff9181c6954b08) 00000080:00200000:1.0:1787826191.223112:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x70:0x0](ffff9181c6952a08) 00000080:00200000:1.0:1787826191.223119:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x16:0x0](ffff9181c4259148) 00000080:00200000:1.0:1787826191.223127:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x16:0x0](ffff9181c425ba88) 00000080:00200000:1.0:1787826191.223135:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x71:0x0](ffff9181c425d348) 00000080:00200000:1.0:1787826191.223362:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x71:0x0](ffff9181c425db88) 00000080:00200000:1.0:1787826191.223375:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x17:0x0](ffff9181c6953248) 00000080:00200000:1.0:1787826191.223383:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x17:0x0](ffff9181c69563c8) 00000080:00200000:1.0:1787826191.223391:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x72:0x0](ffff9181c4257448) 00000080:00200000:1.0:1787826191.223400:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x72:0x0](ffff9181c42500c8) 00000080:00200000:1.0:1787826191.223410:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x18:0x0](ffff9181c4256c08) 00000080:00200000:1.0:1787826191.223418:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x18:0x0](ffff9181c4254b08) 00000080:00200000:1.0:1787826191.223427:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x73:0x0](ffff9181c6950908) 00000080:00200000:1.0:1787826191.223435:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x73:0x0](ffff9181c6951148) 00000080:00200000:1.0:1787826191.223446:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x19:0x0](ffff9181c4250908) 00000080:00200000:1.0:1787826191.223454:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x19:0x0](ffff9181c42563c8) 00000080:00200000:1.0:1787826191.223461:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x74:0x0](ffff9181c6955b88) 00000080:00200000:1.0:1787826191.223469:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x74:0x0](ffff9181c6955348) 00000080:00200000:1.0:1787826191.223477:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x1a:0x0](ffff9181c7889988) 00000080:00200000:1.0:1787826191.223486:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x1a:0x0](ffff9181c7a963c8) 00000080:00200000:1.0:1787826191.223494:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x75:0x0](ffff9181c4259988) 00000080:00200000:1.0:1787826191.223502:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x75:0x0](ffff9181c6b3ba88) 00000080:00200000:1.0:1787826191.223509:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x1b:0x0](ffff9181c6bb9148) 00000080:00200000:1.0:1787826191.223516:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x1b:0x0](ffff9181c6bbb248) 00000080:00200000:1.0:1787826191.223525:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x76:0x0](ffff9181c6b3ec08) 00000080:00200000:1.0:1787826191.223533:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x76:0x0](ffff9181c6b3f448) 00000080:00200000:1.0:1787826191.223541:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x1c:0x0](ffff9181c6bdd348) 00000080:00200000:1.0:1787826191.223551:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x1c:0x0](ffff9181c6bd80c8) 00000080:00200000:1.0:1787826191.223559:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x77:0x0](ffff9181c6b3db88) 00000080:00200000:1.0:1787826191.223566:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x77:0x0](ffff9181c6b380c8) 00000080:00200000:1.0:1787826191.223574:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x1d:0x0](ffff9181c7815348) 00000080:00200000:1.0:1787826191.223582:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x1d:0x0](ffff9181c7816c08) 00000080:00200000:1.0:1787826191.223591:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x78:0x0](ffff9181c6bbec08) 00000080:00200000:1.0:1787826191.223600:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x78:0x0](ffff9181c6bba1c8) 00000080:00200000:1.0:1787826191.223608:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x1e:0x0](ffff9181c7813248) 00000080:00200000:1.0:1787826191.223617:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x1e:0x0](ffff9181c7810908) 00000080:00200000:1.0:1787826191.223625:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x79:0x0](ffff9181c6bdaa08) 00000080:00200000:1.0:1787826191.223634:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x79:0x0](ffff9181c6bdc2c8) 00000080:00200000:1.0:1787826191.223643:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x1f:0x0](ffff9181c6bd9148) 00000080:00200000:1.0:1787826191.223652:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x1f:0x0](ffff9181c6bda1c8) 00000080:00200000:1.0:1787826191.223659:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x7a:0x0](ffff9181c6bd9988) 00000080:00200000:1.0:1787826191.223666:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x7a:0x0](ffff9181c6bde3c8) 00000080:00200000:1.0:1787826191.223674:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x20:0x0](ffff9181c78142c8) 00000080:00200000:1.0:1787826191.223681:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x20:0x0](ffff9181c7811988) 00000080:00200000:1.0:1787826191.223688:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x7b:0x0](ffff9181c6bb80c8) 00000080:00200000:1.0:1787826191.223695:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x7b:0x0](ffff9181c6bbba88) 00000080:00200000:1.0:1787826191.223718:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x21:0x0](ffff9181c6b38908) 00000080:00200000:1.0:1787826191.223726:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x21:0x0](ffff9181c69c6c08) 00000080:00200000:1.0:1787826191.223734:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x7c:0x0](ffff9181c6bddb88) 00000080:00200000:1.0:1787826191.224042:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x7c:0x0](ffff9181c6bdec08) 00000080:00200000:1.0:1787826191.224057:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x22:0x0](ffff9181c78163c8) 00000080:00200000:1.0:1787826191.224065:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x22:0x0](ffff9181c6bcdb88) 00000080:00200000:1.0:1787826191.224074:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x7d:0x0](ffff9181c69c2a08) 00000080:00200000:1.0:1787826191.224083:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x7d:0x0](ffff9181c69c1988) 00000080:00200000:1.0:1787826191.224091:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x23:0x0](ffff9181c6bdb248) 00000080:00200000:1.0:1787826191.224100:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x23:0x0](ffff9181c6bdf448) 00000080:00200000:1.0:1787826191.224107:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x7e:0x0](ffff9181c6bdcb08) 00000080:00200000:1.0:1787826191.224114:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x7e:0x0](ffff9181c6b3cb08) 00000080:00200000:1.0:1787826191.224122:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x24:0x0](ffff9181c6bb8908) 00000080:00200000:1.0:1787826191.224130:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x24:0x0](ffff9181c6bd8908) 00000080:00200000:1.0:1787826191.224139:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x7f:0x0](ffff9181c6bccb08) 00000080:00200000:1.0:1787826191.224158:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x7f:0x0](ffff9181c6bcaa08) 00000080:00200000:1.0:1787826191.224168:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x25:0x0](ffff9181c6951988) 00000080:00200000:1.0:1787826191.224177:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x25:0x0](ffff9181c425e3c8) 00000080:00200000:1.0:1787826191.224186:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x80:0x0](ffff9181c69c63c8) 00000080:00200000:1.0:1787826191.224195:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x80:0x0](ffff9181c69c5b88) 00000080:00200000:1.0:1787826191.224203:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x26:0x0](ffff9181c6bcf448) 00000080:00200000:1.0:1787826191.224211:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x26:0x0](ffff9181c6bc9988) 00000080:00200000:1.0:1787826191.224218:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x81:0x0](ffff9181c4252a08) 00000080:00200000:1.0:1787826191.224226:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x81:0x0](ffff9181c786e3c8) 00000080:00200000:1.0:1787826191.224233:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x27:0x0](ffff9181c786c2c8) 00000080:00200000:1.0:1787826191.224243:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x27:0x0](ffff9181c7868908) 00000080:00200000:1.0:1787826191.224253:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x82:0x0](ffff9181c7869148) 00000080:00200000:1.0:1787826191.224262:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x82:0x0](ffff9181c786b248) 00000080:00200000:1.0:1787826191.224270:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x28:0x0](ffff9181c7a0c2c8) 00000080:00200000:1.0:1787826191.224278:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x28:0x0](ffff9181c7a080c8) 00000080:00200000:1.0:1787826191.224286:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x83:0x0](ffff9181c786db88) 00000080:00200000:1.0:1787826191.224298:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x83:0x0](ffff9181c7869988) 00000080:00200000:1.0:1787826191.224305:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x29:0x0](ffff9181c6979988) 00000080:00200000:1.0:1787826191.224313:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x29:0x0](ffff9181c697d348) 00000080:00200000:1.0:1787826191.224321:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x84:0x0](ffff9181c697ec08) 00000080:00200000:1.0:1787826191.224328:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x84:0x0](ffff9181c697b248) 00000080:00200000:1.0:1787826191.224336:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x2a:0x0](ffff9181c786a1c8) 00000080:00200000:1.0:1787826191.224344:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x2a:0x0](ffff9181c786f448) 00000080:00200000:1.0:1787826191.224350:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x85:0x0](ffff9181c7a0d348) 00000080:00200000:1.0:1787826191.224358:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x85:0x0](ffff9181c7a08908) 00000080:00200000:1.0:1787826191.224367:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x2b:0x0](ffff9181c6bce3c8) 00000080:00200000:1.0:1787826191.224375:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x2b:0x0](ffff9181c6bcec08) 00000080:00200000:1.0:1787826191.224382:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x86:0x0](ffff9181c786ba88) 00000080:00200000:1.0:1787826191.224388:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x86:0x0](ffff9181c697e3c8) 00000080:00200000:1.0:1787826191.224395:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x2c:0x0](ffff9181c69c4b08) 00000080:00200000:1.0:1787826191.224402:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x2c:0x0](ffff9181c4255b88) 00000080:00200000:1.0:1787826191.224411:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x87:0x0](ffff9181c426ba88) 00000080:00200000:1.0:1787826191.224417:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x87:0x0](ffff9181c78680c8) 00000080:00200000:1.0:1787826191.224425:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x2d:0x0](ffff9181c6bc8908) 00000080:00200000:1.0:1787826191.224432:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x2d:0x0](ffff9181c6bca1c8) 00000080:00200000:1.0:1787826191.224438:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x88:0x0](ffff9181c7b23248) 00000080:00200000:1.0:1787826191.224445:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x88:0x0](ffff9181c7b25348) 00000080:00200000:1.0:1787826191.224452:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x2e:0x0](ffff9181c7a09148) 00000080:00200000:1.0:1787826191.224461:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x2e:0x0](ffff9181c7a0ec08) 00000080:00200000:1.0:1787826191.224468:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x89:0x0](ffff9181c6b8ec08) 00000080:00200000:1.0:1787826191.224477:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x89:0x0](ffff9181c6b8c2c8) 00000080:00200000:1.0:1787826191.224485:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x2f:0x0](ffff9181c6b8b248) 00000080:00200000:1.0:1787826191.224492:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x2f:0x0](ffff9181c6b8d348) 00000080:00200000:1.0:1787826191.224500:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x8a:0x0](ffff9181c6b880c8) 00000080:00200000:1.0:1787826191.224508:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x8a:0x0](ffff9181c6b8a1c8) 00000080:00200000:1.0:1787826191.224515:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x30:0x0](ffff9181c7b23a88) 00000080:00200000:1.0:1787826191.224522:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x30:0x0](ffff9181c7b21148) 00000080:00200000:1.0:1787826191.224531:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x8b:0x0](ffff9181c6b8ba88) 00000080:00200000:1.0:1787826191.224538:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x8b:0x0](ffff9181c6b89988) 00000080:00200000:1.0:1787826191.224546:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x31:0x0](ffff9181c6a8e3c8) 00000080:00200000:1.0:1787826191.224555:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x31:0x0](ffff9181c6a8db88) 00000080:00200000:1.0:1787826191.224564:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x8c:0x0](ffff9181c7a0ba88) 00000080:00200000:1.0:1787826191.224573:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x8c:0x0](ffff9181c7a0b248) 00000080:00200000:1.0:1787826191.224580:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x32:0x0](ffff9181c7a0a1c8) 00000080:00200000:1.0:1787826191.224588:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x32:0x0](ffff9181c69900c8) 00000080:00200000:1.0:1787826191.224595:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x8d:0x0](ffff9181c6992a08) 00000080:00200000:1.0:1787826191.224603:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x8d:0x0](ffff9181c6994b08) 00000080:00200000:1.0:1787826191.224611:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x33:0x0](ffff9181c6990908) 00000080:00200000:1.0:1787826191.224617:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x33:0x0](ffff9181c6991148) 00000080:00200000:1.0:1787826191.224625:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x8e:0x0](ffff9181c6a8c2c8) 00000080:00200000:1.0:1787826191.224633:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x8e:0x0](ffff9181c6a89148) 00000080:00200000:1.0:1787826191.224641:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x34:0x0](ffff9181c6993a88) 00000080:00200000:1.0:1787826191.224648:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x34:0x0](ffff9181c69963c8) 00000080:00200000:1.0:1787826191.224655:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x8f:0x0](ffff9181c6b8f448) 00000080:00200000:1.0:1787826191.224663:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x8f:0x0](ffff9181c426db88) 00000080:00200000:1.0:1787826191.224671:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x35:0x0](ffff9181c7b242c8) 00000080:00200000:1.0:1787826191.224679:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x35:0x0](ffff9181c7b20908) 00000080:00200000:1.0:1787826191.224687:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x90:0x0](ffff9181c6bc1148) 00000080:00200000:1.0:1787826191.224694:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x90:0x0](ffff9181c6bc63c8) 00000080:00200000:1.0:1787826191.224971:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x36:0x0](ffff9181c7b24b08) 00000080:00200000:1.0:1787826191.224983:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x36:0x0](ffff9181c7b22a08) 00000080:00200000:1.0:1787826191.224992:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x91:0x0](ffff9181c6991988) 00000080:00200000:1.0:1787826191.224999:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x91:0x0](ffff9181c6995b88) 00000080:00200000:1.0:1787826191.225007:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x37:0x0](ffff9181c6bc7448) 00000080:00200000:1.0:1787826191.225014:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x37:0x0](ffff9181c6bc4b08) 00000080:00200000:1.0:1787826191.225021:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x92:0x0](ffff9181c6bc5b88) 00000080:00200000:1.0:1787826191.225029:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x92:0x0](ffff9181c6bc5348) 00000080:00200000:1.0:1787826191.225036:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x38:0x0](ffff9181c6bc3a88) 00000080:00200000:1.0:1787826191.225044:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x38:0x0](ffff9181c6bc3248) 00000080:00200000:1.0:1787826191.225051:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x93:0x0](ffff9181c69ef448) 00000080:00200000:1.0:1787826191.225058:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x93:0x0](ffff9181c69ea1c8) 00000080:00200000:1.0:1787826191.225066:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x39:0x0](ffff9181c69edb88) 00000080:00200000:1.0:1787826191.225074:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x39:0x0](ffff9181c69eec08) 00000080:00200000:1.0:1787826191.225082:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x94:0x0](ffff9181c69e8908) 00000080:00200000:1.0:1787826191.225089:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x94:0x0](ffff9181c69e9988) 00000080:00200000:1.0:1787826191.225098:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x3a:0x0](ffff9181c7900908) 00000080:00200000:1.0:1787826191.225110:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x3a:0x0](ffff9181c79021c8) 00000080:00200000:1.0:1787826191.225118:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x95:0x0](ffff9181c6a8ba88) 00000080:00200000:1.0:1787826191.225127:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x95:0x0](ffff9181c6a88908) 00000080:00200000:1.0:1787826191.225135:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x3b:0x0](ffff9181c6a880c8) 00000080:00200000:1.0:1787826191.225187:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x3b:0x0](ffff9181c6a8cb08) 00000080:00200000:1.0:1787826191.225200:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x96:0x0](ffff9181c7903248) 00000080:00200000:1.0:1787826191.225211:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x96:0x0](ffff9181c7903a88) 00000080:00200000:1.0:1787826191.225219:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x3c:0x0](ffff9181c7901148) 00000080:00200000:1.0:1787826191.225227:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x3c:0x0](ffff9181c7905b88) 00000080:00200000:1.0:1787826191.225235:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x97:0x0](ffff9181c79063c8) 00000080:00200000:1.0:1787826191.225243:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x97:0x0](ffff9181c7907448) 00000080:00200000:1.0:1787826191.225251:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x3d:0x0](ffff9181c6997448) 00000080:00200000:1.0:1787826191.225259:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x3d:0x0](ffff9181c69921c8) 00000080:00200000:1.0:1787826191.225266:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x98:0x0](ffff9181c6bd6c08) 00000080:00200000:1.0:1787826191.225275:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x98:0x0](ffff9181c6bd5b88) 00000080:00200000:1.0:1787826191.225282:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x3e:0x0](ffff9181c7901988) 00000080:00200000:1.0:1787826191.225290:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x3e:0x0](ffff9181c7904b08) 00000080:00200000:1.0:1787826191.225298:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x99:0x0](ffff9181c6996c08) 00000080:00200000:1.0:1787826191.225306:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x99:0x0](ffff9181c6995348) 00000080:00200000:1.0:1787826191.225314:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x3f:0x0](ffff9181c6993248) 00000080:00200000:1.0:1787826191.225321:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x3f:0x0](ffff9181c7b221c8) 00000080:00200000:1.0:1787826191.225329:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x9a:0x0](ffff9181c6bc2a08) 00000080:00200000:1.0:1787826191.225337:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x9a:0x0](ffff9181c6a2f448) 00000080:00200000:1.0:1787826191.225345:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x40:0x0](ffff9181c6a8a1c8) 00000080:00200000:1.0:1787826191.225352:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x40:0x0](ffff9181c6a8b248) 00000080:00200000:1.0:1787826191.225359:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x9b:0x0](ffff9181c7906c08) 00000080:00200000:1.0:1787826191.225368:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x9b:0x0](ffff9181c79000c8) 00000080:00200000:1.0:1787826191.225375:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x41:0x0](ffff9181c6a2cb08) 00000080:00200000:1.0:1787826191.225382:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x41:0x0](ffff9181c6a2aa08) 00000080:00200000:1.0:1787826191.225390:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x9c:0x0](ffff9181c7851148) 00000080:00200000:1.0:1787826191.225397:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x9c:0x0](ffff9181c7855348) 00000080:00200000:1.0:1787826191.225405:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x42:0x0](ffff9181c7853248) 00000080:00200000:1.0:1787826191.225415:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x42:0x0](ffff9181c78542c8) 00000080:00200000:1.0:1787826191.225425:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x9d:0x0](ffff9181c6a2d348) 00000080:00200000:1.0:1787826191.225433:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x9d:0x0](ffff9181c6a2c2c8) 00000080:00200000:1.0:1787826191.225440:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x43:0x0](ffff9181c6bd4b08) 00000080:00200000:1.0:1787826191.225447:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x43:0x0](ffff9181c6bd1988) 00000080:00200000:1.0:1787826191.225455:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x9e:0x0](ffff9181c7852a08) 00000080:00200000:1.0:1787826191.225463:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x9e:0x0](ffff9181c7853a88) 00000080:00200000:1.0:1787826191.225470:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x44:0x0](ffff9181c7854b08) 00000080:00200000:1.0:1787826191.225477:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x44:0x0](ffff9181c7857448) 00000080:00200000:1.0:1787826191.225486:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x9f:0x0](ffff9181c6bd63c8) 00000080:00200000:1.0:1787826191.225493:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x9f:0x0](ffff9181c6bd42c8) 00000080:00200000:1.0:1787826191.225501:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x45:0x0](ffff9181c6a8f448) 00000080:00200000:1.0:1787826191.225508:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x45:0x0](ffff9181c6b8db88) 00000080:00200000:1.0:1787826191.225515:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xa0:0x0](ffff9181c7856c08) 00000080:00200000:1.0:1787826191.225525:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xa0:0x0](ffff9181c7851988) 00000080:00200000:1.0:1787826191.225532:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x46:0x0](ffff9181c6a280c8) 00000080:00200000:1.0:1787826191.225539:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x46:0x0](ffff9181c6a2db88) 00000080:00200000:1.0:1787826191.225546:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xa1:0x0](ffff9181c7902a08) 00000080:00200000:1.0:1787826191.225552:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xa1:0x0](ffff9181c69eb248) 00000080:00200000:1.0:1787826191.225559:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x47:0x0](ffff9181c6a2ec08) 00000080:00200000:1.0:1787826191.225566:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x47:0x0](ffff9181c6a29148) 00000080:00200000:1.0:1787826191.225573:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xa2:0x0](ffff9181c6a2e3c8) 00000080:00200000:1.0:1787826191.225581:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xa2:0x0](ffff9181c6a28908) 00000080:00200000:1.0:1787826191.225588:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x48:0x0](ffff9181c6a39148) 00000080:00200000:1.0:1787826191.225597:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x48:0x0](ffff9181c6a380c8) 00000080:00200000:1.0:1787826191.225604:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xa3:0x0](ffff9181c69d42c8) 00000080:00200000:1.0:1787826191.225611:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xa3:0x0](ffff9181c69d00c8) 00000080:00200000:1.0:1787826191.225619:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x49:0x0](ffff9181c6ba9148) 00000080:00200000:1.0:1787826191.226125:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x49:0x0](ffff9181c6baba88) 00000080:00200000:1.0:1787826191.227943:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xa4:0x0](ffff9181c6bd0908) 00000080:00200000:1.0:1787826191.227981:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xa4:0x0](ffff9181c6bd3a88) 00000080:00200000:1.0:1787826191.227992:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x4a:0x0](ffff9181c6badb88) 00000080:00200000:1.0:1787826191.228002:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x4a:0x0](ffff9181c6bae3c8) 00000080:00200000:1.0:1787826191.228010:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xa5:0x0](ffff9181c69d3248) 00000080:00200000:1.0:1787826191.228021:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xa5:0x0](ffff9181c69d21c8) 00000080:00200000:1.0:1787826191.228032:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x4b:0x0](ffff9181c6a3db88) 00000080:00200000:1.0:1787826191.228041:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x4b:0x0](ffff9181c6a3aa08) 00000080:00200000:1.0:1787826191.228050:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xa6:0x0](ffff9181c6ba8908) 00000080:00200000:1.0:1787826191.228059:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xa6:0x0](ffff9181c6ba80c8) 00000080:00200000:1.0:1787826191.228068:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x4c:0x0](ffff9181c79e3a88) 00000080:00200000:1.0:1787826191.228076:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x4c:0x0](ffff9181c79e42c8) 00000080:00200000:1.0:1787826191.228085:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xa7:0x0](ffff9181c6a3cb08) 00000080:00200000:1.0:1787826191.228094:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xa7:0x0](ffff9181c6a3ba88) 00000080:00200000:1.0:1787826191.228102:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x4d:0x0](ffff9181c69d3a88) 00000080:00200000:1.0:1787826191.228110:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x4d:0x0](ffff9181c69d0908) 00000080:00200000:1.0:1787826191.228120:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xa8:0x0](ffff9181c6a38908) 00000080:00200000:1.0:1787826191.228128:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xa8:0x0](ffff9181c6a3c2c8) 00000080:00200000:1.0:1787826191.228136:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x4e:0x0](ffff9181c6bab248) 00000080:00200000:3.0:1787826191.232456:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x4e:0x0](ffff9181c6baec08) 00000080:00200000:3.0:1787826191.232492:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xa9:0x0](ffff9181c69d1988) 00000080:00200000:3.0:1787826191.232506:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xa9:0x0](ffff9181c69d7448) 00000080:00200000:3.0:1787826191.232514:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x4f:0x0](ffff9181c6b8aa08) 00000080:00200000:3.0:1787826191.232524:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x4f:0x0](ffff9181c7850908) 00000080:00200000:3.0:1787826191.232533:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xaa:0x0](ffff9181c79e4b08) 00000080:00200000:3.0:1787826191.232548:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xaa:0x0](ffff9181c79e5b88) 00000080:00200000:3.0:1787826191.232558:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x50:0x0](ffff9181c7aa6c08) 00000080:00200000:3.0:1787826191.232566:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x50:0x0](ffff9181c7aa4b08) 00000080:00200000:3.0:1787826191.232574:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xab:0x0](ffff9181c6a3a1c8) 00000080:00200000:3.0:1787826191.232584:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xab:0x0](ffff9181c6bfd348) 00000080:00200000:3.0:1787826191.232594:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x51:0x0](ffff9181c6ba9988) 00000080:00200000:3.0:1787826191.232603:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x51:0x0](ffff9181c6baaa08) 00000080:00200000:3.0:1787826191.232612:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xac:0x0](ffff9181c7aa0908) 00000080:00200000:3.0:1787826191.232623:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xac:0x0](ffff9181c7aa5348) 00000080:00200000:3.0:1787826191.232631:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x52:0x0](ffff9181c6bf9988) 00000080:00200000:3.0:1787826191.232639:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x52:0x0](ffff9181c6bfba88) 00000080:00200000:3.0:1787826191.232647:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xad:0x0](ffff9181c79e6c08) 00000080:00200000:3.0:1787826191.232657:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xad:0x0](ffff9181c79e21c8) 00000080:00200000:3.0:1787826191.232666:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x53:0x0](ffff9181c7aa3a88) 00000080:00200000:3.0:1787826191.232673:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x53:0x0](ffff9181c7aa3248) 00000080:00200000:3.0:1787826191.232681:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xae:0x0](ffff9181c7aa42c8) 00000080:00200000:3.0:1787826191.232690:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xae:0x0](ffff9181c7aa2a08) 00000080:00200000:3.0:1787826191.232698:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x54:0x0](ffff9181c79e1988) 00000080:00200000:3.0:1787826191.232722:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x54:0x0](ffff9181c79e63c8) 00000080:00200000:3.0:1787826191.232731:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xaf:0x0](ffff9181c6baf448) 00000080:00200000:3.0:1787826191.232739:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xaf:0x0](ffff9181c6bacb08) 00000080:00200000:3.0:1787826191.232779:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x55:0x0](ffff9181c6bfb248) 00000080:00200000:3.0:1787826191.232791:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x55:0x0](ffff9181c6bf80c8) 00000080:00200000:3.0:1787826191.232801:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xb0:0x0](ffff9181c41a8908) 00000080:00200000:3.0:1787826191.232811:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xb0:0x0](ffff9181c41ad348) 00000080:00200000:3.0:1787826191.232820:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x56:0x0](ffff9181c7aa5b88) 00000080:00200000:3.0:1787826191.232828:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x56:0x0](ffff9181c7aa7448) 00000080:00200000:3.0:1787826191.232836:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xb1:0x0](ffff9181c6baa1c8) 00000080:00200000:3.0:1787826191.232856:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xb1:0x0](ffff9181c6a3b248) 00000080:00200000:3.0:1787826191.232867:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x57:0x0](ffff9181c41ae3c8) 00000080:00200000:3.0:1787826191.232878:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x57:0x0](ffff9181c41adb88) 00000080:00200000:3.0:1787826191.232886:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xb2:0x0](ffff9181c41ac2c8) 00000080:00200000:3.0:1787826191.232896:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xb2:0x0](ffff9181c41a9148) 00000080:00200000:3.0:1787826191.232904:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x58:0x0](ffff9181c41aaa08) 00000080:00200000:3.0:1787826191.232911:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x58:0x0](ffff9181c41a9988) 00000080:00200000:3.0:1787826191.232919:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xb3:0x0](ffff9181c799ec08) 00000080:00200000:3.0:1787826191.232929:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xb3:0x0](ffff9181c799cb08) 00000080:00200000:3.0:1787826191.232942:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x59:0x0](ffff9181c79e7448) 00000080:00200000:3.0:1787826191.232952:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x59:0x0](ffff9181c79e2a08) 00000080:00200000:3.0:1787826191.232961:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xb4:0x0](ffff9181c7af1988) 00000080:00200000:3.0:1787826191.232969:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xb4:0x0](ffff9181c7af3248) 00000080:00200000:3.0:1787826191.232978:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x5a:0x0](ffff9181c799b248) 00000080:00200000:3.0:1787826191.232986:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x5a:0x0](ffff9181c799d348) 00000080:00200000:3.0:1787826191.232995:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xb5:0x0](ffff9181c7af21c8) 00000080:00200000:3.0:1787826191.233004:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xb5:0x0](ffff9181c7af7448) 00000080:00200000:3.0:1787826191.233012:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x5b:0x0](ffff9181c6bfcb08) 00000080:00200000:3.0:1787826191.233020:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x5b:0x0](ffff9181c6bff448) 00000080:00200000:3.0:1787826191.233027:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xb6:0x0](ffff9181c6bfa1c8) 00000080:00200000:3.0:1787826191.233035:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xb6:0x0](ffff9181c6bfdb88) 00000080:00200000:3.0:1787826191.233043:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x5c:0x0](ffff9181c7af6c08) 00000080:00200000:3.0:1787826191.233050:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x5c:0x0](ffff9181c7af63c8) 00000080:00200000:3.0:1787826191.233058:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xb7:0x0](ffff9181c7af2a08) 00000080:00200000:3.0:1787826191.233066:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xb7:0x0](ffff9181c7af4b08) 00000080:00200000:3.0:1787826191.233074:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x5d:0x0](ffff9181c41aba88) 00000080:00200000:3.0:1787826191.233081:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x5d:0x0](ffff9181c41880c8) 00000080:00200000:3.0:1787826191.233088:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xb8:0x0](ffff9181c418b248) 00000080:00200000:3.0:1787826191.233096:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xb8:0x0](ffff9181c4188908) 00000080:00200000:3.0:1787826191.233104:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x5e:0x0](ffff9181c79e00c8) 00000080:00200000:3.0:1787826191.233113:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x5e:0x0](ffff9181c79e0908) 00000080:00200000:3.0:1787826191.233120:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xb9:0x0](ffff9181c79980c8) 00000080:00200000:3.0:1787826191.233132:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xb9:0x0](ffff9181c799aa08) 00000080:00200000:3.0:1787826191.233140:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x5f:0x0](ffff9181c418ec08) 00000080:00200000:3.0:1787826191.233160:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x5f:0x0](ffff9181c418cb08) 00000080:00200000:3.0:1787826191.233168:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xba:0x0](ffff9181c418d348) 00000080:00200000:3.0:1787826191.233175:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xba:0x0](ffff9181c418aa08) 00000080:00200000:3.0:1787826191.233184:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x60:0x0](ffff9181c79e5348) 00000080:00200000:3.0:1787826191.233192:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x60:0x0](ffff9181c79e3248) 00000080:00200000:3.0:1787826191.233200:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xbb:0x0](ffff9181c4189148) 00000080:00200000:3.0:1787826191.233209:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xbb:0x0](ffff9181c418c2c8) 00000080:00200000:3.0:1787826191.233217:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x61:0x0](ffff9181c7999988) 00000080:00200000:3.0:1787826191.233225:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x61:0x0](ffff9181c6bac2c8) 00000080:00200000:3.0:1787826191.233233:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xbc:0x0](ffff9181c694ba88) 00000080:00200000:3.0:1787826191.233240:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xbc:0x0](ffff9181c694db88) 00000080:00200000:3.0:1787826191.233249:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x62:0x0](ffff9181c6949988) 00000080:00200000:3.0:1787826191.233257:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x62:0x0](ffff9181c6948908) 00000080:00200000:3.0:1787826191.233265:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xbd:0x0](ffff9181c7a13a88) 00000080:00200000:3.0:1787826191.233274:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xbd:0x0](ffff9181c7a142c8) 00000080:00200000:3.0:1787826191.233281:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x63:0x0](ffff9181c7a16c08) 00000080:00200000:3.0:1787826191.233289:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x63:0x0](ffff9181c7a11148) 00000080:00200000:3.0:1787826191.233297:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xbe:0x0](ffff9181c7a14b08) 00000080:00200000:3.0:1787826191.233305:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xbe:0x0](ffff9181c7a121c8) 00000080:00200000:3.0:1787826191.233313:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x64:0x0](ffff9181c7a10908) 00000080:00200000:3.0:1787826191.233321:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x64:0x0](ffff9181c7a15348) 00000080:00200000:3.0:1787826191.233329:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xbf:0x0](ffff9181c4263248) 00000080:00200000:3.0:1787826191.233339:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xbf:0x0](ffff9181c4261148) 00000080:00200000:3.0:1787826191.233348:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x65:0x0](ffff9181c4265348) 00000080:00200000:3.0:1787826191.233355:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x65:0x0](ffff9181c42663c8) 00000080:00200000:3.0:1787826191.233363:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xc0:0x0](ffff9181c7aa00c8) 00000080:00200000:3.0:1787826191.233372:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xc0:0x0](ffff9181c7ae42c8) 00000080:00200000:3.0:1787826191.233381:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x66:0x0](ffff9181c42642c8) 00000080:00200000:3.0:1787826191.233388:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x66:0x0](ffff9181c4267448) 00000080:00200000:3.0:1787826191.233395:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xc1:0x0](ffff9181c7ae6c08) 00000080:00200000:3.0:1787826191.233403:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xc1:0x0](ffff9181c7ae00c8) 00000080:00200000:3.0:1787826191.233410:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x67:0x0](ffff9181c79e1148) 00000080:00200000:3.0:1787826191.233419:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x67:0x0](ffff9181c6a4c2c8) 00000080:00200000:3.0:1787826191.233427:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xc2:0x0](ffff9181c7ae3248) 00000080:00200000:3.0:1787826191.233435:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xc2:0x0](ffff9181c7ae3a88) 00000080:00200000:3.0:1787826191.233443:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x68:0x0](ffff9181c4266c08) 00000080:00200000:3.0:1787826191.233451:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x68:0x0](ffff9181c4265b88) 00000080:00200000:3.0:1787826191.233460:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xc3:0x0](ffff9181c7ae1988) 00000080:00200000:3.0:1787826191.233467:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xc3:0x0](ffff9181c7ae2a08) 00000080:00200000:3.0:1787826191.233474:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x69:0x0](ffff9181c6a49148) 00000080:00200000:3.0:1787826191.233483:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x69:0x0](ffff9181c6a4b248) 00000080:00200000:3.0:1787826191.233493:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xc4:0x0](ffff9181c7ae5b88) 00000080:00200000:3.0:1787826191.233502:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xc4:0x0](ffff9181c7ae21c8) 00000080:00200000:3.0:1787826191.233509:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x6a:0x0](ffff9181c6aea1c8) 00000080:00200000:3.0:1787826191.233518:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x6a:0x0](ffff9181c6aeec08) 00000080:00200000:3.0:1787826191.233526:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xc5:0x0](ffff9181c694d348) 00000080:00200000:3.0:1787826191.233535:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xc5:0x0](ffff9181c694ec08) 00000080:00200000:3.0:1787826191.233541:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x6b:0x0](ffff9181c6ae80c8) 00000080:00200000:3.0:1787826191.233549:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x6b:0x0](ffff9181c6aec2c8) 00000080:00200000:3.0:1787826191.233558:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xc6:0x0](ffff9181c694f448) 00000080:00200000:3.0:1787826191.233567:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xc6:0x0](ffff9181c694b248) 00000080:00200000:3.0:1787826191.233574:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x6c:0x0](ffff9181c6aeba88) 00000080:00200000:3.0:1787826191.233582:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x6c:0x0](ffff9181c6aef448) 00000080:00200000:3.0:1787826191.233590:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xc7:0x0](ffff9181c6aecb08) 00000080:00200000:3.0:1787826191.233598:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xc7:0x0](ffff9181c6aeb248) 00000080:00200000:3.0:1787826191.233606:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x6d:0x0](ffff9181c694cb08) 00000080:00200000:3.0:1787826191.233614:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x6d:0x0](ffff9181c694a1c8) 00000080:00200000:3.0:1787826191.233623:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xc8:0x0](ffff9181c6ae9988) 00000080:00200000:3.0:1787826191.233631:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xc8:0x0](ffff9181c6aedb88) 00000080:00200000:3.0:1787826191.233640:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x6e:0x0](ffff9181c4260908) 00000080:00200000:3.0:1787826191.233648:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x6e:0x0](ffff9181c7a021c8) 00000080:00200000:3.0:1787826191.233656:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xc9:0x0](ffff9181c694aa08) 00000080:00200000:3.0:1787826191.233664:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xc9:0x0](ffff9181c6aed348) 00000080:00200000:3.0:1787826191.233670:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x6f:0x0](ffff9181c7a01148) 00000080:00200000:3.0:1787826191.233679:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x6f:0x0](ffff9181c7a063c8) 00000080:00200000:3.0:1787826191.233687:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xca:0x0](ffff9181c7a05348) 00000080:00200000:3.0:1787826191.233695:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xca:0x0](ffff9181c7a042c8) 00000080:00200000:3.0:1787826191.233702:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x70:0x0](ffff9181c7a000c8) 00000080:00200000:3.0:1787826191.233713:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x70:0x0](ffff9181c7a03a88) 00000080:00200000:3.0:1787826191.233721:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xcb:0x0](ffff9181c6bfaa08) 00000080:00200000:3.0:1787826191.233729:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xcb:0x0](ffff9181c6bfe3c8) 00000080:00200000:3.0:1787826191.233738:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x71:0x0](ffff9181c7a05b88) 00000080:00200000:3.0:1787826191.233777:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x71:0x0](ffff9181c7a06c08) 00000080:00200000:3.0:1787826191.233788:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xcc:0x0](ffff9181c7ae4b08) 00000080:00200000:3.0:1787826191.233797:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xcc:0x0](ffff9181c7ae1148) 00000080:00200000:3.0:1787826191.233805:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x72:0x0](ffff9181c7ae0908) 00000080:00200000:3.0:1787826191.233814:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x72:0x0](ffff9181c41b3a88) 00000080:00200000:3.0:1787826191.233822:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xcd:0x0](ffff9181c7a07448) 00000080:00200000:3.0:1787826191.233830:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xcd:0x0](ffff9181c7a01988) 00000080:00200000:3.0:1787826191.233838:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x73:0x0](ffff9181c7809148) 00000080:00200000:3.0:1787826191.233856:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x73:0x0](ffff9181c780c2c8) 00000080:00200000:3.0:1787826191.233865:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xce:0x0](ffff9181c7808908) 00000080:00200000:3.0:1787826191.233874:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xce:0x0](ffff9181c7809988) 00000080:00200000:3.0:1787826191.233882:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x74:0x0](ffff9181c6a4d348) 00000080:00200000:3.0:1787826191.233890:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x74:0x0](ffff9181c6a49988) 00000080:00200000:3.0:1787826191.233897:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xcf:0x0](ffff9181c78080c8) 00000080:00200000:3.0:1787826191.233908:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xcf:0x0](ffff9181c780b248) 00000080:00200000:3.0:1787826191.233916:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x75:0x0](ffff9181c6a4ec08) 00000080:00200000:3.0:1787826191.233926:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x75:0x0](ffff9181c6a480c8) 00000080:00200000:3.0:1787826191.233936:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xd0:0x0](ffff9181c7a00908) 00000080:00200000:3.0:1787826191.233943:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xd0:0x0](ffff9181c6a4a1c8) 00000080:00200000:3.0:1787826191.233950:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x76:0x0](ffff9181c69480c8) 00000080:00200000:3.0:1787826191.233958:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x76:0x0](ffff9181c694c2c8) 00000080:00200000:3.0:1787826191.233966:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xd1:0x0](ffff9181c793e3c8) 00000080:00200000:3.0:1787826191.233974:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xd1:0x0](ffff9181c793ec08) 00000080:00200000:3.0:1787826191.233982:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x77:0x0](ffff9181c793aa08) 00000080:00200000:3.0:1787826191.233991:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x77:0x0](ffff9181c793d348) 00000080:00200000:3.0:1787826191.233999:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xd2:0x0](ffff9181c793c2c8) 00000080:00200000:3.0:1787826191.234007:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xd2:0x0](ffff9181c793b248) 00000080:00200000:3.0:1787826191.234016:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x78:0x0](ffff9181c7ae5348) 00000080:00200000:3.0:1787826191.234024:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x78:0x0](ffff9181c7a1d348) 00000080:00200000:3.0:1787826191.234033:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xd3:0x0](ffff9181c7939988) 00000080:00200000:3.0:1787826191.234041:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xd3:0x0](ffff9181c7939148) 00000080:00200000:3.0:1787826191.234048:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x79:0x0](ffff9181c780a1c8) 00000080:00200000:3.0:1787826191.234057:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x79:0x0](ffff9181c791ba88) 00000080:00200000:3.0:1787826191.234065:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xd4:0x0](ffff9181c79380c8) 00000080:00200000:3.0:1787826191.234074:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xd4:0x0](ffff9181c7938908) 00000080:00200000:3.0:1787826191.234081:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x7a:0x0](ffff9181c7a1e3c8) 00000080:00200000:3.0:1787826191.234088:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x7a:0x0](ffff9181c7a180c8) 00000080:00200000:3.0:1787826191.234097:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xd5:0x0](ffff9181c7a03248) 00000080:00200000:3.0:1787826191.234104:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xd5:0x0](ffff9181c7914b08) 00000080:00200000:3.0:1787826191.234112:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x7b:0x0](ffff9181c7a19148) 00000080:00200000:3.0:1787826191.234119:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x7b:0x0](ffff9181c7a1ec08) 00000080:00200000:3.0:1787826191.234126:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xd6:0x0](ffff9181c791f448) 00000080:00200000:3.0:1787826191.234134:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xd6:0x0](ffff9181c791ec08) 00000080:00200000:3.0:1787826191.234141:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x7c:0x0](ffff9181c7916c08) 00000080:00200000:3.0:1787826191.234149:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x7c:0x0](ffff9181c79121c8) 00000080:00200000:3.0:1787826191.234157:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xd7:0x0](ffff9181c79163c8) 00000080:00200000:3.0:1787826191.234164:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xd7:0x0](ffff9181c79100c8) 00000080:00200000:3.0:1787826191.234172:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x7d:0x0](ffff9181c7913248) 00000080:00200000:3.0:1787826191.234181:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x7d:0x0](ffff9181c7915b88) 00000080:00200000:3.0:1787826191.234189:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xd8:0x0](ffff9181c7911988) 00000080:00200000:3.0:1787826191.234196:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xd8:0x0](ffff9181c7917448) 00000080:00200000:3.0:1787826191.234204:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x7e:0x0](ffff9181c7915348) 00000080:00200000:3.0:1787826191.234211:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x7e:0x0](ffff9181c7911148) 00000080:00200000:3.0:1787826191.234219:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xd9:0x0](ffff9181c7912a08) 00000080:00200000:3.0:1787826191.234227:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xd9:0x0](ffff9181c793cb08) 00000080:00200000:3.0:1787826191.234235:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x7f:0x0](ffff9181c7919988) 00000080:00200000:3.0:1787826191.234243:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x7f:0x0](ffff9181c791a1c8) 00000080:00200000:3.0:1787826191.234251:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xda:0x0](ffff9181c7919148) 00000080:00200000:3.0:1787826191.234259:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xda:0x0](ffff9181c79180c8) 00000080:00200000:3.0:1787826191.234267:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x80:0x0](ffff9181c79c42c8) 00000080:00200000:3.0:1787826191.234274:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x80:0x0](ffff9181c79c5348) 00000080:00200000:3.0:1787826191.234281:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xdb:0x0](ffff9181c791e3c8) 00000080:00200000:3.0:1787826191.234289:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xdb:0x0](ffff9181c791db88) 00000080:00200000:3.0:1787826191.234297:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x81:0x0](ffff9181c41b63c8) 00000080:00200000:3.0:1787826191.234306:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x81:0x0](ffff9181c41b0908) 00000080:00200000:3.0:1787826191.234313:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xdc:0x0](ffff9181c791cb08) 00000080:00200000:3.0:1787826191.234320:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xdc:0x0](ffff9181c791d348) 00000080:00200000:3.0:1787826191.234327:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x82:0x0](ffff9181c7827448) 00000080:00200000:3.0:1787826191.234335:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x82:0x0](ffff9181c78200c8) 00000080:00200000:3.0:1787826191.234344:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xdd:0x0](ffff9181c7a1c2c8) 00000080:00200000:3.0:1787826191.234352:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xdd:0x0](ffff9181c7a1f448) 00000080:00200000:3.0:1787826191.234360:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x83:0x0](ffff9181c7822a08) 00000080:00200000:3.0:1787826191.234368:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x83:0x0](ffff9181c7825348) 00000080:00200000:3.0:1787826191.234375:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xde:0x0](ffff9181c41b1148) 00000080:00200000:3.0:1787826191.234383:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xde:0x0](ffff9181c41b00c8) 00000080:00200000:3.0:1787826191.234390:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x84:0x0](ffff9181c79c1988) 00000080:00200000:3.0:1787826191.234398:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x84:0x0](ffff9181c79c21c8) 00000080:00200000:3.0:1787826191.234406:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xdf:0x0](ffff9181c7823a88) 00000080:00200000:3.0:1787826191.234414:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xdf:0x0](ffff9181c78221c8) 00000080:00200000:3.0:1787826191.234422:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x85:0x0](ffff9181c41b2a08) 00000080:00200000:3.0:1787826191.234430:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x85:0x0](ffff9181c41b3248) 00000080:00200000:3.0:1787826191.234437:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xe0:0x0](ffff9181c79c3a88) 00000080:00200000:3.0:1787826191.234444:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xe0:0x0](ffff9181c79c5b88) 00000080:00200000:3.0:1787826191.234452:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x86:0x0](ffff9181c78242c8) 00000080:00200000:3.0:1787826191.234460:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x86:0x0](ffff9181c7826c08) 00000080:00200000:3.0:1787826191.234468:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xe1:0x0](ffff9181c7820908) 00000080:00200000:3.0:1787826191.234476:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xe1:0x0](ffff9181c7823248) 00000080:00200000:3.0:1787826191.234482:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x87:0x0](ffff9181c787ba88) 00000080:00200000:3.0:1787826191.234489:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x87:0x0](ffff9181c787ec08) 00000080:00200000:3.0:1787826191.234497:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xe2:0x0](ffff9181c7a1db88) 00000080:00200000:3.0:1787826191.234505:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xe2:0x0](ffff9181c7a1aa08) 00000080:00200000:3.0:1787826191.234519:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x88:0x0](ffff9181c7878908) 00000080:00200000:3.0:1787826191.234528:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x88:0x0](ffff9181c787d348) 00000080:00200000:3.0:1787826191.234535:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xe3:0x0](ffff9181c787cb08) 00000080:00200000:3.0:1787826191.234543:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xe3:0x0](ffff9181c787b248) 00000080:00200000:3.0:1787826191.234551:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x89:0x0](ffff9181c7824b08) 00000080:00200000:3.0:1787826191.234559:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x89:0x0](ffff9181c79c1148) 00000080:00200000:3.0:1787826191.234566:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xe4:0x0](ffff9181c78780c8) 00000080:00200000:3.0:1787826191.234574:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xe4:0x0](ffff9181c7879148) 00000080:00200000:3.0:1787826191.234582:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x8a:0x0](ffff9181c6b20908) 00000080:00200000:3.0:1787826191.234590:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x8a:0x0](ffff9181c6b25b88) 00000080:00200000:3.0:1787826191.234598:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xe5:0x0](ffff9181c6b221c8) 00000080:00200000:3.0:1787826191.234606:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xe5:0x0](ffff9181c6b200c8) 00000080:00200000:3.0:1787826191.234614:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x8b:0x0](ffff9181c787db88) 00000080:00200000:3.0:1787826191.234622:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x8b:0x0](ffff9181c787c2c8) 00000080:00200000:3.0:1787826191.234630:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xe6:0x0](ffff9181c787e3c8) 00000080:00200000:3.0:1787826191.234637:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xe6:0x0](ffff9181c7913a88) 00000080:00200000:3.0:1787826191.234646:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x8c:0x0](ffff9181c6b24b08) 00000080:00200000:3.0:1787826191.234655:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x8c:0x0](ffff9181c6b26c08) 00000080:00200000:3.0:1787826191.234664:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xe7:0x0](ffff9181c6b23a88) 00000080:00200000:3.0:1787826191.234673:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xe7:0x0](ffff9181c6b21988) 00000080:00200000:3.0:1787826191.234680:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x8d:0x0](ffff9181c6b21148) 00000080:00200000:3.0:1787826191.234687:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x8d:0x0](ffff9181c79cba88) 00000080:00200000:3.0:1787826191.234695:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xe8:0x0](ffff9181c69cb248) 00000080:00200000:3.0:1787826191.234703:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xe8:0x0](ffff9181c69cec08) 00000080:00200000:3.0:1787826191.234711:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x8e:0x0](ffff9181c41b1988) 00000080:00200000:3.0:1787826191.234719:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x8e:0x0](ffff9181c6b00908) 00000080:00200000:3.0:1787826191.234727:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xe9:0x0](ffff9181c69ce3c8) 00000080:00200000:3.0:1787826191.234735:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xe9:0x0](ffff9181c69c9148) 00000080:00200000:3.0:1787826191.234772:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x8f:0x0](ffff9181c79ccb08) 00000080:00200000:3.0:1787826191.234785:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x8f:0x0](ffff9181c79c8908) 00000080:00200000:3.0:1787826191.234793:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xea:0x0](ffff9181c79cdb88) 00000080:00200000:3.0:1787826191.234802:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xea:0x0](ffff9181c79cc2c8) 00000080:00200000:3.0:1787826191.234810:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x90:0x0](ffff9181c69cd348) 00000080:00200000:3.0:1787826191.234818:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x90:0x0](ffff9181c69cc2c8) 00000080:00200000:3.0:1787826191.234826:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xeb:0x0](ffff9181c79caa08) 00000080:00200000:3.0:1787826191.234834:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xeb:0x0](ffff9181c79c9988) 00000080:00200000:3.0:1787826191.234843:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x91:0x0](ffff9181c69caa08) 00000080:00200000:3.0:1787826191.234864:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x91:0x0](ffff9181c69cdb88) 00000080:00200000:3.0:1787826191.234871:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xec:0x0](ffff9181c7a1a1c8) 00000080:00200000:3.0:1787826191.234879:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xec:0x0](ffff9181c7a19988) 00000080:00200000:3.0:1787826191.234886:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x92:0x0](ffff9181c69c9988) 00000080:00200000:3.0:1787826191.234894:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x92:0x0](ffff9181c69c80c8) 00000080:00200000:3.0:1787826191.234902:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xed:0x0](ffff9181c69ca1c8) 00000080:00200000:3.0:1787826191.234910:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xed:0x0](ffff9181c69cf448) 00000080:00200000:3.0:1787826191.234918:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x93:0x0](ffff9181c787aa08) 00000080:00200000:3.0:1787826191.234926:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x93:0x0](ffff9181c7992a08) 00000080:00200000:3.0:1787826191.234934:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xee:0x0](ffff9181c79900c8) 00000080:00200000:3.0:1787826191.234942:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xee:0x0](ffff9181c7995348) 00000080:00200000:3.0:1787826191.234950:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x94:0x0](ffff9181c7991988) 00000080:00200000:3.0:1787826191.234958:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x94:0x0](ffff9181c7993248) 00000080:00200000:3.0:1787826191.234966:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xef:0x0](ffff9181c79cb248) 00000080:00200000:3.0:1787826191.234974:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xef:0x0](ffff9181c79ca1c8) 00000080:00200000:3.0:1787826191.234982:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x95:0x0](ffff9181c7990908) 00000080:00200000:3.0:1787826191.234990:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x95:0x0](ffff9181c7994b08) 00000080:00200000:3.0:1787826191.234998:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xf0:0x0](ffff9181c79942c8) 00000080:00200000:3.0:1787826191.235007:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xf0:0x0](ffff9181c79921c8) 00000080:00200000:3.0:1787826191.235014:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x96:0x0](ffff9181c79ce3c8) 00000080:00200000:3.0:1787826191.235022:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x96:0x0](ffff9181c79cf448) 00000080:00200000:3.0:1787826191.235030:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xf1:0x0](ffff9181c4118908) 00000080:00200000:3.0:1787826191.235040:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xf1:0x0](ffff9181c411d348) 00000080:00200000:3.0:1787826191.235049:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x97:0x0](ffff9181c6b07448) 00000080:00200000:3.0:1787826191.235058:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x97:0x0](ffff9181c6b06c08) 00000080:00200000:3.0:1787826191.235066:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xf2:0x0](ffff9181c411b248) 00000080:00200000:3.0:1787826191.235073:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xf2:0x0](ffff9181c411ba88) 00000080:00200000:3.0:1787826191.235081:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x98:0x0](ffff9181c79c9148) 00000080:00200000:3.0:1787826191.235088:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x98:0x0](ffff9181c69cba88) 00000080:00200000:3.0:1787826191.235095:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xf3:0x0](ffff9181c411aa08) 00000080:00200000:3.0:1787826191.235104:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xf3:0x0](ffff9181c411a1c8) 00000080:00200000:3.0:1787826191.235112:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x99:0x0](ffff9181c6941148) 00000080:00200000:3.0:1787826191.235120:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x99:0x0](ffff9181c6945348) 00000080:00200000:3.0:1787826191.235127:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xf4:0x0](ffff9181c6946c08) 00000080:00200000:3.0:1787826191.235136:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xf4:0x0](ffff9181c6945b88) 00000080:00200000:3.0:1787826191.235143:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x9a:0x0](ffff9181c6ae6c08) 00000080:00200000:3.0:1787826191.235152:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x9a:0x0](ffff9181c6ae7448) 00000080:00200000:3.0:1787826191.235160:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xf5:0x0](ffff9181c411ec08) 00000080:00200000:3.0:1787826191.235169:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xf5:0x0](ffff9181c79cd348) 00000080:00200000:3.0:1787826191.235177:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x9b:0x0](ffff9181c7996c08) 00000080:00200000:3.0:1787826191.235184:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x9b:0x0](ffff9181c7991148) 00000080:00200000:3.0:1787826191.235192:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xf6:0x0](ffff9181c6b2f448) 00000080:00200000:3.0:1787826191.235200:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xf6:0x0](ffff9181c6b2ec08) 00000080:00200000:3.0:1787826191.235208:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x9c:0x0](ffff9181c6ae0908) 00000080:00200000:3.0:1787826191.235216:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x9c:0x0](ffff9181c6ae1988) 00000080:00200000:3.0:1787826191.235223:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xf7:0x0](ffff9181c6b2b248) 00000080:00200000:3.0:1787826191.235230:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xf7:0x0](ffff9181c6b280c8) 00000080:00200000:3.0:1787826191.235238:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x9d:0x0](ffff9181c6b29988) 00000080:00200000:3.0:1787826191.235249:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x9d:0x0](ffff9181c6b2a1c8) 00000080:00200000:3.0:1787826191.235258:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xf8:0x0](ffff9181c7807448) 00000080:00200000:3.0:1787826191.235266:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xf8:0x0](ffff9181c7802a08) 00000080:00200000:3.0:1787826191.235274:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x9e:0x0](ffff9181c6ae4b08) 00000080:00200000:3.0:1787826191.235281:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x9e:0x0](ffff9181c6ae5b88) 00000080:00200000:3.0:1787826191.235289:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xf9:0x0](ffff9181c6941988) 00000080:00200000:3.0:1787826191.235297:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xf9:0x0](ffff9181c69442c8) 00000080:00200000:3.0:1787826191.235305:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0x9f:0x0](ffff9181c69421c8) 00000080:00200000:3.0:1787826191.235313:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0x9f:0x0](ffff9181c69400c8) 00000080:00200000:3.0:1787826191.235321:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xfa:0x0](ffff9181c7806c08) 00000080:00200000:3.0:1787826191.235329:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xfa:0x0](ffff9181c78000c8) 00000080:00200000:3.0:1787826191.235337:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0xa0:0x0](ffff9181c6943a88) 00000080:00200000:3.0:1787826191.235345:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0xa0:0x0](ffff9181c6a31988) 00000080:00200000:3.0:1787826191.235353:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xfb:0x0](ffff9181c6a34b08) 00000080:00200000:3.0:1787826191.235361:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xfb:0x0](ffff9181c6a33a88) 00000080:00200000:3.0:1787826191.235369:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0xa1:0x0](ffff9181c6ae1148) 00000080:00200000:3.0:1787826191.235377:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0xa1:0x0](ffff9181c6ae3248) 00000080:00200000:3.0:1787826191.235384:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xfc:0x0](ffff9181c6b02a08) 00000080:00200000:3.0:1787826191.235392:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xfc:0x0](ffff9181c6b063c8) 00000080:00200000:3.0:1787826191.235400:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0xa2:0x0](ffff9181c6a32a08) 00000080:00200000:3.0:1787826191.235408:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0xa2:0x0](ffff9181c6a30908) 00000080:00200000:3.0:1787826191.235415:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xfd:0x0](ffff9181c6a321c8) 00000080:00200000:3.0:1787826191.235423:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xfd:0x0](ffff9181c6a31148) 00000080:00200000:3.0:1787826191.235431:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0xa3:0x0](ffff9181c78042c8) 00000080:00200000:3.0:1787826191.235438:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0xa3:0x0](ffff9181c7805348) 00000080:00200000:3.0:1787826191.235447:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xfe:0x0](ffff9181c7805b88) 00000080:00200000:3.0:1787826191.235455:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xfe:0x0](ffff9181c6938908) 00000080:00200000:3.0:1787826191.235463:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0xa4:0x0](ffff9181c6b03248) 00000080:00200000:3.0:1787826191.235470:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0xa4:0x0](ffff9181c6b03a88) 00000080:00200000:3.0:1787826191.235477:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0xff:0x0](ffff9181c693cb08) 00000080:00200000:3.0:1787826191.235486:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0xff:0x0](ffff9181c693c2c8) 00000080:00200000:3.0:1787826191.235493:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0xa5:0x0](ffff9181c693f448) 00000080:00200000:3.0:1787826191.235500:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0xa5:0x0](ffff9181c693aa08) 00000080:00200000:3.0:1787826191.235508:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x100:0x0](ffff9181c6a33248) 00000080:00200000:3.0:1787826191.235516:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x100:0x0](ffff9181c6b28908) 00000080:00200000:3.0:1787826191.235524:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0xa6:0x0](ffff9181c6b04b08) 00000080:00200000:3.0:1787826191.235531:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0xa6:0x0](ffff9181c6b05b88) 00000080:00200000:3.0:1787826191.235539:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x101:0x0](ffff9181c693ec08) 00000080:00200000:3.0:1787826191.235547:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x101:0x0](ffff9181c6a35b88) 00000080:00200000:3.0:1787826191.235555:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0xa7:0x0](ffff9181c6ae00c8) 00000080:00200000:3.0:1787826191.235563:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0xa7:0x0](ffff9181c6ae2a08) 00000080:00200000:3.0:1787826191.235571:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x102:0x0](ffff9181c7aecb08) 00000080:00200000:3.0:1787826191.235579:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x102:0x0](ffff9181c7aed348) 00000080:00200000:3.0:1787826191.235587:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0xa8:0x0](ffff9181c79842c8) 00000080:00200000:3.0:1787826191.235595:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0xa8:0x0](ffff9181c7985348) 00000080:00200000:3.0:1787826191.235604:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x103:0x0](ffff9181c79800c8) 00000080:00200000:3.0:1787826191.235611:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x103:0x0](ffff9181c7983a88) 00000080:00200000:3.0:1787826191.235619:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0xa9:0x0](ffff9181c7aec2c8) 00000080:00200000:3.0:1787826191.235627:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0xa9:0x0](ffff9181c7ae8908) 00000080:00200000:3.0:1787826191.235634:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x104:0x0](ffff9181c7980908) 00000080:00200000:3.0:1787826191.235642:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x104:0x0](ffff9181c7985b88) 00000080:00200000:3.0:1787826191.235650:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0xaa:0x0](ffff9181c6ae3a88) 00000080:00200000:3.0:1787826191.235658:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0xaa:0x0](ffff9181c6ae63c8) 00000080:00200000:3.0:1787826191.235666:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x105:0x0](ffff9181c7986c08) 00000080:00200000:3.0:1787826191.235674:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x105:0x0](ffff9181c7984b08) 00000080:00200000:3.0:1787826191.235682:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0xab:0x0](ffff9181c7987448) 00000080:00200000:3.0:1787826191.235690:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0xab:0x0](ffff9181c7981988) 00000080:00200000:3.0:1787826191.235697:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x106:0x0](ffff9181c7843a88) 00000080:00200000:3.0:1787826191.235706:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x106:0x0](ffff9181c78442c8) 00000080:00200000:3.0:1787826191.235715:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0xac:0x0](ffff9181c79821c8) 00000080:00200000:3.0:1787826191.235726:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0xac:0x0](ffff9181c693db88) 00000080:00200000:3.0:1787826191.235734:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x107:0x0](ffff9181c6b9b248) 00000080:00200000:3.0:1787826191.235769:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x107:0x0](ffff9181c6b9cb08) 00000080:00200000:3.0:1787826191.235784:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0xad:0x0](ffff9181c6b980c8) 00000080:00200000:3.0:1787826191.235795:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0xad:0x0](ffff9181c6b9db88) 00000080:00200000:3.0:1787826191.235803:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x108:0x0](ffff9181c6b300c8) 00000080:00200000:3.0:1787826191.235813:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x108:0x0](ffff9181c6b37448) 00000080:00200000:3.0:1787826191.235820:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0xae:0x0](ffff9181c6b99148) 00000080:00200000:3.0:1787826191.235828:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0xae:0x0](ffff9181c6b99988) 00000080:00200000:3.0:1787826191.235835:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x109:0x0](ffff9181c6b35348) 00000080:00200000:3.0:1787826191.235842:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x109:0x0](ffff9181c6b36c08) 00000080:00200000:3.0:1787826191.239651:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0xaf:0x0](ffff9181c7842a08) 00000080:00200000:3.0:1787826191.239677:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0xaf:0x0](ffff9181c7843248) 00000080:00200000:3.0:1787826191.239687:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df0:0x10a:0x0](ffff9181c6b35b88) 00000080:00200000:3.0:1787826191.239696:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240000fb9:0x10a:0x0](ffff9181c6b342c8) 00000080:00200000:3.0:1787826191.239716:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0xb0:0x0](ffff9181c6b9c2c8) 00000080:00200000:3.0:1787826191.239727:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0xb0:0x0](ffff9181c6b98908) 00000080:00200000:3.0:1787826191.239735:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000255c0:0x3:0x0](ffff9181c6b9aa08) 00000080:00200000:3.0:1787826191.239837:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001789:0x3:0x0](ffff9181c6b9ba88) 00000080:00200000:3.0:1787826191.239858:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0xb1:0x0](ffff9181c6ae42c8) 00000080:00200000:3.0:1787826191.239866:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0xb1:0x0](ffff9181c798a1c8) 00000080:00200000:3.0:1787826191.239875:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000255c0:0x4:0x0](ffff9181c7841988) 00000080:00200000:3.0:1787826191.239884:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001789:0x4:0x0](ffff9181c78463c8) 00000080:00200000:3.0:1787826191.239895:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0xb2:0x0](ffff9181c798cb08) 00000080:00200000:3.0:1787826191.239906:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0xb2:0x0](ffff9181c798db88) 00000080:00200000:3.0:1787826191.239913:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000255c0:0x5:0x0](ffff9181c6b31148) 00000080:00200000:3.0:1787826191.239921:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001789:0x5:0x0](ffff9181c6b363c8) 00000080:00200000:3.0:1787826191.239929:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0xb3:0x0](ffff9181c6b0aa08) 00000080:00200000:3.0:1787826191.239937:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0xb3:0x0](ffff9181c6b0c2c8) 00000080:00200000:3.0:1787826191.239945:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000255c0:0x6:0x0](ffff9181c6b0ba88) 00000080:00200000:3.0:1787826191.239953:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001789:0x6:0x0](ffff9181c6b0b248) 00000080:00200000:3.0:1787826191.239961:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0xb4:0x0](ffff9181c7ae9148) 00000080:00200000:3.0:1787826191.239969:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0xb4:0x0](ffff9181c7aeba88) 00000080:00200000:3.0:1787826191.239976:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000255c0:0x7:0x0](ffff9181c798b248) 00000080:00200000:3.0:1787826191.239985:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001789:0x7:0x0](ffff9181c798ba88) 00000080:00200000:3.0:1787826191.239994:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0xb5:0x0](ffff9181c7841148) 00000080:00200000:3.0:1787826191.240003:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0xb5:0x0](ffff9181c7844b08) 00000080:00200000:3.0:1787826191.240011:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000255c0:0x8:0x0](ffff9181c7840908) 00000080:00200000:3.0:1787826191.240019:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001789:0x8:0x0](ffff9181c6b9a1c8) 00000080:00200000:3.0:1787826191.240027:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0xb6:0x0](ffff9181c78400c8) 00000080:00200000:3.0:1787826191.240035:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0xb6:0x0](ffff9181c78721c8) 00000080:00200000:3.0:1787826191.240043:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000255c0:0x9:0x0](ffff9181c7870908) 00000080:00200000:3.0:1787826191.240051:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001789:0x9:0x0](ffff9181c7871988) 00000080:00200000:3.0:1787826191.240059:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0xb7:0x0](ffff9181c798e3c8) 00000080:00200000:3.0:1787826191.240067:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0xb7:0x0](ffff9181c798f448) 00000080:00200000:3.0:1787826191.240073:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000255c0:0xa:0x0](ffff9181c7876c08) 00000080:00200000:3.0:1787826191.240082:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001789:0xa:0x0](ffff9181c7877448) 00000080:00200000:3.0:1787826191.240089:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0xb8:0x0](ffff9181c798aa08) 00000080:00200000:3.0:1787826191.240096:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0xb8:0x0](ffff9181c7989988) 00000080:00200000:3.0:1787826191.240104:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000255c0:0xb:0x0](ffff9181c78700c8) 00000080:00200000:3.0:1787826191.240116:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001789:0xb:0x0](ffff9181c7873248) 00000080:00200000:3.0:1787826191.240124:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0xb9:0x0](ffff9181c6b86c08) 00000080:00200000:3.0:1787826191.240134:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0xb9:0x0](ffff9181c6b81148) 00000080:00200000:3.0:1787826191.240152:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000255c0:0xc:0x0](ffff9181c6b83248) 00000080:00200000:3.0:1787826191.240161:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001789:0xc:0x0](ffff9181c6b842c8) 00000080:00200000:3.0:1787826191.240169:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0xba:0x0](ffff9181c798d348) 00000080:00200000:3.0:1787826191.240177:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0xba:0x0](ffff9181c798ec08) 00000080:00200000:3.0:1787826191.240185:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000255c0:0xd:0x0](ffff9181c7aea1c8) 00000080:00200000:3.0:1787826191.240193:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001789:0xd:0x0](ffff9181c7aeaa08) 00000080:00200000:3.0:1787826191.240200:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001788:0xbb:0x0](ffff9181c6b83a88) 00000080:00200000:3.0:1787826191.240207:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200024df2:0xbb:0x0](ffff9181c6b85348) 00000080:00200000:3.0:1787826191.240215:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x145a:0x0](ffff9181c7862a08) 00000080:00200000:3.0:1787826191.240224:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000255c0:0xe:0x0](ffff9181c7866c08) 00000080:00200000:3.0:1787826191.240233:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001789:0xe:0x0](ffff9181c78663c8) 00000080:00200000:3.0:1787826191.240249:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2400007ed:0xa83:0x0](ffff9181c7860908) 00000080:00200000:3.0:1787826191.240268:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x145b:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.240281:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.240309:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x145b:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.240315:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x145b:0x0](ffff9181c6b800c8) 00000080:00200000:3.0:1787826191.240362:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2400007ed:0xa84:0x0](ffff9181c7989148) 00000080:00200000:3.0:1787826191.240372:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x145c:0x0](ffff9181c7864b08) 00000080:00200000:3.0:1787826191.240386:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x145d:0x0](ffff9181c6b821c8) 00000080:00200000:3.0:1787826191.240398:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x145e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.240403:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.240412:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x145e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.240416:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x145e:0x0](ffff9181c79880c8) 00000080:00200000:3.0:1787826191.240442:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x145f:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.240446:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.240453:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x145f:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.240457:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x145f:0x0](ffff9181c7863a88) 00000080:00200000:3.0:1787826191.240481:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1460:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.240486:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.240493:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1460:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.240497:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1460:0x0](ffff9181c6b30908) 00000080:00200000:3.0:1787826191.240518:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1461:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.240522:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.240528:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1461:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.240532:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1461:0x0](ffff9181c79aaa08) 00000080:00200000:3.0:1787826191.240552:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1462:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.240557:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.240564:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1462:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.240568:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1462:0x0](ffff9181c6b0e3c8) 00000080:00200000:3.0:1787826191.240592:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1463:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.240596:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.240603:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1463:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.240607:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1463:0x0](ffff9181c6b80908) 00000080:00200000:3.0:1787826191.240626:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1464:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.240631:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.240638:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1464:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.240641:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1464:0x0](ffff9181c6b82a08) 00000080:00200000:3.0:1787826191.240660:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1465:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.240665:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.240672:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1465:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.240676:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1465:0x0](ffff9181c6b84b08) 00000080:00200000:3.0:1787826191.240695:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1466:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.240699:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.240706:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1466:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.240710:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1466:0x0](ffff9181c79ad348) 00000080:00200000:3.0:1787826191.240734:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1467:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.240739:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.241136:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1467:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.241151:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1467:0x0](ffff9181c7ae9988) 00000080:00200000:3.0:1787826191.241180:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1468:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.241185:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.241193:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1468:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.241197:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1468:0x0](ffff9181c79a80c8) 00000080:00200000:3.0:1787826191.241221:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1469:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.241225:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.241232:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1469:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.241236:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1469:0x0](ffff9181c7aedb88) 00000080:00200000:3.0:1787826191.241258:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x146a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.241263:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.241271:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x146a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.241274:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x146a:0x0](ffff9181c6b0d348) 00000080:00200000:3.0:1787826191.241295:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x146b:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.241300:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.241307:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x146b:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.241311:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x146b:0x0](ffff9181c79ac2c8) 00000080:00200000:3.0:1787826191.241331:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x146c:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.241335:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.241341:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x146c:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.241345:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x146c:0x0](ffff9181c78763c8) 00000080:00200000:3.0:1787826191.241365:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x146d:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.241369:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.241375:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x146d:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.241378:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x146d:0x0](ffff9181c79adb88) 00000080:00200000:3.0:1787826191.241398:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x146e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.241403:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.241408:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x146e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.241412:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x146e:0x0](ffff9181c79aba88) 00000080:00200000:3.0:1787826191.241431:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x146f:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.241435:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.241441:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x146f:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.241444:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x146f:0x0](ffff9181c7861988) 00000080:00200000:3.0:1787826191.241466:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1470:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.241470:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.241475:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1470:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.241479:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1470:0x0](ffff9181c79aa1c8) 00000080:00200000:3.0:1787826191.241498:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1471:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.241502:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.241508:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1471:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.241512:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1471:0x0](ffff9181c69dba88) 00000080:00200000:3.0:1787826191.241531:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1472:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.241535:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.241541:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1472:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.241545:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1472:0x0](ffff9181c7865b88) 00000080:00200000:3.0:1787826191.241567:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1473:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.241571:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.241578:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1473:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.241582:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1473:0x0](ffff9181c79acb08) 00000080:00200000:3.0:1787826191.241605:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1474:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.241610:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.241617:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1474:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.241622:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1474:0x0](ffff9181c79ab248) 00000080:00200000:3.0:1787826191.241644:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1475:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.241649:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.241656:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1475:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.241660:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1475:0x0](ffff9181c79a8908) 00000080:00200000:3.0:1787826191.241682:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1476:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.241687:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.241694:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1476:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.241698:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1476:0x0](ffff9181c79a9988) 00000080:00200000:3.0:1787826191.241735:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1477:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.241740:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.241944:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1477:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.241949:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1477:0x0](ffff9181c79aec08) 00000080:00200000:3.0:1787826191.241979:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1478:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.241984:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.241991:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1478:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.241995:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1478:0x0](ffff9181c69da1c8) 00000080:00200000:3.0:1787826191.242024:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1479:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.242029:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.242036:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1479:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.242039:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1479:0x0](ffff9181c7863248) 00000080:00200000:3.0:1787826191.242067:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x147a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.242071:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.242078:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x147a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.242082:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x147a:0x0](ffff9181c7a8b248) 00000080:00200000:3.0:1787826191.242107:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x147b:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.242111:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.242118:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x147b:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.242122:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x147b:0x0](ffff9181c79af448) 00000080:00200000:3.0:1787826191.242228:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x147c:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.242234:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.242243:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x147c:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.242247:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x147c:0x0](ffff9181c4279988) 00000080:00200000:3.0:1787826191.242273:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x147d:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.242277:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.242282:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x147d:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.242286:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x147d:0x0](ffff9181c69dd348) 00000080:00200000:3.0:1787826191.242308:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x147e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.242312:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.242319:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x147e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.242323:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x147e:0x0](ffff9181c7a8e3c8) 00000080:00200000:3.0:1787826191.242349:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x147f:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.242354:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.242361:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x147f:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.242365:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x147f:0x0](ffff9181c69dcb08) 00000080:00200000:3.0:1787826191.242389:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1480:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.242393:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.242399:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1480:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.242403:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1480:0x0](ffff9181c427ba88) 00000080:00200000:3.0:1787826191.242425:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1481:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.242429:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.242435:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1481:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.242438:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1481:0x0](ffff9181c427b248) 00000080:00200000:3.0:1787826191.242458:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1482:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.242463:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.242469:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1482:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.242472:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1482:0x0](ffff9181c7a8d348) 00000080:00200000:3.0:1787826191.242492:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1483:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.242497:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.242503:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1483:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.242506:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1483:0x0](ffff9181c427db88) 00000080:00200000:3.0:1787826191.242528:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1484:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.242532:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.242540:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1484:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.242543:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1484:0x0](ffff9181c69d9148) 00000080:00200000:3.0:1787826191.242565:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1485:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.242570:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.242576:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1485:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.242579:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1485:0x0](ffff9181c6a20908) 00000080:00200000:3.0:1787826191.242603:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1486:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.242608:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.242615:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1486:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.242619:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1486:0x0](ffff9181c69e5b88) 00000080:00200000:3.0:1787826191.242646:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1487:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.242650:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.242657:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1487:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.242661:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1487:0x0](ffff9181c69e63c8) 00000080:00200000:3.0:1787826191.242689:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1488:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.242694:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.243982:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1488:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.243995:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1488:0x0](ffff9181c6a200c8) 00000080:00200000:3.0:1787826191.244049:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1489:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.244057:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.244069:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1489:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.244073:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1489:0x0](ffff9181c7a8a1c8) 00000080:00200000:3.0:1787826191.244104:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x148a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.244109:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.244116:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x148a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.244119:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x148a:0x0](ffff9181c7a89148) 00000080:00200000:3.0:1787826191.244141:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x148b:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:2.0:1787826191.262414:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x14f1:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.262419:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.262426:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x14f1:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.262430:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x14f1:0x0](ffff9181c695db88) 00000080:00200000:2.0:1787826191.262454:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x14f2:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.262458:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.262465:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x14f2:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.262469:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x14f2:0x0](ffff9181c785d348) 00000080:00200000:2.0:1787826191.262492:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x14f3:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.262497:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.262504:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x14f3:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.262508:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x14f3:0x0](ffff9181c7ade3c8) 00000080:00200000:2.0:1787826191.262532:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x14f4:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.262537:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.262543:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x14f4:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.262547:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x14f4:0x0](ffff9181c695ba88) 00000080:00200000:2.0:1787826191.262568:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x14f5:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.262572:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.262578:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x14f5:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.262581:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x14f5:0x0](ffff9181c785ec08) 00000080:00200000:2.0:1787826191.262601:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x14f6:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.262606:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.262612:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x14f6:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.262615:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x14f6:0x0](ffff9181c695b248) 00000080:00200000:2.0:1787826191.262636:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x14f7:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.262641:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.262647:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x14f7:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.262651:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x14f7:0x0](ffff9181c695aa08) 00000080:00200000:2.0:1787826191.262673:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x14f8:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.262678:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.262684:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x14f8:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.262689:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x14f8:0x0](ffff9181c695cb08) 00000080:00200000:2.0:1787826191.263982:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x14f9:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.263996:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.264012:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x14f9:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.264018:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x14f9:0x0](ffff9181c7adf448) 00000080:00200000:2.0:1787826191.264056:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x14fa:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.264061:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.264069:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x14fa:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.264072:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x14fa:0x0](ffff9181c781aa08) 00000080:00200000:2.0:1787826191.264095:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x14fb:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.264099:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.264105:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x14fb:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.264109:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x14fb:0x0](ffff9181c7ada1c8) 00000080:00200000:2.0:1787826191.264129:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x14fc:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.264133:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.264140:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x14fc:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.264294:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x14fc:0x0](ffff9181c781cb08) 00000080:00200000:2.0:1787826191.264323:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x14fd:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.264328:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.264337:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x14fd:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.264341:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x14fd:0x0](ffff9181c7818908) 00000080:00200000:2.0:1787826191.264366:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x14fe:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.264370:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.264376:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x14fe:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.264380:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x14fe:0x0](ffff9181c78580c8) 00000080:00200000:2.0:1787826191.264408:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x14ff:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.264413:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.264420:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x14ff:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.264424:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x14ff:0x0](ffff9181c4219988) 00000080:00200000:2.0:1787826191.264450:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1500:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.264454:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.264460:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1500:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.264464:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1500:0x0](ffff9181c7859148) 00000080:00200000:2.0:1787826191.264486:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1501:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.264490:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.264496:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1501:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.264500:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1501:0x0](ffff9181c7ad9988) 00000080:00200000:2.0:1787826191.264521:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1502:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.264525:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.264531:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1502:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.264535:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1502:0x0](ffff9181c785e3c8) 00000080:00200000:2.0:1787826191.264557:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1503:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.264561:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.264568:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1503:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.264572:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1503:0x0](ffff9181c7b09988) 00000080:00200000:2.0:1787826191.264595:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1504:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.264600:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.264606:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1504:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.264610:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1504:0x0](ffff9181c781c2c8) 00000080:00200000:2.0:1787826191.264630:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1505:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.264634:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.264641:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1505:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.264645:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1505:0x0](ffff9181c781d348) 00000080:00200000:2.0:1787826191.264665:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1506:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.264670:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.264677:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1506:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.264681:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1506:0x0](ffff9181c7b0db88) 00000080:00200000:2.0:1787826191.265290:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1507:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.265299:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.265309:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1507:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.265314:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1507:0x0](ffff9181c7b0ba88) 00000080:00200000:2.0:1787826191.265343:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1508:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.265348:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.265354:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1508:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.265358:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1508:0x0](ffff9181c7b0b248) 00000080:00200000:2.0:1787826191.265385:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1509:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.265389:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.265396:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1509:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.265400:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1509:0x0](ffff9181c781ec08) 00000080:00200000:2.0:1787826191.265426:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x150a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.265431:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.265438:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x150a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.265441:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x150a:0x0](ffff9181c7b0aa08) 00000080:00200000:2.0:1787826191.265481:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x150b:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.265487:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.265496:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x150b:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.265500:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x150b:0x0](ffff9181c79e9988) 00000080:00200000:2.0:1787826191.265526:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x150c:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.265531:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.265538:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x150c:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.265542:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x150c:0x0](ffff9181c7b0cb08) 00000080:00200000:2.0:1787826191.265565:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x150d:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.265571:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.265577:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x150d:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.265581:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x150d:0x0](ffff9181c7b08908) 00000080:00200000:2.0:1787826191.265603:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x150e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.265608:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.265615:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x150e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.265619:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x150e:0x0](ffff9181c7b0c2c8) 00000080:00200000:2.0:1787826191.265639:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x150f:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.265644:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.265651:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x150f:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.265655:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x150f:0x0](ffff9181c78180c8) 00000080:00200000:2.0:1787826191.265679:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1510:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.265683:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.265690:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1510:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.265694:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1510:0x0](ffff9181c7819148) 00000080:00200000:2.0:1787826191.267111:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1511:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.267124:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.267139:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1511:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.267157:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1511:0x0](ffff9181c781e3c8) 00000080:00200000:2.0:1787826191.267184:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1512:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.267189:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.267196:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1512:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.267201:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1512:0x0](ffff9181c7b0d348) 00000080:00200000:2.0:1787826191.267227:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1513:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.267232:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.267239:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1513:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.267243:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1513:0x0](ffff9181c785f448) 00000080:00200000:2.0:1787826191.267272:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1514:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.267277:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.267284:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1514:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.267287:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1514:0x0](ffff9181c781f448) 00000080:00200000:2.0:1787826191.267311:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1515:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.267316:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.267324:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1515:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.267327:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1515:0x0](ffff9181c79edb88) 00000080:00200000:2.0:1787826191.267354:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1516:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.267358:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.267365:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1516:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.267370:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1516:0x0](ffff9181c785a1c8) 00000080:00200000:2.0:1787826191.267394:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1517:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.267399:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.267406:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1517:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.267410:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1517:0x0](ffff9181c7b0ec08) 00000080:00200000:2.0:1787826191.267433:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1518:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.267438:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.267445:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1518:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.267448:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1518:0x0](ffff9181c781a1c8) 00000080:00200000:2.0:1787826191.267471:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1519:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.267475:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.267481:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1519:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.267484:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1519:0x0](ffff9181c7819988) 00000080:00200000:2.0:1787826191.267504:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x151a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.267508:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.267515:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x151a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.267518:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x151a:0x0](ffff9181c781db88) 00000080:00200000:2.0:1787826191.267540:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x151b:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.267545:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.267551:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x151b:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.267555:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x151b:0x0](ffff9181c79eba88) 00000080:00200000:2.0:1787826191.267579:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x151c:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.267583:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.267590:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x151c:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.267594:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x151c:0x0](ffff9181c79d5348) 00000080:00200000:2.0:1787826191.267617:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x151d:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.267622:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.267629:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x151d:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.267633:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x151d:0x0](ffff9181c7b080c8) 00000080:00200000:2.0:1787826191.267655:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x151e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.267659:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.267666:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x151e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.267670:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x151e:0x0](ffff9181c781ba88) 00000080:00200000:2.0:1787826191.267693:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x151f:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.267697:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.268251:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x151f:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.268257:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x151f:0x0](ffff9181c781b248) 00000080:00200000:2.0:1787826191.268290:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1520:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.268295:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.268303:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1520:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.268307:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1520:0x0](ffff9181c79db248) 00000080:00200000:2.0:1787826191.268327:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1521:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.268331:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.268338:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1521:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.268342:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1521:0x0](ffff9181c7b09148) 00000080:00200000:2.0:1787826191.268365:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1522:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.268370:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.268377:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1522:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.268381:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1522:0x0](ffff9181c79daa08) 00000080:00200000:2.0:1787826191.268404:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1523:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.268408:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.268416:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1523:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.268420:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1523:0x0](ffff9181c79d6c08) 00000080:00200000:2.0:1787826191.268445:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1524:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.268449:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.268456:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1524:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.268459:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1524:0x0](ffff9181c79dcb08) 00000080:00200000:2.0:1787826191.268480:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1525:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.268485:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.268491:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1525:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.268496:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1525:0x0](ffff9181c79d8908) 00000080:00200000:2.0:1787826191.268536:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1526:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.268541:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.268550:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1526:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.268554:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1526:0x0](ffff9181c7b0e3c8) 00000080:00200000:2.0:1787826191.268585:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1527:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.268590:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.268599:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1527:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.268604:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1527:0x0](ffff9181c79dc2c8) 00000080:00200000:2.0:1787826191.268629:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1528:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.268633:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.268640:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1528:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.268644:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1528:0x0](ffff9181c79eb248) 00000080:00200000:2.0:1787826191.268670:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1529:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.268675:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.268682:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1529:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.268686:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1529:0x0](ffff9181c79dd348) 00000080:00200000:2.0:1787826191.271087:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x152a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.271133:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.271195:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x152a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.271204:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x152a:0x0](ffff9181c7b0f448) 00000080:00200000:2.0:1787826191.271251:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x152b:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.271255:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.271264:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x152b:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.271268:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x152b:0x0](ffff9181c79dec08) 00000080:00200000:2.0:1787826191.271290:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x152c:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.271294:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.271301:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x152c:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.271305:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x152c:0x0](ffff9181c79d80c8) 00000080:00200000:2.0:1787826191.271328:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x152d:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.271333:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.271340:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x152d:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.271344:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x152d:0x0](ffff9181c79d9148) 00000080:00200000:2.0:1787826191.271367:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x152e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.271371:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.271378:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x152e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.271382:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x152e:0x0](ffff9181c79de3c8) 00000080:00200000:2.0:1787826191.271402:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x152f:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.271406:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.271412:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x152f:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.271416:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x152f:0x0](ffff9181c79d00c8) 00000080:00200000:2.0:1787826191.271446:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1530:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.271451:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.271457:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1530:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.271461:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1530:0x0](ffff9181c79d1148) 00000080:00200000:2.0:1787826191.271484:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1531:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.271489:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.271495:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1531:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.271499:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1531:0x0](ffff9181c79df448) 00000080:00200000:2.0:1787826191.271520:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1532:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.271524:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.271531:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1532:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.271535:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1532:0x0](ffff9181c7b0a1c8) 00000080:00200000:2.0:1787826191.271558:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1533:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.271562:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.271568:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1533:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.271572:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1533:0x0](ffff9181c79d63c8) 00000080:00200000:2.0:1787826191.271594:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1534:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.271599:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.271606:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1534:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.271611:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1534:0x0](ffff9181c79f00c8) 00000080:00200000:2.0:1787826191.271635:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1535:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.271641:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.271648:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1535:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.271652:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1535:0x0](ffff9181c79eaa08) 00000080:00200000:2.0:1787826191.271679:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1536:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.271684:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.271691:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1536:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.271695:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1536:0x0](ffff9181c79ecb08) 00000080:00200000:2.0:1787826191.272068:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1537:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.272074:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.272083:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1537:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.272087:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1537:0x0](ffff9181c79da1c8) 00000080:00200000:2.0:1787826191.272112:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1538:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.272117:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.272124:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1538:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.272127:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1538:0x0](ffff9181c79d9988) 00000080:00200000:2.0:1787826191.272162:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1539:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.272167:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.272174:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1539:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.272178:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1539:0x0](ffff9181c79ddb88) 00000080:00200000:2.0:1787826191.272202:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x153a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.272206:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.272213:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x153a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.272217:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x153a:0x0](ffff9181c79f1148) 00000080:00200000:2.0:1787826191.272240:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x153b:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.272244:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.272249:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x153b:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.272253:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x153b:0x0](ffff9181c79dba88) 00000080:00200000:2.0:1787826191.272273:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x153c:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.272278:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.272284:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x153c:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.272288:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x153c:0x0](ffff9181c79e8908) 00000080:00200000:2.0:1787826191.272309:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x153d:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.272313:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.272320:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x153d:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.272323:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x153d:0x0](ffff9181c79fec08) 00000080:00200000:2.0:1787826191.272345:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x153e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.272350:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.272356:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x153e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.272360:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x153e:0x0](ffff9181c79f63c8) 00000080:00200000:2.0:1787826191.272380:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x153f:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.272384:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.272389:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x153f:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.272392:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x153f:0x0](ffff9181c79f80c8) 00000080:00200000:2.0:1787826191.272409:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1540:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.272413:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.272419:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1540:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.272423:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1540:0x0](ffff9181c79f9148) 00000080:00200000:2.0:1787826191.272443:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1541:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.272447:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.272453:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1541:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.272456:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1541:0x0](ffff9181c79d7448) 00000080:00200000:2.0:1787826191.272476:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1542:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.272480:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.272487:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1542:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.272491:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1542:0x0](ffff9181c79f7448) 00000080:00200000:2.0:1787826191.272512:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1543:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.272517:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.272524:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1543:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.272528:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1543:0x0](ffff9181c79fe3c8) 00000080:00200000:2.0:1787826191.272548:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1544:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.272553:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.272560:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1544:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.272563:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1544:0x0](ffff9181c79ff448) 00000080:00200000:2.0:1787826191.272585:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1545:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.272590:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.272597:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1545:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.272600:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1545:0x0](ffff9181c79fa1c8) 00000080:00200000:2.0:1787826191.272622:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1546:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.272627:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.272634:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1546:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.272638:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1546:0x0](ffff9181c79f9988) 00000080:00200000:2.0:1787826191.272661:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1547:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.272666:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.272674:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1547:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.272678:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1547:0x0](ffff9181c79d21c8) 00000080:00200000:2.0:1787826191.272714:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1548:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.272720:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.272727:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1548:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.272731:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1548:0x0](ffff9181c79d1988) 00000080:00200000:2.0:1787826191.273207:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1549:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.273214:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.273225:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1549:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.273229:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1549:0x0](ffff9181c79fdb88) 00000080:00200000:2.0:1787826191.273253:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x154a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.273259:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.273265:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x154a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.273269:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x154a:0x0](ffff9181c79fba88) 00000080:00200000:2.0:1787826191.273290:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x154b:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.273296:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.273303:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x154b:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.273307:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x154b:0x0](ffff9181c79f21c8) 00000080:00200000:2.0:1787826191.273330:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x154c:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.273336:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.273343:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x154c:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.273346:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x154c:0x0](ffff9181c79d5b88) 00000080:00200000:2.0:1787826191.273370:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x154d:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.273375:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.273381:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x154d:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.273385:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x154d:0x0](ffff9181c79d3a88) 00000080:00200000:2.0:1787826191.273405:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x154e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.273410:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.273415:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x154e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.273418:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x154e:0x0](ffff9181c79fb248) 00000080:00200000:2.0:1787826191.273441:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x154f:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.273446:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.273451:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x154f:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.273455:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x154f:0x0](ffff9181c79faa08) 00000080:00200000:2.0:1787826191.273473:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1550:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.273478:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.273484:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1550:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.273488:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1550:0x0](ffff9181c79f1988) 00000080:00200000:2.0:1787826191.273508:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1551:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.273512:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.273517:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1551:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.273521:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1551:0x0](ffff9181c79d3248) 00000080:00200000:2.0:1787826191.273539:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1552:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.273543:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.273548:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1552:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.273552:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1552:0x0](ffff9181c79d2a08) 00000080:00200000:2.0:1787826191.273574:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1553:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.273579:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.273586:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1553:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.273590:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1553:0x0](ffff9181c79fcb08) 00000080:00200000:2.0:1787826191.273611:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1554:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.273616:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.273622:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1554:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.273626:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1554:0x0](ffff9181c79f8908) 00000080:00200000:2.0:1787826191.273647:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1555:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.273651:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.273657:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1555:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.273661:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1555:0x0](ffff9181c79fc2c8) 00000080:00200000:2.0:1787826191.273682:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1556:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.273686:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.273693:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1556:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.273696:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1556:0x0](ffff9181c79f5b88) 00000080:00200000:2.0:1787826191.275258:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1557:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.275273:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.275292:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1557:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.275298:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1557:0x0](ffff9181c79fd348) 00000080:00200000:2.0:1787826191.275330:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1558:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.275335:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.275343:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1558:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.275346:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1558:0x0](ffff9181c7942a08) 00000080:00200000:2.0:1787826191.275376:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1559:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.275381:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.275389:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1559:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.275394:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1559:0x0](ffff9181c7944b08) 00000080:00200000:2.0:1787826191.275435:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x155a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.275441:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.275450:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x155a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.275454:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x155a:0x0](ffff9181c79ec2c8) 00000080:00200000:2.0:1787826191.275486:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x155b:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.275490:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.275497:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x155b:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.275501:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x155b:0x0](ffff9181c7940908) 00000080:00200000:2.0:1787826191.275523:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x155c:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.275527:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.275534:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x155c:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.275537:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x155c:0x0](ffff9181c79f3a88) 00000080:00200000:2.0:1787826191.275564:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x155d:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.275569:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.275575:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x155d:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.275579:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x155d:0x0](ffff9181c79d4b08) 00000080:00200000:2.0:1787826191.275608:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x155e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.275613:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.275619:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x155e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.275623:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x155e:0x0](ffff9181c79d0908) 00000080:00200000:2.0:1787826191.275643:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x155f:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.275647:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.275654:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x155f:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.275659:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x155f:0x0](ffff9181c79442c8) 00000080:00200000:2.0:1787826191.275678:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1560:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.275683:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.275690:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1560:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.275693:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1560:0x0](ffff9181c79f3248) 00000080:00200000:2.0:1787826191.275736:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1561:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.276019:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.276033:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1561:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.276037:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1561:0x0](ffff9181c7945348) 00000080:00200000:2.0:1787826191.276065:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1562:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.276070:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.276077:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1562:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.276080:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1562:0x0](ffff9181c7946c08) 00000080:00200000:2.0:1787826191.276103:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1563:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.276109:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.276115:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1563:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.276119:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1563:0x0](ffff9181c79d42c8) 00000080:00200000:2.0:1787826191.276141:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1564:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.276145:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.276151:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1564:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.276155:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1564:0x0](ffff9181c79ed348) 00000080:00200000:2.0:1787826191.276179:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1565:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.276183:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.276189:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1565:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.276192:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1565:0x0](ffff9181c79400c8) 00000080:00200000:2.0:1787826191.276214:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1566:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.276218:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.276224:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1566:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.276228:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1566:0x0](ffff9181c79f2a08) 00000080:00200000:2.0:1787826191.276250:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1567:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.276256:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.276264:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1567:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.276268:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1567:0x0](ffff9181c7941148) 00000080:00200000:2.0:1787826191.276288:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1568:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.276293:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.276300:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1568:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.276304:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1568:0x0](ffff9181c79463c8) 00000080:00200000:2.0:1787826191.276325:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1569:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.276330:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.276336:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1569:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.276340:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1569:0x0](ffff9181c79eec08) 00000080:00200000:2.0:1787826191.276363:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x156a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.276368:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.276375:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x156a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.276378:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x156a:0x0](ffff9181c79e80c8) 00000080:00200000:2.0:1787826191.276402:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x156b:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.276407:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.276415:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x156b:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.276419:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x156b:0x0](ffff9181c79e9148) 00000080:00200000:2.0:1787826191.276443:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x156c:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.276447:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.276454:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x156c:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.276458:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x156c:0x0](ffff9181c79ee3c8) 00000080:00200000:2.0:1787826191.276480:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x156d:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.276485:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.276492:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x156d:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.276496:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x156d:0x0](ffff9181c79f4b08) 00000080:00200000:2.0:1787826191.276517:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x156e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.276521:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.276527:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x156e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.276531:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x156e:0x0](ffff9181c7947448) 00000080:00200000:2.0:1787826191.276551:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x156f:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.276557:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.276563:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x156f:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.276567:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x156f:0x0](ffff9181c79ef448) 00000080:00200000:2.0:1787826191.276592:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1570:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.276597:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.276604:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1570:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.276608:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1570:0x0](ffff9181c79421c8) 00000080:00200000:2.0:1787826191.276631:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1571:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.276636:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.276643:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1571:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.276647:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1571:0x0](ffff9181c79f0908) 00000080:00200000:2.0:1787826191.276671:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1572:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.276675:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.276682:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1572:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.276685:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1572:0x0](ffff9181c79f42c8) 00000080:00200000:0.0:1787826191.280350:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1573:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.280375:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.280416:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1573:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.280430:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1573:0x0](ffff9181c79f5348) 00000080:00200000:0.0:1787826191.280566:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1574:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.280572:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.280587:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1574:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.280596:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1574:0x0](ffff9181c79f6c08) 00000080:00200000:0.0:1787826191.280676:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1575:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.280683:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.280695:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1575:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.280700:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1575:0x0](ffff9181c79ea1c8) 00000080:00200000:0.0:1787826191.280766:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1576:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.280772:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.280783:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1576:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.280788:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1576:0x0](ffff9181c7941988) 00000080:00200000:0.0:1787826191.280829:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1577:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.280836:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.280844:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1577:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.280863:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1577:0x0](ffff9181c7945b88) 00000080:00200000:0.0:1787826191.281688:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1578:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.281699:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.281713:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1578:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.281718:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1578:0x0](ffff9181c794f448) 00000080:00200000:0.0:1787826191.281783:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1579:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.281788:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.281795:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1579:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.281799:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1579:0x0](ffff9181c7943a88) 00000080:00200000:0.0:1787826191.281825:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x157a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.281829:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.281836:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x157a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.281840:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x157a:0x0](ffff9181c7943248) 00000080:00200000:0.0:1787826191.281873:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x157b:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.281878:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.281886:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x157b:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.281890:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x157b:0x0](ffff9181c794a1c8) 00000080:00200000:0.0:1787826191.281918:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x157c:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.281923:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.281931:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x157c:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.281935:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x157c:0x0](ffff9181c7951148) 00000080:00200000:0.0:1787826191.281958:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x157d:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.281963:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.281971:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x157d:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.281975:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x157d:0x0](ffff9181c79563c8) 00000080:00200000:0.0:1787826191.282002:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x157e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.282008:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.282016:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x157e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.282020:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x157e:0x0](ffff9181c7957448) 00000080:00200000:0.0:1787826191.282059:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x157f:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.282065:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.282073:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x157f:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.282077:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x157f:0x0](ffff9181c79521c8) 00000080:00200000:0.0:1787826191.282102:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1580:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.282107:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.282113:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1580:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.282118:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1580:0x0](ffff9181c7951988) 00000080:00200000:0.0:1787826191.282141:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1581:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.282145:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.282151:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1581:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.282155:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1581:0x0](ffff9181c7955b88) 00000080:00200000:0.0:1787826191.282176:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1582:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.282182:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.282190:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1582:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.282194:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1582:0x0](ffff9181c79580c8) 00000080:00200000:0.0:1787826191.282228:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1583:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.282234:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.282241:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1583:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.282245:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1583:0x0](ffff9181c7953a88) 00000080:00200000:0.0:1787826191.282267:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1584:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.282272:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.282279:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1584:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.282282:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1584:0x0](ffff9181c7949988) 00000080:00200000:0.0:1787826191.282304:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1585:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.282309:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.282315:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1585:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.282319:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1585:0x0](ffff9181c7953248) 00000080:00200000:0.0:1787826191.282341:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1586:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.282346:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.282353:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1586:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.282357:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1586:0x0](ffff9181c794db88) 00000080:00200000:0.0:1787826191.282377:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1587:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.282382:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.282388:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1587:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.282393:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1587:0x0](ffff9181c7960908) 00000080:00200000:0.0:1787826191.282429:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1588:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.282434:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.282440:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1588:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.282444:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1588:0x0](ffff9181c794ba88) 00000080:00200000:0.0:1787826191.282465:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1589:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.282470:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.282477:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1589:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.282481:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1589:0x0](ffff9181c79642c8) 00000080:00200000:0.0:1787826191.282502:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x158a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.282507:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.282513:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x158a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.282517:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x158a:0x0](ffff9181c7952a08) 00000080:00200000:0.0:1787826191.282539:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x158b:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.282544:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.282550:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x158b:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.282555:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x158b:0x0](ffff9181c794b248) 00000080:00200000:0.0:1787826191.282575:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x158c:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.282580:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.282587:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x158c:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.282591:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x158c:0x0](ffff9181c7954b08) 00000080:00200000:0.0:1787826191.282616:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x158d:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.282622:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.282629:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x158d:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.282633:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x158d:0x0](ffff9181c794aa08) 00000080:00200000:0.0:1787826191.282657:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x158e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.282662:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.282669:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x158e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.282674:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x158e:0x0](ffff9181c794cb08) 00000080:00200000:0.0:1787826191.282697:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x158f:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.282703:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.282710:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x158f:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.282714:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x158f:0x0](ffff9181c7948908) 00000080:00200000:0.0:1787826191.282735:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1590:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.282740:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.282775:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1590:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.282780:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1590:0x0](ffff9181c7959148) 00000080:00200000:0.0:1787826191.282810:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1591:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.282815:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.282837:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1591:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.282842:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1591:0x0](ffff9181c7965348) 00000080:00200000:0.0:1787826191.282876:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1592:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.282881:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.282889:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1592:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.282893:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1592:0x0](ffff9181c7966c08) 00000080:00200000:0.0:1787826191.282916:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1593:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.282921:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.282927:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1593:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.282931:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1593:0x0](ffff9181c7950908) 00000080:00200000:0.0:1787826191.282955:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1594:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.282960:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.282967:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1594:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.282970:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1594:0x0](ffff9181c794c2c8) 00000080:00200000:0.0:1787826191.282994:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1595:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.282999:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.283005:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1595:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.283010:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1595:0x0](ffff9181c79542c8) 00000080:00200000:0.0:1787826191.283032:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1596:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.283037:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.283044:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1596:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.283048:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1596:0x0](ffff9181c79600c8) 00000080:00200000:0.0:1787826191.283073:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1597:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.283078:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.283085:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1597:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.283090:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1597:0x0](ffff9181c7955348) 00000080:00200000:0.0:1787826191.283112:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1598:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.283117:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.283124:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1598:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.283128:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1598:0x0](ffff9181c7961148) 00000080:00200000:0.0:1787826191.283153:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1599:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.283158:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.283165:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1599:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.283169:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1599:0x0](ffff9181c794d348) 00000080:00200000:0.0:1787826191.283193:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x159a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.283199:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.283205:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x159a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.283208:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x159a:0x0](ffff9181c7956c08) 00000080:00200000:0.0:1787826191.283231:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x159b:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.283236:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.283243:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x159b:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.283247:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x159b:0x0](ffff9181c79500c8) 00000080:00200000:0.0:1787826191.283267:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x159c:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.283271:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.283278:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x159c:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.283282:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x159c:0x0](ffff9181c796b248) 00000080:00200000:0.0:1787826191.283302:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x159d:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.283307:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.283313:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x159d:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.283317:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x159d:0x0](ffff9181c79663c8) 00000080:00200000:0.0:1787826191.283337:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x159e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.283342:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.283349:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x159e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.283353:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x159e:0x0](ffff9181c796aa08) 00000080:00200000:0.0:1787826191.283377:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x159f:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.283381:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.283388:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x159f:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.283392:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x159f:0x0](ffff9181c7967448) 00000080:00200000:0.0:1787826191.283415:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15a0:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.283420:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.283428:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15a0:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.283433:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15a0:0x0](ffff9181c794ec08) 00000080:00200000:0.0:1787826191.283453:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15a1:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.283459:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.283465:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15a1:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.283469:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15a1:0x0](ffff9181c79480c8) 00000080:00200000:0.0:1787826191.283490:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15a2:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.283496:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.283503:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15a2:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.283507:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15a2:0x0](ffff9181c796cb08) 00000080:00200000:0.0:1787826191.283528:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15a3:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.283533:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.283541:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15a3:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.283545:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15a3:0x0](ffff9181c7949148) 00000080:00200000:0.0:1787826191.283568:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15a4:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.283573:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.283580:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15a4:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.283585:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15a4:0x0](ffff9181c795e3c8) 00000080:00200000:0.0:1787826191.283611:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15a5:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.283617:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.283624:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15a5:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.283628:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15a5:0x0](ffff9181c794e3c8) 00000080:00200000:0.0:1787826191.283650:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15a6:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.283655:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.283663:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15a6:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.283668:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15a6:0x0](ffff9181c7968908) 00000080:00200000:0.0:1787826191.283691:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15a7:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.283695:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.283702:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15a7:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.283707:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15a7:0x0](ffff9181c796c2c8) 00000080:00200000:0.0:1787826191.283730:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15a8:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.283736:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.283770:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15a8:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.283775:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15a8:0x0](ffff9181c79621c8) 00000080:00200000:0.0:1787826191.283801:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15a9:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.283806:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.283815:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15a9:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.283819:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15a9:0x0](ffff9181c795f448) 00000080:00200000:0.0:1787826191.292312:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15aa:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.292474:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.292679:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15aa:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.292692:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15aa:0x0](ffff9181c7975348) 00000080:00200000:0.0:1787826191.293128:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15ab:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.293134:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.293147:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15ab:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.293153:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15ab:0x0](ffff9181c7961988) 00000080:00200000:0.0:1787826191.293447:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15ac:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.293454:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.293463:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15ac:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.293468:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15ac:0x0](ffff9181c7976c08) 00000080:00200000:0.0:1787826191.293496:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15ad:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.293501:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.293797:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15ad:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.293803:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15ad:0x0](ffff9181c796d348) 00000080:00200000:0.0:1787826191.293856:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15ae:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.293862:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.293875:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15ae:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.293880:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15ae:0x0](ffff9181c796ec08) 00000080:00200000:1.0:1787826191.294100:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15af:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.294115:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.294138:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15af:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.294159:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15af:0x0](ffff9181c79680c8) 00000080:00200000:1.0:1787826191.294194:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15b0:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.294199:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.294207:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15b0:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.294211:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15b0:0x0](ffff9181c7969148) 00000080:00200000:1.0:1787826191.294231:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15b1:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.294236:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.294242:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15b1:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.294246:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15b1:0x0](ffff9181c79700c8) 00000080:00200000:1.0:1787826191.294271:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15b2:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.294276:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.294284:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15b2:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.294287:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15b2:0x0](ffff9181c7965b88) 00000080:00200000:1.0:1787826191.294314:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15b3:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.294319:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.294325:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15b3:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.294329:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15b3:0x0](ffff9181c7971148) 00000080:00200000:1.0:1787826191.294349:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15b4:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.294354:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.294361:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15b4:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.294365:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15b4:0x0](ffff9181c796e3c8) 00000080:00200000:1.0:1787826191.294385:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15b5:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.294391:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.294398:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15b5:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.294401:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15b5:0x0](ffff9181c796f448) 00000080:00200000:1.0:1787826191.294423:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15b6:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.294427:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.294435:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15b6:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.294439:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15b6:0x0](ffff9181c7963a88) 00000080:00200000:1.0:1787826191.294464:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15b7:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.294469:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.294476:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15b7:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.294479:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15b7:0x0](ffff9181c796a1c8) 00000080:00200000:1.0:1787826191.294504:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15b8:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.294508:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.294515:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15b8:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.294519:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15b8:0x0](ffff9181c7969988) 00000080:00200000:1.0:1787826191.294541:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15b9:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.294546:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.294553:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15b9:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.294556:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15b9:0x0](ffff9181c79763c8) 00000080:00200000:1.0:1787826191.294578:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15ba:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.294582:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.294589:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15ba:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.294593:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15ba:0x0](ffff9181c796db88) 00000080:00200000:1.0:1787826191.294616:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15bb:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.294621:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.294628:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15bb:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.294632:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15bb:0x0](ffff9181c795a1c8) 00000080:00200000:1.0:1787826191.294659:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15bc:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.294664:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.294670:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15bc:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.294674:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15bc:0x0](ffff9181c796ba88) 00000080:00200000:1.0:1787826191.294709:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15bd:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.294716:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.294723:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15bd:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.294727:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15bd:0x0](ffff9181c7977448) 00000080:00200000:1.0:1787826191.294791:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15be:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.294797:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.294806:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15be:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.294809:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15be:0x0](ffff9181c7963248) 00000080:00200000:1.0:1787826191.294831:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15bf:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.294835:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.294841:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15bf:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.294856:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15bf:0x0](ffff9181c79721c8) 00000080:00200000:1.0:1787826191.294879:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15c0:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.294884:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.294891:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15c0:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.294894:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15c0:0x0](ffff9181c797c2c8) 00000080:00200000:1.0:1787826191.294917:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15c1:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.294921:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.294929:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15c1:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.294933:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15c1:0x0](ffff9181c7959988) 00000080:00200000:1.0:1787826191.294956:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15c2:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.294961:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.294968:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15c2:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.294973:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15c2:0x0](ffff9181c795db88) 00000080:00200000:1.0:1787826191.294996:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15c3:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.295000:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.295007:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15c3:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.295011:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15c3:0x0](ffff9181c795ba88) 00000080:00200000:1.0:1787826191.295034:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15c4:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.295039:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.295045:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15c4:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.295049:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15c4:0x0](ffff9181c7971988) 00000080:00200000:1.0:1787826191.295071:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15c5:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.295075:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.295082:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15c5:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.295085:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15c5:0x0](ffff9181c7975b88) 00000080:00200000:1.0:1787826191.295110:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15c6:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.295115:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.295122:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15c6:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.295126:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15c6:0x0](ffff9181c7962a08) 00000080:00200000:1.0:1787826191.295149:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15c7:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.295154:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.295160:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15c7:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.295164:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15c7:0x0](ffff9181c7964b08) 00000080:00200000:1.0:1787826191.295185:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15c8:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.295190:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.295197:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15c8:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.295201:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15c8:0x0](ffff9181c7973a88) 00000080:00200000:1.0:1787826191.295222:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15c9:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.295227:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.295233:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15c9:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.295237:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15c9:0x0](ffff9181c7973248) 00000080:00200000:1.0:1787826191.295257:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15ca:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.295261:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.295267:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15ca:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.295271:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15ca:0x0](ffff9181c78c7448) 00000080:00200000:1.0:1787826191.295291:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15cb:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.295295:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.295302:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15cb:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.295306:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15cb:0x0](ffff9181c78c21c8) 00000080:00200000:1.0:1787826191.295328:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15cc:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.295333:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.295340:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15cc:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.295343:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15cc:0x0](ffff9181c7972a08) 00000080:00200000:1.0:1787826191.295364:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15cd:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.295369:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.295375:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15cd:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.295379:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15cd:0x0](ffff9181c797d348) 00000080:00200000:1.0:1787826191.295401:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15ce:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.295406:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.295412:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15ce:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.295416:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15ce:0x0](ffff9181c78c1988) 00000080:00200000:1.0:1787826191.295436:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15cf:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.295441:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.295447:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15cf:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.295452:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15cf:0x0](ffff9181c7974b08) 00000080:00200000:1.0:1787826191.295472:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15d0:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.295476:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.295483:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15d0:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.295487:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15d0:0x0](ffff9181c78c5b88) 00000080:00200000:1.0:1787826191.295506:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15d1:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.295510:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.295516:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15d1:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.295520:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15d1:0x0](ffff9181c795b248) 00000080:00200000:1.0:1787826191.295540:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15d2:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.295544:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.295550:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15d2:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.295554:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15d2:0x0](ffff9181c7970908) 00000080:00200000:1.0:1787826191.295574:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15d3:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.295579:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.295586:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15d3:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.295589:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15d3:0x0](ffff9181c797ec08) 00000080:00200000:1.0:1787826191.295612:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15d4:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.295616:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.295622:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15d4:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.295626:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15d4:0x0](ffff9181c78c3a88) 00000080:00200000:1.0:1787826191.295647:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15d5:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.295651:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.295658:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15d5:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.295662:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15d5:0x0](ffff9181c78c3248) 00000080:00200000:1.0:1787826191.295684:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15d6:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.295689:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.295695:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15d6:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.295700:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15d6:0x0](ffff9181c795aa08) 00000080:00200000:1.0:1787826191.295722:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15d7:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.295726:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.295733:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15d7:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.295737:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15d7:0x0](ffff9181c795cb08) 00000080:00200000:1.0:1787826191.295816:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15d8:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.295822:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.295829:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15d8:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.295834:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15d8:0x0](ffff9181c78c2a08) 00000080:00200000:1.0:1787826191.295867:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15d9:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.295872:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.295879:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15d9:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.295883:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15d9:0x0](ffff9181c79742c8) 00000080:00200000:1.0:1787826191.295902:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15da:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.295906:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.295912:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15da:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.295916:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15da:0x0](ffff9181c79780c8) 00000080:00200000:1.0:1787826191.295936:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15db:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.295940:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.295947:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15db:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.295951:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15db:0x0](ffff9181c78c9148) 00000080:00200000:1.0:1787826191.295971:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15dc:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.295976:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.295983:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15dc:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.295987:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15dc:0x0](ffff9181c7958908) 00000080:00200000:1.0:1787826191.296007:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15dd:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.296011:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.296018:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15dd:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.296022:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15dd:0x0](ffff9181c78ce3c8) 00000080:00200000:1.0:1787826191.296043:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15de:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.296048:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.296054:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15de:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.296059:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15de:0x0](ffff9181c7979148) 00000080:00200000:1.0:1787826191.296082:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15df:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.296086:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.296093:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15df:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.296097:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15df:0x0](ffff9181c797e3c8) 00000080:00200000:1.0:1787826191.296119:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15e0:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.296123:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.296130:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15e0:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.296135:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15e0:0x0](ffff9181c78c4b08) 00000080:00200000:1.0:1787826191.296156:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15e1:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.296160:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.296167:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15e1:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.296171:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15e1:0x0](ffff9181c797f448) 00000080:00200000:1.0:1787826191.296193:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15e2:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.296197:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.296204:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15e2:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.296207:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15e2:0x0](ffff9181c797a1c8) 00000080:00200000:1.0:1787826191.296227:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15e3:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.296231:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.296238:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15e3:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.296242:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15e3:0x0](ffff9181c7979988) 00000080:00200000:1.0:1787826191.296262:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15e4:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.296266:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.296272:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15e4:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.296277:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15e4:0x0](ffff9181c797db88) 00000080:00200000:1.0:1787826191.296297:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15e5:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.296301:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.296307:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15e5:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.296311:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15e5:0x0](ffff9181c78cf448) 00000080:00200000:1.0:1787826191.296333:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15e6:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.296337:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.296343:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15e6:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.296346:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15e6:0x0](ffff9181c78ca1c8) 00000080:00200000:1.0:1787826191.296366:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15e7:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.296370:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.296376:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15e7:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.296380:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15e7:0x0](ffff9181c78c9988) 00000080:00200000:1.0:1787826191.296399:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15e8:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.296403:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.296410:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15e8:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.296413:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15e8:0x0](ffff9181c78cdb88) 00000080:00200000:1.0:1787826191.296432:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15e9:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.296437:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.296443:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15e9:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.296446:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15e9:0x0](ffff9181c795c2c8) 00000080:00200000:1.0:1787826191.296465:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15ea:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.296469:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.296477:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15ea:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.296480:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15ea:0x0](ffff9181c78cba88) 00000080:00200000:1.0:1787826191.296502:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15eb:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.296506:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.296512:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15eb:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.296516:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15eb:0x0](ffff9181c797ba88) 00000080:00200000:1.0:1787826191.296535:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15ec:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.296539:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.296546:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15ec:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.296550:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15ec:0x0](ffff9181c797b248) 00000080:00200000:1.0:1787826191.296570:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15ed:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.296575:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.296582:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15ed:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.296586:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15ed:0x0](ffff9181c78c0908) 00000080:00200000:1.0:1787826191.296606:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15ee:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.296611:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.296618:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15ee:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.296623:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15ee:0x0](ffff9181c78c42c8) 00000080:00200000:1.0:1787826191.296644:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15ef:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.296649:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.296656:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15ef:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.296660:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15ef:0x0](ffff9181c78c5348) 00000080:00200000:1.0:1787826191.296682:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15f0:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.296686:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.296693:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15f0:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.296697:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15f0:0x0](ffff9181c78c6c08) 00000080:00200000:1.0:1787826191.296720:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15f1:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.296725:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.296731:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15f1:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.296735:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15f1:0x0](ffff9181c795d348) 00000080:00200000:1.0:1787826191.296802:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15f2:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.296807:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.296815:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15f2:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.296819:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15f2:0x0](ffff9181c78cb248) 00000080:00200000:1.0:1787826191.296841:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15f3:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.298226:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.298251:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15f3:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.298257:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15f3:0x0](ffff9181c78caa08) 00000080:00200000:1.0:1787826191.298305:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15f4:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.298311:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.298319:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15f4:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.298323:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15f4:0x0](ffff9181c78ccb08) 00000080:00200000:1.0:1787826191.298347:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15f5:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.298352:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.298359:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15f5:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.298363:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15f5:0x0](ffff9181c78c8908) 00000080:00200000:1.0:1787826191.298387:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15f6:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.298392:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.298399:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15f6:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.298402:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15f6:0x0](ffff9181c78cc2c8) 00000080:00200000:1.0:1787826191.298425:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15f7:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.298430:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.298437:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15f7:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.298441:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15f7:0x0](ffff9181c78cd348) 00000080:00200000:1.0:1787826191.298461:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15f8:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.298466:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.298472:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15f8:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.298475:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15f8:0x0](ffff9181c78c00c8) 00000080:00200000:1.0:1787826191.298498:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15f9:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.298502:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.298510:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15f9:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.298513:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15f9:0x0](ffff9181c795ec08) 00000080:00200000:1.0:1787826191.298540:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15fa:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.298545:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.298552:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15fa:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.298556:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15fa:0x0](ffff9181c78cec08) 00000080:00200000:1.0:1787826191.298579:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15fb:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.298584:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.298591:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15fb:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.298595:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15fb:0x0](ffff9181c78c80c8) 00000080:00200000:1.0:1787826191.298616:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15fc:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.298621:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.298628:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15fc:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.298632:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15fc:0x0](ffff9181c78d00c8) 00000080:00200000:1.0:1787826191.298654:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15fd:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.298659:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.298667:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15fd:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.298670:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15fd:0x0](ffff9181c797aa08) 00000080:00200000:1.0:1787826191.298694:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15fe:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.303287:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.303334:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15fe:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.303344:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15fe:0x0](ffff9181c78de3c8) 00000080:00200000:1.0:1787826191.303409:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x15ff:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.303417:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.303426:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x15ff:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.303430:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x15ff:0x0](ffff9181c797cb08) 00000080:00200000:1.0:1787826191.303456:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1600:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.303461:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.303467:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1600:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.303471:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1600:0x0](ffff9181c78d1148) 00000080:00200000:1.0:1787826191.303498:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1601:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.303502:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.303507:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1601:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.303511:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1601:0x0](ffff9181c78d63c8) 00000080:00200000:1.0:1787826191.303532:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1602:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.303537:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.303543:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1602:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.303547:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1602:0x0](ffff9181c7978908) 00000080:00200000:1.0:1787826191.303648:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1603:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.303653:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.303661:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1603:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.303666:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1603:0x0](ffff9181c78e3248) 00000080:00200000:1.0:1787826191.303690:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1604:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.303695:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.304036:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1604:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.304042:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1604:0x0](ffff9181c78e2a08) 00000080:00200000:1.0:1787826191.304070:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1605:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.304076:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.304084:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1605:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.304089:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1605:0x0](ffff9181c78c1148) 00000080:00200000:1.0:1787826191.304122:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1606:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.304128:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.304136:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1606:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.304140:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1606:0x0](ffff9181c78c63c8) 00000080:00200000:1.0:1787826191.304183:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1607:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.304187:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.304194:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1607:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.304199:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1607:0x0](ffff9181c78e9988) 00000080:00200000:1.0:1787826191.304220:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1608:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.304224:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.304232:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1608:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.304236:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1608:0x0](ffff9181c78edb88) 00000080:00200000:1.0:1787826191.304259:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1609:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.304264:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.304271:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1609:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.304276:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1609:0x0](ffff9181c78e4b08) 00000080:00200000:1.0:1787826191.304299:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x160a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.304304:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.304311:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x160a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.304315:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x160a:0x0](ffff9181c78d7448) 00000080:00200000:1.0:1787826191.304336:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x160b:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.304340:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.304346:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x160b:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.304350:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x160b:0x0](ffff9181c78e0908) 00000080:00200000:1.0:1787826191.304368:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x160c:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.304372:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.304379:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x160c:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.304382:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x160c:0x0](ffff9181c78e42c8) 00000080:00200000:1.0:1787826191.304401:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x160d:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.304405:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.304411:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x160d:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.304415:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x160d:0x0](ffff9181c78d21c8) 00000080:00200000:1.0:1787826191.304435:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x160e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.304439:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.304446:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x160e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.304449:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x160e:0x0](ffff9181c78e5348) 00000080:00200000:1.0:1787826191.304468:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x160f:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.304472:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.304479:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x160f:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.304484:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x160f:0x0](ffff9181c78e6c08) 00000080:00200000:1.0:1787826191.304505:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1610:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.304509:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.304516:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1610:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.304521:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1610:0x0](ffff9181c78e00c8) 00000080:00200000:1.0:1787826191.304542:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1611:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.304546:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.304553:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1611:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.304557:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1611:0x0](ffff9181c78e1148) 00000080:00200000:1.0:1787826191.304578:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1612:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.304583:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.304590:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1612:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.304594:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1612:0x0](ffff9181c78d1988) 00000080:00200000:1.0:1787826191.304616:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1613:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.304621:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.304627:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1613:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.304632:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1613:0x0](ffff9181c78e63c8) 00000080:00200000:1.0:1787826191.304653:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1614:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.304658:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.304665:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1614:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.304669:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1614:0x0](ffff9181c78e7448) 00000080:00200000:1.0:1787826191.304712:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1615:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.304717:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.304725:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1615:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.304730:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1615:0x0](ffff9181c78d5b88) 00000080:00200000:1.0:1787826191.304774:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1616:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.304779:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.304786:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1616:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.304790:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1616:0x0](ffff9181c78eba88) 00000080:00200000:1.0:1787826191.304810:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1617:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.304814:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.304819:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1617:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.304823:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1617:0x0](ffff9181c78d3a88) 00000080:00200000:1.0:1787826191.304840:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1618:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.304855:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.304863:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1618:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.304866:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1618:0x0](ffff9181c78e21c8) 00000080:00200000:1.0:1787826191.304890:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1619:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.304894:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.304901:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1619:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.304905:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1619:0x0](ffff9181c78e1988) 00000080:00200000:1.0:1787826191.304926:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x161a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.304931:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.304938:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x161a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.304942:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x161a:0x0](ffff9181c78d3248) 00000080:00200000:1.0:1787826191.304962:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x161b:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.304969:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.304978:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x161b:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.304983:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x161b:0x0](ffff9181c78eb248) 00000080:00200000:1.0:1787826191.305005:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x161c:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.305009:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.305016:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x161c:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.305020:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x161c:0x0](ffff9181c78e5b88) 00000080:00200000:1.0:1787826191.305043:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x161d:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.305047:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.305054:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x161d:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.305057:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x161d:0x0](ffff9181c78d2a08) 00000080:00200000:1.0:1787826191.305085:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x161e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.305090:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.305098:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x161e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.305102:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x161e:0x0](ffff9181c78eaa08) 00000080:00200000:1.0:1787826191.305125:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x161f:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.305129:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.305136:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x161f:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.305139:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x161f:0x0](ffff9181c78e3a88) 00000080:00200000:1.0:1787826191.305161:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1620:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.305165:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.305172:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1620:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.305176:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1620:0x0](ffff9181c78f1988) 00000080:00200000:1.0:1787826191.305198:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1621:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.305203:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.305209:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1621:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.305213:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1621:0x0](ffff9181c78d4b08) 00000080:00200000:1.0:1787826191.305234:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1622:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.305240:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.305247:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1622:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.305250:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1622:0x0](ffff9181c78df448) 00000080:00200000:1.0:1787826191.305274:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1623:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.305278:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.305285:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1623:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.305289:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1623:0x0](ffff9181c78d0908) 00000080:00200000:1.0:1787826191.305310:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1624:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.305315:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.305322:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1624:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.305326:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1624:0x0](ffff9181c78d42c8) 00000080:00200000:1.0:1787826191.305347:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1625:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.305352:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.305359:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1625:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.305363:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1625:0x0](ffff9181c78f5b88) 00000080:00200000:1.0:1787826191.305384:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1626:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.305388:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.305395:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1626:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.305399:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1626:0x0](ffff9181c78da1c8) 00000080:00200000:1.0:1787826191.305421:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1627:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.305425:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.305431:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1627:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.305435:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1627:0x0](ffff9181c78f3a88) 00000080:00200000:1.0:1787826191.305454:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1628:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.305459:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.305465:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1628:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.305469:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1628:0x0](ffff9181c78f3248) 00000080:00200000:1.0:1787826191.305490:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1629:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.305494:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.305501:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1629:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.305505:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1629:0x0](ffff9181c78ecb08) 00000080:00200000:1.0:1787826191.305527:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x162a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.305531:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.305538:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x162a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.305542:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x162a:0x0](ffff9181c78f2a08) 00000080:00200000:1.0:1787826191.305564:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x162b:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.305568:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.305574:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x162b:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.305579:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x162b:0x0](ffff9181c78e8908) 00000080:00200000:1.0:1787826191.305600:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x162c:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.305605:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.305612:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x162c:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.305616:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x162c:0x0](ffff9181c78f4b08) 00000080:00200000:1.0:1787826191.305639:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x162d:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.305644:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.305652:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x162d:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.305655:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x162d:0x0](ffff9181c78d9988) 00000080:00200000:1.0:1787826191.305678:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x162e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.305683:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.305690:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x162e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.305694:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x162e:0x0](ffff9181c78ec2c8) 00000080:00200000:1.0:1787826191.305716:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x162f:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.305721:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.305728:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x162f:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.305733:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x162f:0x0](ffff9181c78d5348) 00000080:00200000:1.0:1787826191.305780:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1630:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.305785:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.305794:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1630:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.305801:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1630:0x0](ffff9181c78d6c08) 00000080:00200000:1.0:1787826191.305827:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1631:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.305831:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.305838:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1631:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.305842:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1631:0x0](ffff9181c78ed348) 00000080:00200000:1.0:1787826191.308409:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1632:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.308427:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.308451:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1632:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.308457:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1632:0x0](ffff9181c78f0908) 00000080:00200000:1.0:1787826191.308533:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1633:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.308539:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.308547:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1633:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.308552:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1633:0x0](ffff9181c78f42c8) 00000080:00200000:1.0:1787826191.308577:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1634:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.308582:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.308589:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1634:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.308593:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1634:0x0](ffff9181c78ddb88) 00000080:00200000:1.0:1787826191.308619:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1635:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.308624:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.308631:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1635:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.308635:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1635:0x0](ffff9181c78f5348) 00000080:00200000:1.0:1787826191.308660:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1636:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.308665:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.308671:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1636:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.308675:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1636:0x0](ffff9181c78fd348) 00000080:00200000:1.0:1787826191.308697:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1637:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.308716:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.308724:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1637:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.308728:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1637:0x0](ffff9181c78f6c08) 00000080:00200000:1.0:1787826191.308768:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1638:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.308772:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.308780:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1638:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.308783:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1638:0x0](ffff9181c78f00c8) 00000080:00200000:1.0:1787826191.308806:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1639:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.308811:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.308818:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1639:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.308821:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1639:0x0](ffff9181c78eec08) 00000080:00200000:1.0:1787826191.308843:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x163a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.308857:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.308864:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x163a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.308869:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x163a:0x0](ffff9181c78e80c8) 00000080:00200000:1.0:1787826191.308894:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x163b:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.308899:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.308906:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x163b:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.308910:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x163b:0x0](ffff9181c78fec08) 00000080:00200000:1.0:1787826191.308932:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x163c:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.308936:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.308943:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x163c:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.308947:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x163c:0x0](ffff9181c78f1148) 00000080:00200000:1.0:1787826191.308969:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x163d:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.308974:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.308982:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x163d:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.308986:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x163d:0x0](ffff9181c78f63c8) 00000080:00200000:1.0:1787826191.309008:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x163e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.309013:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.309020:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x163e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.309024:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x163e:0x0](ffff9181c78e9148) 00000080:00200000:1.0:1787826191.309049:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x163f:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.309054:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.309061:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x163f:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.309064:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x163f:0x0](ffff9181c78ee3c8) 00000080:00200000:1.0:1787826191.309085:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1640:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.309090:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.309097:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1640:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.309101:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1640:0x0](ffff9181c78dba88) 00000080:00200000:1.0:1787826191.309625:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1641:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.309632:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.309642:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1641:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.309646:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1641:0x0](ffff9181c78f80c8) 00000080:00200000:1.0:1787826191.309673:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1642:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.309677:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.309685:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1642:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.309689:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1642:0x0](ffff9181c78f9148) 00000080:00200000:1.0:1787826191.309723:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1643:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.309729:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.309736:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1643:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.309740:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1643:0x0](ffff9181c78f7448) 00000080:00200000:1.0:1787826191.309787:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1644:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.309791:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.309798:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1644:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.309802:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1644:0x0](ffff9181c78ef448) 00000080:00200000:1.0:1787826191.309837:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1645:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.309842:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.309862:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1645:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.309866:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1645:0x0](ffff9181c78fe3c8) 00000080:00200000:1.0:1787826191.309889:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1646:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.309893:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.309899:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1646:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.309903:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1646:0x0](ffff9181c78ff448) 00000080:00200000:1.0:1787826191.309920:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1647:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.309924:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.309930:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1647:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.309934:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1647:0x0](ffff9181c78db248) 00000080:00200000:1.0:1787826191.309964:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1648:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.309970:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.309977:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1648:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.309982:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1648:0x0](ffff9181c78f21c8) 00000080:00200000:1.0:1787826191.310006:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1649:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.310011:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.310017:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1649:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.310021:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1649:0x0](ffff9181c78daa08) 00000080:00200000:1.0:1787826191.310043:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x164a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.310047:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.310054:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x164a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.310058:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x164a:0x0](ffff9181c78fa1c8) 00000080:00200000:1.0:1787826191.310079:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x164b:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.310083:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.310090:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x164b:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.310094:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x164b:0x0](ffff9181c78ea1c8) 00000080:00200000:1.0:1787826191.310116:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x164c:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.310120:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.310127:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x164c:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.310131:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x164c:0x0](ffff9181c6b46c08) 00000080:00200000:1.0:1787826191.310153:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x164d:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.310157:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.310163:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x164d:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.310167:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x164d:0x0](ffff9181c6b4ec08) 00000080:00200000:1.0:1787826191.310188:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x164e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.310193:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.310200:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x164e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.310204:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x164e:0x0](ffff9181c6b480c8) 00000080:00200000:1.0:1787826191.310227:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x164f:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.310230:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.310236:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x164f:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.310239:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x164f:0x0](ffff9181c6b400c8) 00000080:00200000:1.0:1787826191.310259:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1650:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.310264:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.310270:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1650:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.310273:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1650:0x0](ffff9181c6b49148) 00000080:00200000:1.0:1787826191.310295:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1651:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.310299:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.310305:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1651:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.310309:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1651:0x0](ffff9181c6b4e3c8) 00000080:00200000:1.0:1787826191.310330:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1652:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.310335:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.310341:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1652:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.310345:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1652:0x0](ffff9181c78f9988) 00000080:00200000:1.0:1787826191.310368:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1653:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.310372:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.310379:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1653:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.310382:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1653:0x0](ffff9181c6b4f448) 00000080:00200000:1.0:1787826191.310403:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1654:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.310407:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.310413:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1654:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.310417:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1654:0x0](ffff9181c6b4a1c8) 00000080:00200000:1.0:1787826191.310440:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1655:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.310444:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.310451:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1655:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.310455:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1655:0x0](ffff9181c78fdb88) 00000080:00200000:1.0:1787826191.310474:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1656:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.310478:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.310484:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1656:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.310488:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1656:0x0](ffff9181c6b49988) 00000080:00200000:1.0:1787826191.310508:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1657:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.310513:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.310519:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1657:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.310523:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1657:0x0](ffff9181c6b4db88) 00000080:00200000:1.0:1787826191.310557:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1658:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.310562:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.310570:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1658:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.310574:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1658:0x0](ffff9181c6b4ba88) 00000080:00200000:1.0:1787826191.310600:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1659:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.310605:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.310612:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1659:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.310616:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1659:0x0](ffff9181c6b41148) 00000080:00200000:1.0:1787826191.310638:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x165a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.310643:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.310650:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x165a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.310654:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x165a:0x0](ffff9181c6b4b248) 00000080:00200000:1.0:1787826191.310678:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x165b:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.310683:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.310690:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x165b:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.310694:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x165b:0x0](ffff9181c6b4aa08) 00000080:00200000:1.0:1787826191.310716:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x165c:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.310720:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.310727:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x165c:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.310731:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x165c:0x0](ffff9181c6b4cb08) 00000080:00200000:1.0:1787826191.311013:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x165d:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.311018:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.311028:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x165d:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.311033:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x165d:0x0](ffff9181c6b48908) 00000080:00200000:1.0:1787826191.311060:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x165e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.311065:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.311072:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x165e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.311075:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x165e:0x0](ffff9181c78fba88) 00000080:00200000:1.0:1787826191.311098:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x165f:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.311103:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.311110:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x165f:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.311113:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x165f:0x0](ffff9181c6b4c2c8) 00000080:00200000:1.0:1787826191.311138:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1660:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.311266:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.311276:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1660:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.311280:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1660:0x0](ffff9181c6b4d348) 00000080:00200000:1.0:1787826191.311308:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1661:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.311314:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.311321:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1661:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.311325:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1661:0x0](ffff9181c6b463c8) 00000080:00200000:1.0:1787826191.311349:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1662:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.311354:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.311361:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1662:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.311365:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1662:0x0](ffff9181c78dcb08) 00000080:00200000:1.0:1787826191.311393:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1663:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.311399:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.311406:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1663:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.311410:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1663:0x0](ffff9181c78d8908) 00000080:00200000:1.0:1787826191.311431:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1664:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.311436:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.311443:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1664:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.311447:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1664:0x0](ffff9181c78fb248) 00000080:00200000:1.0:1787826191.311470:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1665:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.311474:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.311481:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1665:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.311485:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1665:0x0](ffff9181c78dc2c8) 00000080:00200000:1.0:1787826191.311506:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1666:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.311510:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.311517:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1666:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.311521:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1666:0x0](ffff9181c6b57448) 00000080:00200000:1.0:1787826191.311544:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1667:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.311549:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.311556:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1667:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.311561:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1667:0x0](ffff9181c78faa08) 00000080:00200000:1.0:1787826191.311582:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1668:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.311637:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.311645:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1668:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.311649:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1668:0x0](ffff9181c6b47448) 00000080:00200000:1.0:1787826191.311674:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1669:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.311883:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.311892:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1669:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.311897:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1669:0x0](ffff9181c6b521c8) 00000080:00200000:1.0:1787826191.311922:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x166a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.311926:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.311932:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x166a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.311936:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x166a:0x0](ffff9181c78fcb08) 00000080:00200000:1.0:1787826191.311955:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x166b:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.311959:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.311965:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x166b:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.311969:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x166b:0x0](ffff9181c6b51988) 00000080:00200000:1.0:1787826191.311987:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x166c:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.311990:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.311996:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x166c:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.312000:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x166c:0x0](ffff9181c6b421c8) 00000080:00200000:1.0:1787826191.312021:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x166d:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.312026:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.312033:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x166d:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.312037:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x166d:0x0](ffff9181c6b41988) 00000080:00200000:1.0:1787826191.312061:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x166e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.312065:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.312073:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x166e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.312077:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x166e:0x0](ffff9181c6b55b88) 00000080:00200000:1.0:1787826191.312100:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x166f:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.312105:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.312111:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x166f:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.312115:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x166f:0x0](ffff9181c78f8908) 00000080:00200000:1.0:1787826191.312135:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1670:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.312140:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.312146:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1670:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.312150:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1670:0x0](ffff9181c78dd348) 00000080:00200000:1.0:1787826191.312521:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1671:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.312527:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.312536:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1671:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.312540:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1671:0x0](ffff9181c78fc2c8) 00000080:00200000:1.0:1787826191.312564:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1672:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.312569:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.312576:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1672:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.312581:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1672:0x0](ffff9181c6b53a88) 00000080:00200000:1.0:1787826191.312604:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1673:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.312609:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.312617:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1673:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.312621:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1673:0x0](ffff9181c6b45b88) 00000080:00200000:1.0:1787826191.312643:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1674:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.312647:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.312654:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1674:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.312659:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1674:0x0](ffff9181c6b53248) 00000080:00200000:1.0:1787826191.312681:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1675:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.312685:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.312692:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1675:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.312696:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1675:0x0](ffff9181c6b43a88) 00000080:00200000:1.0:1787826191.314277:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1676:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.314292:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.314310:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1676:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.314316:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1676:0x0](ffff9181c6b5ec08) 00000080:00200000:1.0:1787826191.314345:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1677:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.314350:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.314358:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1677:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.314362:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1677:0x0](ffff9181c6b52a08) 00000080:00200000:1.0:1787826191.314423:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1678:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.314428:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.314436:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1678:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.314440:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1678:0x0](ffff9181c6b54b08) 00000080:00200000:1.0:1787826191.314491:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1679:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.314496:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.314504:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1679:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.314508:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1679:0x0](ffff9181c78dec08) 00000080:00200000:1.0:1787826191.314535:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x167a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.314539:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.314545:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x167a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.314549:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x167a:0x0](ffff9181c78d80c8) 00000080:00200000:1.0:1787826191.314570:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x167b:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.314574:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.314581:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x167b:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.314585:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x167b:0x0](ffff9181c6b50908) 00000080:00200000:1.0:1787826191.314610:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x167c:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.314614:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.314621:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x167c:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.314624:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x167c:0x0](ffff9181c78d9148) 00000080:00200000:1.0:1787826191.314647:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x167d:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.314652:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.314659:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x167d:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.314663:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x167d:0x0](ffff9181c6b580c8) 00000080:00200000:1.0:1787826191.314687:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x167e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.314692:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.314714:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x167e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.314719:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x167e:0x0](ffff9181c6b542c8) 00000080:00200000:1.0:1787826191.314930:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x167f:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.314935:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.314943:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x167f:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.314947:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x167f:0x0](ffff9181c6b59148) 00000080:00200000:1.0:1787826191.314975:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1680:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.314980:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.314987:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1680:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.314990:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1680:0x0](ffff9181c6b43248) 00000080:00200000:1.0:1787826191.315012:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1681:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.315016:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.315021:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1681:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.315025:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1681:0x0](ffff9181c6b5e3c8) 00000080:00200000:1.0:1787826191.315044:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1682:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.315049:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.315055:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1682:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.315058:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1682:0x0](ffff9181c6b5f448) 00000080:00200000:1.0:1787826191.315076:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1683:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.315080:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.315085:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1683:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.315088:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1683:0x0](ffff9181c6b5a1c8) 00000080:00200000:1.0:1787826191.315108:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1684:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.315112:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.315119:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1684:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.315122:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1684:0x0](ffff9181c6b55348) 00000080:00200000:1.0:1787826191.315143:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1685:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.315147:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.315153:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1685:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.315156:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1685:0x0](ffff9181c6b59988) 00000080:00200000:1.0:1787826191.315175:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1686:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.315179:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.315184:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1686:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.315188:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1686:0x0](ffff9181c6b42a08) 00000080:00200000:1.0:1787826191.315208:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1687:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.315212:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.315218:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1687:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.315221:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1687:0x0](ffff9181c6b56c08) 00000080:00200000:1.0:1787826191.315240:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1688:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.315243:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.315249:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1688:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.315253:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1688:0x0](ffff9181c6b5db88) 00000080:00200000:1.0:1787826191.315272:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1689:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.315276:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.315282:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1689:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.315286:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1689:0x0](ffff9181c6b642c8) 00000080:00200000:1.0:1787826191.315310:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x168a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.315316:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.315322:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x168a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.315326:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x168a:0x0](ffff9181c6b5ba88) 00000080:00200000:1.0:1787826191.315348:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x168b:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.315353:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.315358:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x168b:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.315362:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x168b:0x0](ffff9181c6b500c8) 00000080:00200000:1.0:1787826191.315381:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x168c:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.315385:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.315390:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x168c:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.315394:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x168c:0x0](ffff9181c6b44b08) 00000080:00200000:1.0:1787826191.315415:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x168d:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.315419:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.315425:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x168d:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.315428:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x168d:0x0](ffff9181c6b51148) 00000080:00200000:1.0:1787826191.315448:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x168e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.315451:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.315457:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x168e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.315460:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x168e:0x0](ffff9181c6b5b248) 00000080:00200000:1.0:1787826191.315479:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x168f:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.315483:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.315488:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x168f:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.315492:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x168f:0x0](ffff9181c6b5aa08) 00000080:00200000:1.0:1787826191.315515:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1690:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.315518:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.315524:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1690:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.315528:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1690:0x0](ffff9181c6b40908) 00000080:00200000:1.0:1787826191.315549:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1691:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.315553:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.315559:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1691:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.315563:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1691:0x0](ffff9181c6b442c8) 00000080:00200000:1.0:1787826191.315586:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1692:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.315592:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.315598:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1692:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.315602:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1692:0x0](ffff9181c6b65348) 00000080:00200000:1.0:1787826191.315627:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1693:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.315632:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.315639:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1693:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.315642:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1693:0x0](ffff9181c6b563c8) 00000080:00200000:1.0:1787826191.315666:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1694:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.315670:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.315678:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1694:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.315682:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1694:0x0](ffff9181c6b6db88) 00000080:00200000:1.0:1787826191.315703:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1695:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.315707:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.315713:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1695:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.315718:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1695:0x0](ffff9181c6b6ba88) 00000080:00200000:1.0:1787826191.315739:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1696:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.315767:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.315775:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1696:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.315779:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1696:0x0](ffff9181c6b5cb08) 00000080:00200000:1.0:1787826191.315805:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1697:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.315810:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.315816:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1697:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.315820:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1697:0x0](ffff9181c6b6b248) 00000080:00200000:1.0:1787826191.315841:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1698:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.315856:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.315864:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1698:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.315868:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1698:0x0](ffff9181c6b6aa08) 00000080:00200000:1.0:1787826191.315893:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1699:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.315898:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.315904:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1699:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.315908:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1699:0x0](ffff9181c6b66c08) 00000080:00200000:1.0:1787826191.315932:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x169a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.315938:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.315945:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x169a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.315949:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x169a:0x0](ffff9181c6b58908) 00000080:00200000:1.0:1787826191.315974:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x169b:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.315978:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.315985:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x169b:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.315988:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x169b:0x0](ffff9181c6b45348) 00000080:00200000:1.0:1787826191.316013:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x169c:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.316017:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.316024:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x169c:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.316028:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x169c:0x0](ffff9181c6b600c8) 00000080:00200000:1.0:1787826191.316053:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x169d:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.316057:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.316066:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x169d:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.316071:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x169d:0x0](ffff9181c6b742c8) 00000080:00200000:1.0:1787826191.316094:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x169e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.316098:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.316104:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x169e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.316108:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x169e:0x0](ffff9181c6b6cb08) 00000080:00200000:1.0:1787826191.316129:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x169f:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.316134:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.316140:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x169f:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.316144:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x169f:0x0](ffff9181c6b5c2c8) 00000080:00200000:1.0:1787826191.316166:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16a0:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.316170:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.316177:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16a0:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.316181:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16a0:0x0](ffff9181c6b68908) 00000080:00200000:1.0:1787826191.316203:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16a1:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.316208:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.316215:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16a1:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.316219:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16a1:0x0](ffff9181c6b6c2c8) 00000080:00200000:1.0:1787826191.316241:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16a2:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.316245:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.316252:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16a2:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.316256:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16a2:0x0](ffff9181c6b5d348) 00000080:00200000:1.0:1787826191.316280:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16a3:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.316284:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.316291:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16a3:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.316295:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16a3:0x0](ffff9181c6b7cb08) 00000080:00200000:1.0:1787826191.316319:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16a4:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.316323:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.316330:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16a4:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.316334:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16a4:0x0](ffff9181c6b78908) 00000080:00200000:1.0:1787826191.316357:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16a5:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.316362:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.316369:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16a5:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.316373:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16a5:0x0](ffff9181c6b75348) 00000080:00200000:1.0:1787826191.316398:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16a6:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.316402:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.316410:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16a6:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.316414:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16a6:0x0](ffff9181c6b6d348) 00000080:00200000:1.0:1787826191.316437:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16a7:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.316441:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.316447:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16a7:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.316451:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16a7:0x0](ffff9181c6b76c08) 00000080:00200000:1.0:1787826191.316492:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16a8:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.316497:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.316505:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16a8:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.316509:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16a8:0x0](ffff9181c6b700c8) 00000080:00200000:1.0:1787826191.316533:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16a9:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.316538:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.316544:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16a9:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.316548:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16a9:0x0](ffff9181c6b6ec08) 00000080:00200000:1.0:1787826191.316571:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16aa:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826191.316576:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826191.316583:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16aa:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826191.316587:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16aa:0x0](ffff9181c6b61148) 00000080:00200000:2.0:1787826191.319365:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16ab:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.319381:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.319406:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16ab:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.319414:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16ab:0x0](ffff9181c6b7c2c8) 00000080:00200000:2.0:1787826191.319508:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16ac:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.319515:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.319524:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16ac:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.319528:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16ac:0x0](ffff9181c6b71148) 00000080:00200000:2.0:1787826191.319555:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16ad:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.319560:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.319567:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16ad:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.319571:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16ad:0x0](ffff9181c6b680c8) 00000080:00200000:2.0:1787826191.319597:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16ae:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.319603:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.319609:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16ae:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.319613:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16ae:0x0](ffff9181c6b763c8) 00000080:00200000:2.0:1787826191.319633:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16af:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.319638:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.319644:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16af:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.319648:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16af:0x0](ffff9181c6b69148) 00000080:00200000:2.0:1787826191.319670:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16b0:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.319674:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.319680:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16b0:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.319683:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16b0:0x0](ffff9181c6b6e3c8) 00000080:00200000:2.0:1787826191.319719:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16b1:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.319724:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.319730:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16b1:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.319735:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16b1:0x0](ffff9181c6b663c8) 00000080:00200000:2.0:1787826191.319784:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16b2:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.319789:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.319797:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16b2:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.319801:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16b2:0x0](ffff9181c6b6f448) 00000080:00200000:2.0:1787826191.319824:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16b3:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.319829:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.319836:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16b3:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.319840:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16b3:0x0](ffff9181c6b6a1c8) 00000080:00200000:2.0:1787826191.319874:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16b4:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.319879:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.319887:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16b4:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.319891:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16b4:0x0](ffff9181c6b69988) 00000080:00200000:2.0:1787826191.319910:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16b5:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.319915:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.319921:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16b5:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.319925:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16b5:0x0](ffff9181c7b43a88) 00000080:00200000:2.0:1787826191.319945:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16b6:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.319949:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.319956:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16b6:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.319960:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16b6:0x0](ffff9181c7b43248) 00000080:00200000:2.0:1787826191.319980:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16b7:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.319984:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.319990:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16b7:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.319994:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16b7:0x0](ffff9181c7b42a08) 00000080:00200000:2.0:1787826191.320022:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16b8:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.320027:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.320034:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16b8:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.320039:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16b8:0x0](ffff9181c7b44b08) 00000080:00200000:2.0:1787826191.320094:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16b9:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.320099:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.320109:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16b9:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.320115:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16b9:0x0](ffff9181c6b77448) 00000080:00200000:2.0:1787826191.320286:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16ba:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.320291:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.320300:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16ba:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.320304:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16ba:0x0](ffff9181c7b40908) 00000080:00200000:2.0:1787826191.320332:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16bb:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.320337:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.320344:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16bb:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.320348:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16bb:0x0](ffff9181c7b442c8) 00000080:00200000:2.0:1787826191.320375:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16bc:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.320380:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.320388:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16bc:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.320393:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16bc:0x0](ffff9181c6b7d348) 00000080:00200000:2.0:1787826191.320437:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16bd:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.320443:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.320452:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16bd:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.320457:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16bd:0x0](ffff9181c7b45348) 00000080:00200000:2.0:1787826191.320502:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16be:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.320507:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.320515:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16be:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.320519:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16be:0x0](ffff9181c6b67448) 00000080:00200000:2.0:1787826191.320546:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16bf:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.320551:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.320558:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16bf:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.320562:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16bf:0x0](ffff9181c6b7ec08) 00000080:00200000:2.0:1787826191.320583:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16c0:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.320587:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.320594:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16c0:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.320598:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16c0:0x0](ffff9181c6b721c8) 00000080:00200000:2.0:1787826191.320620:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16c1:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.320624:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.320631:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16c1:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.320634:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16c1:0x0](ffff9181c7b46c08) 00000080:00200000:2.0:1787826191.320656:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16c2:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.320660:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.320667:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16c2:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.320671:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16c2:0x0](ffff9181c6b71988) 00000080:00200000:2.0:1787826191.320691:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16c3:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.320696:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.320813:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16c3:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.320818:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16c3:0x0](ffff9181c7b400c8) 00000080:00200000:2.0:1787826191.320856:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16c4:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.320861:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.320869:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16c4:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.320873:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16c4:0x0](ffff9181c6b621c8) 00000080:00200000:2.0:1787826191.320897:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16c5:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.320901:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.320908:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16c5:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.320912:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16c5:0x0](ffff9181c7b41148) 00000080:00200000:2.0:1787826191.320937:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16c6:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.320941:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.320948:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16c6:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.320952:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16c6:0x0](ffff9181c7b463c8) 00000080:00200000:2.0:1787826191.320975:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16c7:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.320980:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.320987:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16c7:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.320990:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16c7:0x0](ffff9181c7b47448) 00000080:00200000:2.0:1787826191.321034:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16c8:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.321040:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.321052:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16c8:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.321056:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16c8:0x0](ffff9181c6b780c8) 00000080:00200000:2.0:1787826191.321122:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16c9:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.321128:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.321136:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16c9:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.321635:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16c9:0x0](ffff9181c6b75b88) 00000080:00200000:2.0:1787826191.321667:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16ca:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.321673:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.321684:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16ca:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.321688:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16ca:0x0](ffff9181c6b73a88) 00000080:00200000:3.0:1787826191.325451:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16cb:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.325467:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.325490:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16cb:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.325497:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16cb:0x0](ffff9181c6b61988) 00000080:00200000:3.0:1787826191.325542:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16cc:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.325547:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.325555:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16cc:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.325559:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16cc:0x0](ffff9181c7b421c8) 00000080:00200000:3.0:1787826191.325591:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16cd:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.325596:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.325603:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16cd:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.325607:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16cd:0x0](ffff9181c7b41988) 00000080:00200000:3.0:1787826191.325631:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16ce:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.325636:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.325643:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16ce:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.325647:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16ce:0x0](ffff9181c7b45b88) 00000080:00200000:3.0:1787826191.325672:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16cf:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.325677:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.325684:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16cf:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.325688:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16cf:0x0](ffff9181c7b4d348) 00000080:00200000:3.0:1787826191.325726:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16d0:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.325731:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.325739:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16d0:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.325772:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16d0:0x0](ffff9181c7b4ec08) 00000080:00200000:3.0:1787826191.325797:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16d1:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.325802:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.325810:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16d1:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.325813:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16d1:0x0](ffff9181c6b79148) 00000080:00200000:3.0:1787826191.325839:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16d2:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.325854:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.325862:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16d2:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.325866:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16d2:0x0](ffff9181c6b73248) 00000080:00200000:3.0:1787826191.325887:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16d3:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.325892:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.325897:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16d3:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.325901:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16d3:0x0](ffff9181c6b7e3c8) 00000080:00200000:3.0:1787826191.325918:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16d4:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.325922:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.325928:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16d4:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.325933:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16d4:0x0](ffff9181c7b480c8) 00000080:00200000:3.0:1787826191.325956:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16d5:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.325961:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.325968:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16d5:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.325972:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16d5:0x0](ffff9181c6b72a08) 00000080:00200000:3.0:1787826191.325996:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16d6:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.326002:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.326010:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16d6:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.326015:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16d6:0x0](ffff9181c7b49148) 00000080:00200000:3.0:1787826191.326046:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16d7:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.326051:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.326059:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16d7:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.326063:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16d7:0x0](ffff9181c7b4e3c8) 00000080:00200000:3.0:1787826191.326084:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16d8:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.326088:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.326094:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16d8:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.326097:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16d8:0x0](ffff9181c6b7f448) 00000080:00200000:3.0:1787826191.326116:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16d9:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.326120:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.326127:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16d9:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.326130:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16d9:0x0](ffff9181c6b65b88) 00000080:00200000:3.0:1787826191.326171:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16da:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.326175:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.326182:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16da:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.326186:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16da:0x0](ffff9181c6b63a88) 00000080:00200000:3.0:1787826191.326208:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16db:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.326213:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.326219:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16db:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.326223:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16db:0x0](ffff9181c7b4f448) 00000080:00200000:3.0:1787826191.326245:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16dc:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.326250:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.326257:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16dc:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.326260:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16dc:0x0](ffff9181c6b74b08) 00000080:00200000:3.0:1787826191.326283:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16dd:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.326287:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.326294:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16dd:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.326298:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16dd:0x0](ffff9181c6b63248) 00000080:00200000:3.0:1787826191.326320:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16de:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.326324:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.326331:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16de:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.326334:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16de:0x0](ffff9181c7b4a1c8) 00000080:00200000:3.0:1787826191.326357:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16df:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.326362:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.326370:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16df:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.326376:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16df:0x0](ffff9181c6b70908) 00000080:00200000:3.0:1787826191.326401:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16e0:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.326405:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.326411:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16e0:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.326414:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16e0:0x0](ffff9181c6b62a08) 00000080:00200000:3.0:1787826191.326435:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16e1:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.326440:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.326446:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16e1:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.326450:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16e1:0x0](ffff9181c7b57448) 00000080:00200000:3.0:1787826191.326473:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16e2:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.326476:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.326483:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16e2:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.326487:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16e2:0x0](ffff9181c7b521c8) 00000080:00200000:3.0:1787826191.326507:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16e3:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.326512:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.326518:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16e3:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.326522:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16e3:0x0](ffff9181c6b64b08) 00000080:00200000:3.0:1787826191.326544:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16e4:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.326548:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.326555:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16e4:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.326559:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16e4:0x0](ffff9181c7b49988) 00000080:00200000:3.0:1787826191.326580:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16e5:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.326584:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.326592:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16e5:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.326596:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16e5:0x0](ffff9181c7b4db88) 00000080:00200000:3.0:1787826191.326617:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16e6:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.326622:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.326629:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16e6:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.326634:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16e6:0x0](ffff9181c7b51988) 00000080:00200000:3.0:1787826191.326658:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16e7:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.326663:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.326669:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16e7:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.326673:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16e7:0x0](ffff9181c7b4ba88) 00000080:00200000:3.0:1787826191.326693:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16e8:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.326698:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.326704:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16e8:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.326708:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16e8:0x0](ffff9181c6b7a1c8) 00000080:00200000:3.0:1787826191.326730:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16e9:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.326734:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.326766:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16e9:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.326772:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16e9:0x0](ffff9181c7b55b88) 00000080:00200000:3.0:1787826191.326796:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16ea:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.326800:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.326807:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16ea:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.326810:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16ea:0x0](ffff9181c6b79988) 00000080:00200000:3.0:1787826191.326831:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16eb:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.326835:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.326841:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16eb:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.326855:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16eb:0x0](ffff9181c6b7db88) 00000080:00200000:3.0:1787826191.326876:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16ec:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.326881:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.326887:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16ec:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.326890:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16ec:0x0](ffff9181c7b4b248) 00000080:00200000:3.0:1787826191.326907:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16ed:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.326912:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.326918:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16ed:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.326921:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16ed:0x0](ffff9181c7b53a88) 00000080:00200000:3.0:1787826191.326943:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16ee:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.326947:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.326954:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16ee:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.326958:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16ee:0x0](ffff9181c7b53248) 00000080:00200000:3.0:1787826191.326980:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16ef:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.326985:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.326992:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16ef:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.326996:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16ef:0x0](ffff9181c7b4aa08) 00000080:00200000:3.0:1787826191.327018:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16f0:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.327022:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.327029:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16f0:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.327033:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16f0:0x0](ffff9181c7b4cb08) 00000080:00200000:3.0:1787826191.327054:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16f1:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.327058:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.327064:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16f1:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.327068:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16f1:0x0](ffff9181c7b48908) 00000080:00200000:3.0:1787826191.327086:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16f2:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.327090:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.327095:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16f2:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.327099:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16f2:0x0](ffff9181c7b4c2c8) 00000080:00200000:3.0:1787826191.327117:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16f3:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.327120:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.327126:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16f3:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.327129:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16f3:0x0](ffff9181c6b7ba88) 00000080:00200000:3.0:1787826191.327155:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16f4:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.327160:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.327167:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16f4:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.327171:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16f4:0x0](ffff9181c7b5e3c8) 00000080:00200000:3.0:1787826191.327192:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16f5:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.327197:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.327203:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16f5:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.327207:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16f5:0x0](ffff9181c6b60908) 00000080:00200000:3.0:1787826191.327229:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16f6:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.327234:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.327240:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16f6:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.327245:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16f6:0x0](ffff9181c7b52a08) 00000080:00200000:3.0:1787826191.327269:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16f7:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.327273:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.327280:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16f7:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.327283:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16f7:0x0](ffff9181c7b5f448) 00000080:00200000:3.0:1787826191.327306:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16f8:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.327310:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.327317:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16f8:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.327321:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16f8:0x0](ffff9181c6b7b248) 00000080:00200000:3.0:1787826191.327345:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16f9:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.327349:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.327356:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16f9:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.327361:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16f9:0x0](ffff9181c7b5a1c8) 00000080:00200000:3.0:1787826191.327382:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16fa:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.327387:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.327394:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16fa:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.327397:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16fa:0x0](ffff9181c7b59988) 00000080:00200000:3.0:1787826191.327418:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16fb:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.327423:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.327429:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16fb:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.327433:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16fb:0x0](ffff9181c6b7aa08) 00000080:00200000:3.0:1787826191.327456:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16fc:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.327460:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.327467:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16fc:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.327470:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16fc:0x0](ffff9181c7b5db88) 00000080:00200000:3.0:1787826191.327492:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16fd:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.327497:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.327503:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16fd:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.327508:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16fd:0x0](ffff9181c7b61988) 00000080:00200000:3.0:1787826191.327532:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16fe:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.327536:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.327542:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16fe:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.327546:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16fe:0x0](ffff9181c7b65b88) 00000080:00200000:3.0:1787826191.327568:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x16ff:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.327572:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.327579:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x16ff:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.327583:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x16ff:0x0](ffff9181c7b5ba88) 00000080:00200000:3.0:1787826191.327606:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1700:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.327611:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.327618:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1700:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.327622:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1700:0x0](ffff9181c7b63a88) 00000080:00200000:3.0:1787826191.327646:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1701:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.327651:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.327657:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1701:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.327661:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1701:0x0](ffff9181c7b63248) 00000080:00200000:3.0:1787826191.327683:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1702:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.327688:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.327695:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1702:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.327699:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1702:0x0](ffff9181c7b62a08) 00000080:00200000:3.0:1787826191.327722:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1703:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.327726:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.327734:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1703:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.327738:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1703:0x0](ffff9181c7b5b248) 00000080:00200000:3.0:1787826191.327785:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1704:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.327790:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.327798:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1704:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.327802:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1704:0x0](ffff9181c7b5aa08) 00000080:00200000:3.0:1787826191.327827:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1705:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.327832:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.327839:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1705:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.327842:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1705:0x0](ffff9181c7b5cb08) 00000080:00200000:3.0:1787826191.327876:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1706:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.327881:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.327888:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1706:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.327892:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1706:0x0](ffff9181c7b6c2c8) 00000080:00200000:3.0:1787826191.327917:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1707:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.327921:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.327928:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1707:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.327931:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1707:0x0](ffff9181c7b64b08) 00000080:00200000:3.0:1787826191.327955:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1708:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.327959:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.327966:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1708:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.327970:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1708:0x0](ffff9181c7b60908) 00000080:00200000:3.0:1787826191.327994:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1709:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.327999:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.328006:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1709:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.328011:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1709:0x0](ffff9181c7b642c8) 00000080:00200000:3.0:1787826191.328034:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x170a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.328038:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.328046:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x170a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.328050:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x170a:0x0](ffff9181c7b58908) 00000080:00200000:3.0:1787826191.328073:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x170b:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.328078:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.328085:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x170b:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.328089:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x170b:0x0](ffff9181c7b5c2c8) 00000080:00200000:3.0:1787826191.328113:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x170c:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.328118:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.328125:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x170c:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.328130:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x170c:0x0](ffff9181c7b54b08) 00000080:00200000:3.0:1787826191.328156:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x170d:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.328161:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.328168:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x170d:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.328173:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x170d:0x0](ffff9181c7b5d348) 00000080:00200000:3.0:1787826191.328194:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x170e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.328199:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.328207:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x170e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.328211:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x170e:0x0](ffff9181c7b65348) 00000080:00200000:3.0:1787826191.328232:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x170f:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.328236:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.328242:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x170f:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.328246:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x170f:0x0](ffff9181c7b50908) 00000080:00200000:3.0:1787826191.328288:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1710:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.328293:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.328301:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1710:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.328305:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1710:0x0](ffff9181c7b5ec08) 00000080:00200000:3.0:1787826191.328338:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1711:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.328343:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.328351:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1711:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.328356:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1711:0x0](ffff9181c7b542c8) 00000080:00200000:3.0:1787826191.328382:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1712:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.328387:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.328394:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1712:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.328397:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1712:0x0](ffff9181c7b580c8) 00000080:00200000:3.0:1787826191.328419:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1713:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.328424:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.328430:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1713:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.328434:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1713:0x0](ffff9181c7b59148) 00000080:00200000:3.0:1787826191.328455:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1714:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.328459:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.328465:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1714:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.328469:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1714:0x0](ffff9181c7b66c08) 00000080:00200000:3.0:1787826191.328491:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1715:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.328496:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.328503:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1715:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.328507:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1715:0x0](ffff9181c7b55348) 00000080:00200000:3.0:1787826191.328528:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1716:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.328532:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.328538:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1716:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.328542:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1716:0x0](ffff9181c7b600c8) 00000080:00200000:3.0:1787826191.328564:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1717:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.328569:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.328576:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1717:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.328579:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1717:0x0](ffff9181c7b61148) 00000080:00200000:3.0:1787826191.328602:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1718:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.328607:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.328614:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1718:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.328618:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1718:0x0](ffff9181c7b73248) 00000080:00200000:3.0:1787826191.328641:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1719:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.328646:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.328653:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1719:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.328657:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1719:0x0](ffff9181c7b72a08) 00000080:00200000:3.0:1787826191.328679:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x171a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.328684:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.328690:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x171a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.328694:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x171a:0x0](ffff9181c7b56c08) 00000080:00200000:3.0:1787826191.328716:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x171b:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.328720:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.328727:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x171b:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.328733:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x171b:0x0](ffff9181c7b663c8) 00000080:00200000:3.0:1787826191.328786:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x171c:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.328791:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.328799:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x171c:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.328803:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x171c:0x0](ffff9181c7b67448) 00000080:00200000:3.0:1787826191.328829:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x171d:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.328834:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.328841:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x171d:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.329409:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x171d:0x0](ffff9181c7b500c8) 00000080:00200000:3.0:1787826191.329444:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x171e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.329450:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.329459:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x171e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.329463:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x171e:0x0](ffff9181c7b51148) 00000080:00200000:3.0:1787826191.329487:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x171f:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.329492:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.329498:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x171f:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.329502:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x171f:0x0](ffff9181c7b563c8) 00000080:00200000:3.0:1787826191.329525:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1721:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.329529:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.329536:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1721:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.329540:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1721:0x0](ffff9181c7b621c8) 00000080:00200000:3.0:1787826191.329563:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1722:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.329568:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.329575:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1722:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.329578:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1722:0x0](ffff9181c7b74b08) 00000080:00200000:3.0:1787826191.329604:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1723:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.329609:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.329616:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1723:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.329620:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1723:0x0](ffff9181c7b70908) 00000080:00200000:3.0:1787826191.329643:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1724:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.329648:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.329656:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1724:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.329660:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1724:0x0](ffff9181c7b6d348) 00000080:00200000:3.0:1787826191.329692:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1725:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.329696:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.335712:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1725:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.335729:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1725:0x0](ffff9181c7b7aa08) 00000080:00200000:3.0:1787826191.335832:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1726:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.335843:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.335864:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1726:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.335868:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1726:0x0](ffff9181c7b742c8) 00000080:00200000:3.0:1787826191.335900:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1727:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.335905:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.335911:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1727:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.335915:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1727:0x0](ffff9181c7b6ec08) 00000080:00200000:3.0:1787826191.335958:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1728:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.335963:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.335970:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1728:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.335974:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1728:0x0](ffff9181c7b680c8) 00000080:00200000:3.0:1787826191.336002:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1729:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.336007:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.336014:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1729:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.336019:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1729:0x0](ffff9181c7b7cb08) 00000080:00200000:3.0:1787826191.336041:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x172a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.336047:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.336053:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x172a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.336056:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x172a:0x0](ffff9181c7b800c8) 00000080:00200000:3.0:1787826191.336084:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x172b:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.336088:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.336093:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x172b:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.336097:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x172b:0x0](ffff9181c7b78908) 00000080:00200000:3.0:1787826191.336116:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x172c:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.336121:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.336127:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x172c:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.336131:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x172c:0x0](ffff9181c7b81148) 00000080:00200000:3.0:1787826191.336154:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x172d:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.336172:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.336179:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x172d:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.336183:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x172d:0x0](ffff9181c7b75348) 00000080:00200000:3.0:1787826191.336212:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x172e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.336216:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.336223:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x172e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.336227:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x172e:0x0](ffff9181c7b76c08) 00000080:00200000:3.0:1787826191.336250:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x172f:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.336254:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.336260:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x172f:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.336264:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x172f:0x0](ffff9181c7b7c2c8) 00000080:00200000:3.0:1787826191.336285:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1730:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.336290:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.336296:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1730:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.336300:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1730:0x0](ffff9181c7b7d348) 00000080:00200000:3.0:1787826191.336322:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1731:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.336327:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.336333:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1731:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.336337:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1731:0x0](ffff9181c7b7ec08) 00000080:00200000:3.0:1787826191.336361:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1732:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.336366:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.336372:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1732:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.336376:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1732:0x0](ffff9181c7b69148) 00000080:00200000:3.0:1787826191.336400:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1733:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.336405:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.336412:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1733:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.336416:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1733:0x0](ffff9181c7b700c8) 00000080:00200000:3.0:1787826191.336436:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1734:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.336441:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.336447:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1734:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.336452:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1734:0x0](ffff9181c7b71148) 00000080:00200000:3.0:1787826191.336474:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1735:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.336479:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.336485:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1735:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.336489:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1735:0x0](ffff9181c7b763c8) 00000080:00200000:3.0:1787826191.336511:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1736:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.336516:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.336522:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1736:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.336527:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1736:0x0](ffff9181c7b780c8) 00000080:00200000:3.0:1787826191.336552:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1737:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.336556:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.336563:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1737:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.336567:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1737:0x0](ffff9181c7b77448) 00000080:00200000:3.0:1787826191.336590:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1738:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.336593:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.336600:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1738:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.336604:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1738:0x0](ffff9181c7b721c8) 00000080:00200000:3.0:1787826191.336626:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1739:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.336630:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.336636:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1739:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.336640:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1739:0x0](ffff9181c7b6e3c8) 00000080:00200000:3.0:1787826191.336665:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x173a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.336670:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.336677:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x173a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.336680:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x173a:0x0](ffff9181c7b79148) 00000080:00200000:3.0:1787826191.336702:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x173b:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.336707:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.336714:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x173b:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.336718:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x173b:0x0](ffff9181c7b71988) 00000080:00200000:3.0:1787826191.336768:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x173c:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.336774:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.336781:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x173c:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.336785:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x173c:0x0](ffff9181c7b6f448) 00000080:00200000:3.0:1787826191.336809:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x173d:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.336814:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.336819:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x173d:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.336823:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x173d:0x0](ffff9181c7b7e3c8) 00000080:00200000:3.0:1787826191.336857:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x173e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.336862:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.336869:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x173e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.336873:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x173e:0x0](ffff9181c7b6a1c8) 00000080:00200000:3.0:1787826191.336898:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x173f:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.336903:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.336909:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x173f:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.336913:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x173f:0x0](ffff9181c7b7f448) 00000080:00200000:3.0:1787826191.336934:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1740:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.336939:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.336946:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1740:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.336950:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1740:0x0](ffff9181c7b75b88) 00000080:00200000:3.0:1787826191.336975:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1741:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.336980:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.336986:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1741:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.336989:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1741:0x0](ffff9181c7b73a88) 00000080:00200000:3.0:1787826191.337011:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1742:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.337016:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.337022:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1742:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.337026:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1742:0x0](ffff9181c7b863c8) 00000080:00200000:3.0:1787826191.337051:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1743:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.337056:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.337063:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1743:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.337066:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1743:0x0](ffff9181c7b8c2c8) 00000080:00200000:3.0:1787826191.337091:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1744:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.337096:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.337102:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1744:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.337106:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1744:0x0](ffff9181c7b7a1c8) 00000080:00200000:3.0:1787826191.337127:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1745:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.337132:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.337138:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1745:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.337143:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1745:0x0](ffff9181c7b87448) 00000080:00200000:3.0:1787826191.337379:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1746:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.337385:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.337393:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1746:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.337397:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1746:0x0](ffff9181c7b8d348) 00000080:00200000:3.0:1787826191.337424:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1747:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.337428:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.337435:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1747:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.337439:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1747:0x0](ffff9181c7b8ec08) 00000080:00200000:3.0:1787826191.337461:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1748:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.337465:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.337471:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1748:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.337474:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1748:0x0](ffff9181c7b880c8) 00000080:00200000:3.0:1787826191.337496:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1749:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.337501:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.337508:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1749:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.337511:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1749:0x0](ffff9181c7b79988) 00000080:00200000:3.0:1787826191.337536:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x174a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.337540:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.337546:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x174a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.337549:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x174a:0x0](ffff9181c7b69988) 00000080:00200000:3.0:1787826191.337573:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x174b:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.337578:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.337585:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x174b:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.337589:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x174b:0x0](ffff9181c7b7db88) 00000080:00200000:3.0:1787826191.337614:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x174c:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.337619:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.337625:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x174c:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.337629:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x174c:0x0](ffff9181c7b821c8) 00000080:00200000:3.0:1787826191.337653:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x174d:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.337657:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.337664:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x174d:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.337668:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x174d:0x0](ffff9181c7b89148) 00000080:00200000:3.0:1787826191.337693:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x174e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.337698:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.337704:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x174e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.337708:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x174e:0x0](ffff9181c7b81988) 00000080:00200000:3.0:1787826191.337731:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x174f:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.337736:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.337772:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x174f:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.337776:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x174f:0x0](ffff9181c7b85b88) 00000080:00200000:3.0:1787826191.337814:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1750:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.337820:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.337830:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1750:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.337835:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1750:0x0](ffff9181c7b8e3c8) 00000080:00200000:3.0:1787826191.337873:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1751:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.337878:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.337884:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1751:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.337888:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1751:0x0](ffff9181c7b8f448) 00000080:00200000:3.0:1787826191.337909:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1752:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.337914:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.337919:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1752:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.337923:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1752:0x0](ffff9181c7b83a88) 00000080:00200000:3.0:1787826191.337945:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1753:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.337950:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.337956:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1753:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.337960:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1753:0x0](ffff9181c7b8a1c8) 00000080:00200000:3.0:1787826191.337984:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1754:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.337988:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.337996:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1754:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.338000:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1754:0x0](ffff9181c7b83248) 00000080:00200000:3.0:1787826191.338024:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1755:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.338028:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.338036:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1755:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.338040:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1755:0x0](ffff9181c7b89988) 00000080:00200000:3.0:1787826191.338061:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1756:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.338065:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.338071:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1756:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.338075:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1756:0x0](ffff9181c7b6db88) 00000080:00200000:3.0:1787826191.338094:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1757:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.338098:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.338103:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1757:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.338107:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1757:0x0](ffff9181c7b6ba88) 00000080:00200000:3.0:1787826191.338125:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1758:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.338129:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.338134:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1758:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.338137:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1758:0x0](ffff9181c7b8db88) 00000080:00200000:3.0:1787826191.338158:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1759:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.338162:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.338169:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1759:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.338173:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1759:0x0](ffff9181c7b8ba88) 00000080:00200000:3.0:1787826191.338198:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x175a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.338202:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.338209:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x175a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.338213:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x175a:0x0](ffff9181c7b82a08) 00000080:00200000:3.0:1787826191.338235:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x175b:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.338239:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.338246:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x175b:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.338250:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x175b:0x0](ffff9181c7b8b248) 00000080:00200000:3.0:1787826191.338276:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x175c:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.338280:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.338286:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x175c:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.338290:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x175c:0x0](ffff9181c7b8aa08) 00000080:00200000:3.0:1787826191.338311:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x175d:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.338316:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.338323:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x175d:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.338327:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x175d:0x0](ffff9181c7b6b248) 00000080:00200000:3.0:1787826191.338350:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x175e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.338355:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.338361:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x175e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.338365:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x175e:0x0](ffff9181c7b6aa08) 00000080:00200000:3.0:1787826191.338386:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x175f:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.338389:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.338394:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x175f:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.338398:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x175f:0x0](ffff9181c7b8cb08) 00000080:00200000:3.0:1787826191.338416:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1760:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.338419:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.338424:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1760:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.338428:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1760:0x0](ffff9181c7b84b08) 00000080:00200000:3.0:1787826191.338446:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1761:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.338450:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.338455:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1761:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.338459:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1761:0x0](ffff9181c7b7ba88) 00000080:00200000:3.0:1787826191.338478:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1762:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.338482:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.338488:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1762:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.338491:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1762:0x0](ffff9181c7b7b248) 00000080:00200000:3.0:1787826191.338509:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1763:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.338513:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.338518:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1763:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.338521:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1763:0x0](ffff9181c7b88908) 00000080:00200000:3.0:1787826191.338539:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1764:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.338542:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.338547:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1764:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.338551:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1764:0x0](ffff9181c7b91148) 00000080:00200000:3.0:1787826191.338572:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1765:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.338577:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.338583:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1765:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.338587:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1765:0x0](ffff9181c7b80908) 00000080:00200000:3.0:1787826191.338610:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1766:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.338614:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.338620:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1766:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.338624:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1766:0x0](ffff9181c7b6cb08) 00000080:00200000:3.0:1787826191.338643:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1767:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.338647:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.338653:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1767:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.338658:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1767:0x0](ffff9181c7b9d348) 00000080:00200000:3.0:1787826191.338680:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1768:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.338685:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.338691:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1768:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.338695:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1768:0x0](ffff9181c7b963c8) 00000080:00200000:3.0:1787826191.338719:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1769:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.338724:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.338731:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1769:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.338735:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1769:0x0](ffff9181c7b9ec08) 00000080:00200000:3.0:1787826191.338789:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x176a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.338794:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.338801:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x176a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.338804:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x176a:0x0](ffff9181c7b980c8) 00000080:00200000:3.0:1787826191.338826:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x176b:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.338829:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.338835:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x176b:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.338838:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x176b:0x0](ffff9181c7b97448) 00000080:00200000:3.0:1787826191.342331:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x176c:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.342349:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.342374:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x176c:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.342381:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x176c:0x0](ffff9181c7b68908) 00000080:00200000:3.0:1787826191.342425:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x176d:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.342430:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.342438:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x176d:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.342442:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x176d:0x0](ffff9181c7b921c8) 00000080:00200000:3.0:1787826191.342464:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x176e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.342470:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.342476:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x176e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.342480:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x176e:0x0](ffff9181c7b99148) 00000080:00200000:3.0:1787826191.342506:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x176f:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.342512:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.342518:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x176f:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.342522:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x176f:0x0](ffff9181c7b842c8) 00000080:00200000:3.0:1787826191.342559:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1770:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.342564:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.342573:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1770:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.342577:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1770:0x0](ffff9181c7b85348) 00000080:00200000:3.0:1787826191.342601:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1771:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.342606:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.342613:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1771:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.342617:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1771:0x0](ffff9181c7b91988) 00000080:00200000:3.0:1787826191.342642:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1772:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.342646:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.342653:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1772:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.342657:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1772:0x0](ffff9181c7b9e3c8) 00000080:00200000:3.0:1787826191.342680:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1773:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.342685:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.342691:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1773:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.342695:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1773:0x0](ffff9181c7b95b88) 00000080:00200000:3.0:1787826191.342940:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1774:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.342945:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.342954:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1774:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.342958:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1774:0x0](ffff9181c7b93a88) 00000080:00200000:3.0:1787826191.342982:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1775:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.342987:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.342994:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1775:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.342999:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1775:0x0](ffff9181c7b9f448) 00000080:00200000:3.0:1787826191.343022:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1776:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.343026:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.343033:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1776:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.343037:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1776:0x0](ffff9181c7b9a1c8) 00000080:00200000:3.0:1787826191.343057:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1777:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.343062:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.343069:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1777:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.343072:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1777:0x0](ffff9181c7b86c08) 00000080:00200000:3.0:1787826191.343096:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1778:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.343101:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.343108:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1778:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.343112:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1778:0x0](ffff9181c7b93248) 00000080:00200000:3.0:1787826191.343135:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1779:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.343139:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.343146:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1779:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.343150:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1779:0x0](ffff9181c7b99988) 00000080:00200000:3.0:1787826191.343279:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x177a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.343284:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.343293:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x177a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.343297:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x177a:0x0](ffff9181c7b9db88) 00000080:00200000:3.0:1787826191.343319:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x177b:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.343324:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.343331:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x177b:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.343335:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x177b:0x0](ffff9181c7ba2a08) 00000080:00200000:3.0:1787826191.343358:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x177c:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.343363:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.343370:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x177c:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.343374:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x177c:0x0](ffff9181c7b9ba88) 00000080:00200000:3.0:1787826191.343398:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x177d:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.343403:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.343410:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x177d:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.343415:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x177d:0x0](ffff9181c7b92a08) 00000080:00200000:3.0:1787826191.343438:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x177e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.343443:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.343449:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x177e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.343454:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x177e:0x0](ffff9181c7b94b08) 00000080:00200000:3.0:1787826191.343477:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x177f:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.343481:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.343488:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x177f:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.343492:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x177f:0x0](ffff9181c7b90908) 00000080:00200000:3.0:1787826191.343514:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1780:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.343519:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.343525:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1780:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.343529:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1780:0x0](ffff9181c7ba4b08) 00000080:00200000:3.0:1787826191.343552:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1781:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.343557:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.343564:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1781:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.343568:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1781:0x0](ffff9181c7b942c8) 00000080:00200000:3.0:1787826191.343592:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1782:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.343597:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.343604:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1782:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.343608:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1782:0x0](ffff9181c7b9b248) 00000080:00200000:3.0:1787826191.343632:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1783:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.343636:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.343644:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1783:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.343648:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1783:0x0](ffff9181c7b9aa08) 00000080:00200000:3.0:1787826191.343669:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1784:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.343673:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.343680:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1784:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.343684:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1784:0x0](ffff9181c7ba0908) 00000080:00200000:3.0:1787826191.344076:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1785:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.344082:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.344092:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1785:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.344096:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1785:0x0](ffff9181c7b9cb08) 00000080:00200000:3.0:1787826191.344177:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1786:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.344183:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.344192:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1786:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.344196:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1786:0x0](ffff9181c7b98908) 00000080:00200000:3.0:1787826191.344222:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1787:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.344227:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.344234:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1787:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.344238:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1787:0x0](ffff9181c7b9c2c8) 00000080:00200000:3.0:1787826191.344260:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1788:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.344266:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.344272:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1788:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.344276:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1788:0x0](ffff9181c7b95348) 00000080:00200000:3.0:1787826191.344299:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1789:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.344304:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.344310:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1789:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.344314:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1789:0x0](ffff9181c7b96c08) 00000080:00200000:3.0:1787826191.344336:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x178a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.344341:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.344347:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x178a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.344351:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x178a:0x0](ffff9181c7bae3c8) 00000080:00200000:3.0:1787826191.344375:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x178b:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.344380:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.344387:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x178b:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.344391:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x178b:0x0](ffff9181c7baf448) 00000080:00200000:3.0:1787826191.344414:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x178c:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.344418:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.344425:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x178c:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.344429:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x178c:0x0](ffff9181c7baa1c8) 00000080:00200000:3.0:1787826191.344450:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x178d:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.344454:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.344461:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x178d:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.344465:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x178d:0x0](ffff9181c7b900c8) 00000080:00200000:3.0:1787826191.344487:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x178e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.344492:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.344498:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x178e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.344502:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x178e:0x0](ffff9181c7bb21c8) 00000080:00200000:3.0:1787826191.344524:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x178f:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.344529:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.344536:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x178f:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.344540:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x178f:0x0](ffff9181c7bb1988) 00000080:00200000:3.0:1787826191.344561:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1790:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.344565:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.344572:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1790:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.344575:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1790:0x0](ffff9181c7bb5b88) 00000080:00200000:3.0:1787826191.344596:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1791:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.344601:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.344608:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1791:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.344612:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1791:0x0](ffff9181c7ba9988) 00000080:00200000:3.0:1787826191.344634:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1792:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.344639:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.344646:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1792:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.344650:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1792:0x0](ffff9181c7ba42c8) 00000080:00200000:2.0:1787826191.347515:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1793:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.347532:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.347558:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1793:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.347566:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1793:0x0](ffff9181c7badb88) 00000080:00200000:2.0:1787826191.347606:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1794:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.347611:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.347618:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1794:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.347622:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1794:0x0](ffff9181c7baba88) 00000080:00200000:2.0:1787826191.347646:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1795:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.347651:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.347658:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1795:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.347662:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1795:0x0](ffff9181c7bb3a88) 00000080:00200000:2.0:1787826191.347691:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1796:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.347696:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.347717:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1796:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.347721:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1796:0x0](ffff9181c7bab248) 00000080:00200000:2.0:1787826191.347771:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1797:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.347777:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.347785:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1797:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.347789:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1797:0x0](ffff9181c7bb9988) 00000080:00200000:2.0:1787826191.347817:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1798:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.347822:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.347829:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1798:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.347833:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1798:0x0](ffff9181c7baaa08) 00000080:00200000:2.0:1787826191.347868:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x1799:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.347874:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.347880:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x1799:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.347885:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1799:0x0](ffff9181c7bacb08) 00000080:00200000:2.0:1787826191.347906:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x179a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.347911:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.347918:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x179a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.347921:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x179a:0x0](ffff9181c7bb3248) 00000080:00200000:2.0:1787826191.347947:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x179b:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.347953:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.347959:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x179b:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.347963:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x179b:0x0](ffff9181c7ba8908) 00000080:00200000:2.0:1787826191.347985:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x179c:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.347990:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.347997:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x179c:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.348001:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x179c:0x0](ffff9181c7bb2a08) 00000080:00200000:2.0:1787826191.348024:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x179d:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.348029:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.348036:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x179d:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.348040:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x179d:0x0](ffff9181c7bac2c8) 00000080:00200000:2.0:1787826191.348063:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x179e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.348067:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.348073:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x179e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.348077:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x179e:0x0](ffff9181c7bb4b08) 00000080:00200000:2.0:1787826191.348100:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x179f:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.348104:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.348111:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x179f:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.348115:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x179f:0x0](ffff9181c7bad348) 00000080:00200000:2.0:1787826191.348137:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17a0:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.348142:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.348148:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17a0:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.348152:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17a0:0x0](ffff9181c7bb0908) 00000080:00200000:2.0:1787826191.348277:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17a1:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.348282:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.348290:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17a1:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.348294:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17a1:0x0](ffff9181c7ba5348) 00000080:00200000:2.0:1787826191.348317:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17a2:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.348321:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.348329:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17a2:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.348333:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17a2:0x0](ffff9181c7ba6c08) 00000080:00200000:2.0:1787826191.348355:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17a3:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.348358:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.348364:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17a3:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.348368:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17a3:0x0](ffff9181c7baec08) 00000080:00200000:2.0:1787826191.348387:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17a4:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.348390:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.348396:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17a4:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.348400:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17a4:0x0](ffff9181c7ba00c8) 00000080:00200000:2.0:1787826191.348419:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17a5:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.348423:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.348429:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17a5:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.348432:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17a5:0x0](ffff9181c7bb42c8) 00000080:00200000:2.0:1787826191.348449:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17a6:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.348453:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.348458:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17a6:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.348461:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17a6:0x0](ffff9181c7bbdb88) 00000080:00200000:2.0:1787826191.348481:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17a7:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.348486:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.348491:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17a7:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.348495:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17a7:0x0](ffff9181c7ba1148) 00000080:00200000:2.0:1787826191.348666:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17a8:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.348674:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.348684:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17a8:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.348688:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17a8:0x0](ffff9181c7bb5348) 00000080:00200000:3.0:1787826191.348924:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17a9:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.348931:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.348941:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17a9:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.348945:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17a9:0x0](ffff9181c7ba80c8) 00000080:00200000:3.0:1787826191.348969:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17aa:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.348974:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.348980:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17aa:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.348983:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17aa:0x0](ffff9181c7ba9148) 00000080:00200000:3.0:1787826191.349002:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17ab:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.349006:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.349013:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17ab:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.349016:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17ab:0x0](ffff9181c7bc00c8) 00000080:00200000:3.0:1787826191.349036:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17ac:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.349041:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.349047:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17ac:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.349051:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17ac:0x0](ffff9181c7bb6c08) 00000080:00200000:3.0:1787826191.349070:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17ad:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.349074:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.349080:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17ad:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.349084:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17ad:0x0](ffff9181c7bb00c8) 00000080:00200000:3.0:1787826191.349104:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17ae:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.349108:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.349115:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17ae:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.349119:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17ae:0x0](ffff9181c7bc1148) 00000080:00200000:3.0:1787826191.349143:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17af:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.349148:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.349154:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17af:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.349215:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17af:0x0](ffff9181c7bb1148) 00000080:00200000:3.0:1787826191.349258:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17b0:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.349264:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.349273:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17b0:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.349276:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17b0:0x0](ffff9181c7bc63c8) 00000080:00200000:3.0:1787826191.349299:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17b1:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.349303:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.349309:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17b1:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.349313:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17b1:0x0](ffff9181c7bc7448) 00000080:00200000:3.0:1787826191.349336:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17b2:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.349340:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.349347:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17b2:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.349350:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17b2:0x0](ffff9181c7bbba88) 00000080:00200000:3.0:1787826191.349373:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17b3:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.349378:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.349384:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17b3:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.349388:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17b3:0x0](ffff9181c7bc21c8) 00000080:00200000:3.0:1787826191.349415:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17b4:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.349420:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.349427:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17b4:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.349430:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17b4:0x0](ffff9181c7bb63c8) 00000080:00200000:3.0:1787826191.349450:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17b5:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.349454:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.349459:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17b5:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.349463:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17b5:0x0](ffff9181c7ba63c8) 00000080:00200000:3.0:1787826191.349484:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17b6:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.349489:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.349495:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17b6:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.349498:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17b6:0x0](ffff9181c7bb7448) 00000080:00200000:3.0:1787826191.349517:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17b7:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.349522:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.349528:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17b7:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.349531:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17b7:0x0](ffff9181c7bccb08) 00000080:00200000:3.0:1787826191.349550:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17b8:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.349555:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.349561:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17b8:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.349564:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17b8:0x0](ffff9181c7bc8908) 00000080:00200000:3.0:1787826191.349661:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17b9:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.349666:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.349673:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17b9:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.349677:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17b9:0x0](ffff9181c7bcc2c8) 00000080:00200000:3.0:1787826191.349695:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17ba:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826191.350419:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826191.350437:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17ba:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826191.350442:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17ba:0x0](ffff9181c7bcd348) 00000080:00200000:2.0:1787826191.351107:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17bb:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.351116:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.351128:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17bb:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.351133:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17bb:0x0](ffff9181c7bc1988) 00000080:00200000:2.0:1787826191.351237:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17bc:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.351246:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.351256:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17bc:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.351260:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17bc:0x0](ffff9181c7ba7448) 00000080:00200000:2.0:1787826191.351295:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17bd:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.351301:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.351310:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17bd:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.351314:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17bd:0x0](ffff9181c7bbb248) 00000080:00200000:2.0:1787826191.351341:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17be:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.351346:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.351353:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17be:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.351357:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17be:0x0](ffff9181c7bbaa08) 00000080:00200000:2.0:1787826191.351378:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17bf:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.351383:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.351390:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17bf:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.351394:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17bf:0x0](ffff9181c7bcec08) 00000080:00200000:2.0:1787826191.351415:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17c0:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.351420:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.351427:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17c0:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.351431:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17c0:0x0](ffff9181c7bc5b88) 00000080:00200000:2.0:1787826191.351455:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17c1:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.351459:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.351467:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17c1:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.351471:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17c1:0x0](ffff9181c7ba21c8) 00000080:00200000:2.0:1787826191.351493:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17c2:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.351497:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.351505:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17c2:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.351508:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17c2:0x0](ffff9181c7bc3a88) 00000080:00200000:2.0:1787826191.351532:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17c3:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.351537:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.351543:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17c3:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.351548:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17c3:0x0](ffff9181c7bc80c8) 00000080:00200000:2.0:1787826191.351568:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17c4:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.351573:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.351580:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17c4:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.351584:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17c4:0x0](ffff9181c7ba1988) 00000080:00200000:2.0:1787826191.351608:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17c5:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.351612:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.351619:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17c5:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.351623:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17c5:0x0](ffff9181c7bc3248) 00000080:00200000:2.0:1787826191.351646:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17c6:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.351650:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.351658:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17c6:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.351662:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17c6:0x0](ffff9181c7bc2a08) 00000080:00200000:2.0:1787826191.351686:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17c7:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.351691:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.351698:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17c7:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.351702:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17c7:0x0](ffff9181c7bbcb08) 00000080:00200000:2.0:1787826191.351727:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17c8:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.351732:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.351739:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17c8:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.352230:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17c8:0x0](ffff9181c7bc9148) 00000080:00200000:2.0:1787826191.352259:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17c9:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.352264:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.352273:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17c9:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.352277:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17c9:0x0](ffff9181c7bce3c8) 00000080:00200000:2.0:1787826191.352296:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17ca:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.352300:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.352308:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17ca:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.352312:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17ca:0x0](ffff9181c7bb8908) 00000080:00200000:2.0:1787826191.352336:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17cb:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.352340:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.352347:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17cb:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.352351:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17cb:0x0](ffff9181c7bcf448) 00000080:00200000:2.0:1787826191.352431:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17cc:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.352436:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.352443:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17cc:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.352447:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17cc:0x0](ffff9181c7bca1c8) 00000080:00200000:2.0:1787826191.352469:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17cd:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.352474:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.352480:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17cd:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.352485:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17cd:0x0](ffff9181c7ba5b88) 00000080:00200000:2.0:1787826191.352506:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17ce:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.352511:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.352517:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17ce:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.352522:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17ce:0x0](ffff9181c7bc4b08) 00000080:00200000:2.0:1787826191.352544:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17cf:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.352548:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.352554:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17cf:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.352558:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17cf:0x0](ffff9181c7bc9988) 00000080:00200000:2.0:1787826191.352579:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17d0:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.352584:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.352591:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17d0:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.352595:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17d0:0x0](ffff9181c7bc0908) 00000080:00200000:2.0:1787826191.352614:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17d1:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.352620:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.352627:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17d1:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.352631:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17d1:0x0](ffff9181c7bbc2c8) 00000080:00200000:2.0:1787826191.352651:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17d2:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.352656:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.352662:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17d2:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.352665:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17d2:0x0](ffff9181c7bbd348) 00000080:00200000:2.0:1787826191.352687:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17d3:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.352691:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.352698:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17d3:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.355437:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17d3:0x0](ffff9181c7bc42c8) 00000080:00200000:2.0:1787826191.355509:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17d4:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.355519:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.355541:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17d4:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.355546:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17d4:0x0](ffff9181c7bbec08) 00000080:00200000:2.0:1787826191.355572:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17d5:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826191.355578:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826191.355585:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17d5:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826191.355589:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17d5:0x0](ffff9181c7bb80c8) 00000080:00200000:0.0:1787826191.356385:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17d6:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.356394:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.356406:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17d6:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.356411:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17d6:0x0](ffff9181c7bcdb88) 00000080:00200000:0.0:1787826191.356443:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17d7:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.356448:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.356455:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17d7:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.356459:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17d7:0x0](ffff9181c7bc5348) 00000080:00200000:0.0:1787826191.356482:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17d8:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.356487:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.356494:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17d8:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.356498:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17d8:0x0](ffff9181c7bcba88) 00000080:00200000:0.0:1787826191.356521:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17d9:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.356526:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.356532:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17d9:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.356536:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17d9:0x0](ffff9181c7bc6c08) 00000080:00200000:0.0:1787826191.356573:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17da:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.356578:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.356587:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17da:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.356591:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17da:0x0](ffff9181c7bcb248) 00000080:00200000:0.0:1787826191.357697:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17db:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.357830:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.357856:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17db:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.357862:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17db:0x0](ffff9181c7bb9148) 00000080:00200000:0.0:1787826191.357906:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17dc:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.357912:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.357921:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17dc:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.357925:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17dc:0x0](ffff9181c7bbe3c8) 00000080:00200000:0.0:1787826191.357958:0:159403:0:(vvp_io.c:1826:vvp_io_init()) [0x20001a9e3:0x17dd:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1787826191.357964:0:159403:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1787826191.357972:0:159403:0:(vvp_io.c:304:vvp_io_fini()) [0x20001a9e3:0x17dd:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826191.357976:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17dd:0x0](ffff9181c7bbf448) 00000080:00200000:0.0:1787826191.358007:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2400007ed:0xa85:0x0](ffff9181c7bd42c8) 00000080:00200000:0.0:1787826191.358018:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x17e0:0x0](ffff9181c7bcaa08) 00000002:00100000:0.0:1787826191.358077:0:159403:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181c3573480 x1874663041393408/t330712482189(330712482189) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 520/608 e 0 to 0 dl 1787826186 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:0 00000020:00100000:0.0:1787826191.358089:0:159403:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c4d2b000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:0.0:1787826191.358094:0:159403:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c55a5180 x1874663042981376/t0(0) o35->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:0.0:1787826191.358115:0:159403:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job umount:a9ce6693-0634-46c5-bf21-5ca7d026d44e:159403:1874663042981376:192.168.203.147@tcp:35:umount.0 00000100:00100000:2.0:1787826191.360112:0:159403:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c7749e40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826191.364253:0:159403:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job umount:a9ce6693-0634-46c5-bf21-5ca7d026d44e:159403:1874663042981376:192.168.203.147@tcp:35:umount.0 00000002:00100000:2.0:1787826191.364291:0:159403:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181c3573480 x1874663041393408/t330712482189(330712482189) o101->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 520/608 e 0 to 0 dl 1787826186 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1787826191.364333:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1:0x0](ffff9181c7bd5348) 00000080:00200000:2.0:1787826191.364350:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1:0x0](ffff9181c7bd6c08) 00000080:00200000:2.0:1787826191.364359:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x4:0x0](ffff9181c7bd00c8) 00000080:00200000:2.0:1787826191.364367:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x4:0x0](ffff9181c7bd1148) 00000080:00200000:2.0:1787826191.364374:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x5:0x0](ffff9181c7bd63c8) 00000080:00200000:2.0:1787826191.364382:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x5:0x0](ffff9181c7bd7448) 00000080:00200000:2.0:1787826191.364391:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x6:0x0](ffff9181c7bde3c8) 00000080:00200000:2.0:1787826191.364399:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x6:0x0](ffff9181c7bdf448) 00000080:00200000:2.0:1787826191.364407:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x7:0x0](ffff9181c7bd1988) 00000080:00200000:2.0:1787826191.364414:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x7:0x0](ffff9181c7bd5b88) 00000080:00200000:2.0:1787826191.364421:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x8:0x0](ffff9181c7bd21c8) 00000080:00200000:2.0:1787826191.364428:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x8:0x0](ffff9181c7bd3a88) 00000080:00200000:2.0:1787826191.364435:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x9:0x0](ffff9181c7bd3248) 00000080:00200000:2.0:1787826191.364442:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x9:0x0](ffff9181c7bd2a08) 00000080:00200000:2.0:1787826191.364449:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xa:0x0](ffff9181c7bda1c8) 00000080:00200000:2.0:1787826191.364457:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xa:0x0](ffff9181c7bd9988) 00000080:00200000:2.0:1787826191.364463:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xb:0x0](ffff9181c7bddb88) 00000080:00200000:2.0:1787826191.364471:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xb:0x0](ffff9181c7bdba88) 00000080:00200000:2.0:1787826191.364478:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xc:0x0](ffff9181c7bd4b08) 00000080:00200000:2.0:1787826191.364485:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xc:0x0](ffff9181c7be7448) 00000080:00200000:2.0:1787826191.364493:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xd:0x0](ffff9181c7bdb248) 00000080:00200000:2.0:1787826191.364501:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xd:0x0](ffff9181c7bdaa08) 00000080:00200000:2.0:1787826191.364508:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xe:0x0](ffff9181c7be21c8) 00000080:00200000:2.0:1787826191.364516:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xe:0x0](ffff9181c7be1988) 00000080:00200000:2.0:1787826191.364524:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xf:0x0](ffff9181c7ba3a88) 00000080:00200000:2.0:1787826191.364532:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xf:0x0](ffff9181c7be9988) 00000080:00200000:2.0:1787826191.364540:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x10:0x0](ffff9181c7be5b88) 00000080:00200000:2.0:1787826191.364548:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x10:0x0](ffff9181c7be3a88) 00000080:00200000:2.0:1787826191.364556:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x11:0x0](ffff9181c7bd8908) 00000080:00200000:2.0:1787826191.364565:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x11:0x0](ffff9181c7bdc2c8) 00000080:00200000:2.0:1787826191.364573:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x12:0x0](ffff9181c7bdcb08) 00000080:00200000:2.0:1787826191.364581:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x12:0x0](ffff9181c7bdd348) 00000080:00200000:2.0:1787826191.364588:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x13:0x0](ffff9181c7be4b08) 00000080:00200000:2.0:1787826191.364595:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x13:0x0](ffff9181c7be0908) 00000080:00200000:2.0:1787826191.364602:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x14:0x0](ffff9181c7be5348) 00000080:00200000:3.0:1787826191.365091:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x14:0x0](ffff9181c7be6c08) 00000080:00200000:3.0:1787826191.365106:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x15:0x0](ffff9181c7beba88) 00000080:00200000:3.0:1787826191.365114:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x15:0x0](ffff9181c7beb248) 00000080:00200000:3.0:1787826191.365122:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x16:0x0](ffff9181c7bba1c8) 00000080:00200000:3.0:1787826191.365129:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x16:0x0](ffff9181c7bf3a88) 00000080:00200000:3.0:1787826191.365136:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x17:0x0](ffff9181c7be00c8) 00000080:00200000:3.0:1787826191.365144:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x17:0x0](ffff9181c7be1148) 00000080:00200000:3.0:1787826191.365161:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x18:0x0](ffff9181c7be42c8) 00000080:00200000:3.0:1787826191.365170:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x18:0x0](ffff9181c7be2a08) 00000080:00200000:3.0:1787826191.365177:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x19:0x0](ffff9181c7be63c8) 00000080:00200000:3.0:1787826191.365186:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x19:0x0](ffff9181c7be3248) 00000080:00200000:3.0:1787826191.365193:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1a:0x0](ffff9181c7bd9148) 00000080:00200000:3.0:1787826191.365202:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1a:0x0](ffff9181c7bd80c8) 00000080:00200000:3.0:1787826191.365209:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1b:0x0](ffff9181c7beaa08) 00000080:00200000:3.0:1787826191.365217:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1b:0x0](ffff9181c7becb08) 00000080:00200000:3.0:1787826191.365225:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1c:0x0](ffff9181c7bf3248) 00000080:00200000:3.0:1787826191.365232:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1c:0x0](ffff9181c7bf2a08) 00000080:00200000:3.0:1787826191.365239:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1d:0x0](ffff9181c7bf4b08) 00000080:00200000:3.0:1787826191.365247:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1d:0x0](ffff9181c7bf0908) 00000080:00200000:3.0:1787826191.365254:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1e:0x0](ffff9181c7ba3248) 00000080:00200000:3.0:1787826191.365262:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1e:0x0](ffff9181c7bd0908) 00000080:00200000:3.0:1787826191.365268:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1f:0x0](ffff9181c7bf9148) 00000080:00200000:3.0:1787826191.365276:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1f:0x0](ffff9181c7bfe3c8) 00000080:00200000:3.0:1787826191.365283:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x20:0x0](ffff9181c7bff448) 00000080:00200000:3.0:1787826191.365291:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x20:0x0](ffff9181c7bfa1c8) 00000080:00200000:3.0:1787826191.365299:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x21:0x0](ffff9181c7bf9988) 00000080:00200000:3.0:1787826191.365306:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x21:0x0](ffff9181c7bfdb88) 00000080:00200000:3.0:1787826191.365313:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x22:0x0](ffff9181c7bf5348) 00000080:00200000:3.0:1787826191.365321:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x22:0x0](ffff9181c7bf6c08) 00000080:00200000:3.0:1787826191.365328:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x23:0x0](ffff9181c7bdec08) 00000080:00200000:3.0:1787826191.365336:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x23:0x0](ffff9181c8001988) 00000080:00200000:3.0:1787826191.365343:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x24:0x0](ffff9181c7bfb248) 00000080:00200000:3.0:1787826191.365351:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x24:0x0](ffff9181c7bfaa08) 00000080:00200000:3.0:1787826191.365358:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x25:0x0](ffff9181c8005b88) 00000080:00200000:3.0:1787826191.365367:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x25:0x0](ffff9181c8003a88) 00000080:00200000:3.0:1787826191.365374:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x26:0x0](ffff9181c7bfcb08) 00000080:00200000:3.0:1787826191.365381:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x26:0x0](ffff9181c7bf8908) 00000080:00200000:3.0:1787826191.365389:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x27:0x0](ffff9181c7bed348) 00000080:00200000:3.0:1787826191.365397:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x27:0x0](ffff9181c7beec08) 00000080:00200000:3.0:1787826191.365403:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x28:0x0](ffff9181c7bfd348) 00000080:00200000:3.0:1787826191.365410:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x28:0x0](ffff9181c7bfec08) 00000080:00200000:3.0:1787826191.365417:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x29:0x0](ffff9181c7bfc2c8) 00000080:00200000:3.0:1787826191.365425:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x29:0x0](ffff9181c7bfba88) 00000080:00200000:3.0:1787826191.365432:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2a:0x0](ffff9181c800db88) 00000080:00200000:3.0:1787826191.365440:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2a:0x0](ffff9181c800ba88) 00000080:00200000:3.0:1787826191.365448:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2b:0x0](ffff9181c7bf1148) 00000080:00200000:3.0:1787826191.365456:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2b:0x0](ffff9181c7bf63c8) 00000080:00200000:3.0:1787826191.365464:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2c:0x0](ffff9181c800aa08) 00000080:00200000:3.0:1787826191.365472:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2c:0x0](ffff9181c800cb08) 00000080:00200000:3.0:1787826191.365479:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2d:0x0](ffff9181c800c2c8) 00000080:00200000:3.0:1787826191.365486:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2d:0x0](ffff9181c800d348) 00000080:00200000:3.0:1787826191.365494:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2e:0x0](ffff9181c7bf7448) 00000080:00200000:3.0:1787826191.365502:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2e:0x0](ffff9181c7bf21c8) 00000080:00200000:3.0:1787826191.365509:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2f:0x0](ffff9181c7be80c8) 00000080:00200000:3.0:1787826191.365517:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2f:0x0](ffff9181c7be9148) 00000080:00200000:3.0:1787826191.365525:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x30:0x0](ffff9181c7bee3c8) 00000080:00200000:3.0:1787826191.365533:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x30:0x0](ffff9181c7bef448) 00000080:00200000:3.0:1787826191.365542:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x31:0x0](ffff9181c8000908) 00000080:00200000:3.0:1787826191.365550:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x31:0x0](ffff9181c80042c8) 00000080:00200000:3.0:1787826191.365558:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x32:0x0](ffff9181c80080c8) 00000080:00200000:3.0:1787826191.365566:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x32:0x0](ffff9181c8009148) 00000080:00200000:3.0:1787826191.365573:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x33:0x0](ffff9181c8006c08) 00000080:00200000:3.0:1787826191.365581:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x33:0x0](ffff9181c80000c8) 00000080:00200000:3.0:1787826191.365589:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x34:0x0](ffff9181c8001148) 00000080:00200000:3.0:1787826191.365596:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x34:0x0](ffff9181c80063c8) 00000080:00200000:3.0:1787826191.365604:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x35:0x0](ffff9181c8009988) 00000080:00200000:3.0:1787826191.365730:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x35:0x0](ffff9181c800f448) 00000080:00200000:3.0:1787826191.365776:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x36:0x0](ffff9181c7bf1988) 00000080:00200000:3.0:1787826191.365786:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x36:0x0](ffff9181c7bf5b88) 00000080:00200000:3.0:1787826191.365793:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x37:0x0](ffff9181c7bf00c8) 00000080:00200000:3.0:1787826191.365800:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x37:0x0](ffff9181c7bf42c8) 00000080:00200000:3.0:1787826191.365807:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x38:0x0](ffff9181c80021c8) 00000080:00200000:3.0:1787826191.365816:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x38:0x0](ffff9181c8007448) 00000080:00200000:3.0:1787826191.365823:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x39:0x0](ffff9181c800e3c8) 00000080:00200000:3.0:1787826191.365830:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x39:0x0](ffff9181c800ec08) 00000080:00200000:3.0:1787826191.365838:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x3a:0x0](ffff9181c80100c8) 00000080:00200000:3.0:1787826191.365854:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x3a:0x0](ffff9181c8011148) 00000080:00200000:3.0:1787826191.365862:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x3b:0x0](ffff9181c7bea1c8) 00000080:00200000:3.0:1787826191.365869:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x3b:0x0](ffff9181c7bec2c8) 00000080:00200000:3.0:1787826191.365875:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x3c:0x0](ffff9181c80163c8) 00000080:00200000:3.0:1787826191.365882:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x3c:0x0](ffff9181c8017448) 00000080:00200000:3.0:1787826191.365888:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x3d:0x0](ffff9181c8011988) 00000080:00200000:3.0:1787826191.365895:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x3d:0x0](ffff9181c8015b88) 00000080:00200000:3.0:1787826191.365911:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x3e:0x0](ffff9181c8003248) 00000080:00200000:3.0:1787826191.365918:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x3e:0x0](ffff9181c8002a08) 00000080:00200000:3.0:1787826191.365925:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x3f:0x0](ffff9181c8005348) 00000080:00200000:3.0:1787826191.365933:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x3f:0x0](ffff9181c801b248) 00000080:00200000:3.0:1787826191.365940:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x40:0x0](ffff9181c7bedb88) 00000080:00200000:3.0:1787826191.365948:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x40:0x0](ffff9181c7be8908) 00000080:00200000:3.0:1787826191.365954:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x41:0x0](ffff9181c8008908) 00000080:00200000:3.0:1787826191.365963:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x41:0x0](ffff9181c800a1c8) 00000080:00200000:3.0:1787826191.365970:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x42:0x0](ffff9181c8013a88) 00000080:00200000:3.0:1787826191.365978:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x42:0x0](ffff9181c8013248) 00000080:00200000:3.0:1787826191.365985:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x43:0x0](ffff9181c8022a08) 00000080:00200000:3.0:1787826191.365992:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x43:0x0](ffff9181c8024b08) 00000080:00200000:3.0:1787826191.366000:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x44:0x0](ffff9181c7bf80c8) 00000080:00200000:3.0:1787826191.366008:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x44:0x0](ffff9181c802f448) 00000080:00200000:3.0:1787826191.366016:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x45:0x0](ffff9181c802a1c8) 00000080:00200000:3.0:1787826191.366023:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x45:0x0](ffff9181c8029988) 00000080:00200000:3.0:1787826191.366030:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x46:0x0](ffff9181c802ba88) 00000080:00200000:3.0:1787826191.366036:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x46:0x0](ffff9181c802b248) 00000080:00200000:3.0:1787826191.366043:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x47:0x0](ffff9181c801aa08) 00000080:00200000:3.0:1787826191.366052:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x47:0x0](ffff9181c801cb08) 00000080:00200000:3.0:1787826191.366059:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x48:0x0](ffff9181c802cb08) 00000080:00200000:3.0:1787826191.366067:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x48:0x0](ffff9181c8028908) 00000080:00200000:3.0:1787826191.366075:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x49:0x0](ffff9181c8020908) 00000080:00200000:3.0:1787826191.366083:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x49:0x0](ffff9181c80242c8) 00000080:00200000:3.0:1787826191.366089:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x4a:0x0](ffff9181c802d348) 00000080:00200000:3.0:1787826191.366098:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x4a:0x0](ffff9181c802ec08) 00000080:00200000:3.0:1787826191.366106:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x4b:0x0](ffff9181c801c2c8) 00000080:00200000:3.0:1787826191.366114:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x4b:0x0](ffff9181c801d348) 00000080:00200000:3.0:1787826191.366121:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x4c:0x0](ffff9181c8025348) 00000080:00200000:3.0:1787826191.366128:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x4c:0x0](ffff9181c8026c08) 00000080:00200000:3.0:1787826191.366135:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x4d:0x0](ffff9181c801ec08) 00000080:00200000:3.0:1787826191.366142:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x4d:0x0](ffff9181c80180c8) 00000080:00200000:3.0:1787826191.366150:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x4e:0x0](ffff9181c8019148) 00000080:00200000:3.0:1787826191.367963:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x4e:0x0](ffff9181c801e3c8) 00000080:00200000:3.0:1787826191.367975:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x4f:0x0](ffff9181c80200c8) 00000080:00200000:3.0:1787826191.367987:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x4f:0x0](ffff9181c8021148) 00000080:00200000:3.0:1787826191.367995:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x50:0x0](ffff9181c8029148) 00000080:00200000:3.0:1787826191.368008:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x50:0x0](ffff9181c80280c8) 00000080:00200000:3.0:1787826191.368017:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x51:0x0](ffff9181c802c2c8) 00000080:00200000:3.0:1787826191.368025:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x51:0x0](ffff9181c802aa08) 00000080:00200000:3.0:1787826191.368032:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x52:0x0](ffff9181c80142c8) 00000080:00200000:3.0:1787826191.368042:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x52:0x0](ffff9181c8015348) 00000080:00200000:3.0:1787826191.368050:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x53:0x0](ffff9181c8010908) 00000080:00200000:3.0:1787826191.368058:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x53:0x0](ffff9181c8016c08) 00000080:00200000:3.0:1787826191.368066:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x54:0x0](ffff9181c8019988) 00000080:00200000:3.0:1787826191.368075:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x54:0x0](ffff9181c801db88) 00000080:00200000:3.0:1787826191.368083:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x55:0x0](ffff9181c802db88) 00000080:00200000:3.0:1787826191.368092:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x55:0x0](ffff9181c802e3c8) 00000080:00200000:3.0:1787826191.368099:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x56:0x0](ffff9181c8014b08) 00000080:00200000:3.0:1787826191.368108:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x56:0x0](ffff9181c8012a08) 00000080:00200000:3.0:1787826191.368116:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x57:0x0](ffff9181c80321c8) 00000080:00200000:3.0:1787826191.368125:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x57:0x0](ffff9181c8031988) 00000080:00200000:3.0:1787826191.368133:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x58:0x0](ffff9181c8035b88) 00000080:00200000:3.0:1787826191.368140:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x58:0x0](ffff9181c8033a88) 00000080:00200000:3.0:1787826191.368148:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x59:0x0](ffff9181c8032a08) 00000080:00200000:3.0:1787826191.368324:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x59:0x0](ffff9181c8034b08) 00000080:00200000:3.0:1787826191.368336:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x5a:0x0](ffff9181c8030908) 00000080:00200000:3.0:1787826191.368343:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x5a:0x0](ffff9181c80342c8) 00000080:00200000:3.0:1787826191.368350:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x5b:0x0](ffff9181c803e3c8) 00000080:00200000:3.0:1787826191.368359:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x5b:0x0](ffff9181c803f448) 00000080:00200000:3.0:1787826191.368367:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x5c:0x0](ffff9181c80263c8) 00000080:00200000:3.0:1787826191.368375:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x5c:0x0](ffff9181c8027448) 00000080:00200000:3.0:1787826191.368383:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x5d:0x0](ffff9181c801ba88) 00000080:00200000:3.0:1787826191.368392:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x5d:0x0](ffff9181c801a1c8) 00000080:00200000:3.0:1787826191.368400:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x5e:0x0](ffff9181c8033248) 00000080:00200000:3.0:1787826191.368407:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x5e:0x0](ffff9181c8035348) 00000080:00200000:3.0:1787826191.368413:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x5f:0x0](ffff9181c80300c8) 00000080:00200000:3.0:1787826191.368421:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x5f:0x0](ffff9181c8031148) 00000080:00200000:3.0:1787826191.368427:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x60:0x0](ffff9181c8036c08) 00000080:00200000:3.0:1787826191.368435:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x60:0x0](ffff9181c80363c8) 00000080:00200000:3.0:1787826191.368441:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x61:0x0](ffff9181c80121c8) 00000080:00200000:3.0:1787826191.368448:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x61:0x0](ffff9181c8045b88) 00000080:00200000:3.0:1787826191.368455:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x62:0x0](ffff9181c8043a88) 00000080:00200000:3.0:1787826191.368462:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x62:0x0](ffff9181c8043248) 00000080:00200000:3.0:1787826191.368470:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x63:0x0](ffff9181c8018908) 00000080:00200000:3.0:1787826191.368478:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x63:0x0](ffff9181c801f448) 00000080:00200000:3.0:1787826191.368486:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x64:0x0](ffff9181c803db88) 00000080:00200000:3.0:1787826191.368494:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x64:0x0](ffff9181c803ba88) 00000080:00200000:3.0:1787826191.368502:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x65:0x0](ffff9181c800b248) 00000080:00200000:3.0:1787826191.368510:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x65:0x0](ffff9181c8004b08) 00000080:00200000:3.0:1787826191.368517:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x66:0x0](ffff9181c8042a08) 00000080:00200000:3.0:1787826191.368524:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x66:0x0](ffff9181c8044b08) 00000080:00200000:3.0:1787826191.368532:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x67:0x0](ffff9181c8021988) 00000080:00200000:3.0:1787826191.368540:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x67:0x0](ffff9181c8025b88) 00000080:00200000:3.0:1787826191.368548:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x68:0x0](ffff9181c8040908) 00000080:00200000:3.0:1787826191.368557:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x68:0x0](ffff9181c80442c8) 00000080:00200000:3.0:1787826191.368564:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x69:0x0](ffff9181c8045348) 00000080:00200000:3.0:1787826191.368571:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x69:0x0](ffff9181c8046c08) 00000080:00200000:3.0:1787826191.368578:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x6a:0x0](ffff9181c8023248) 00000080:00200000:3.0:1787826191.368584:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x6a:0x0](ffff9181c8023a88) 00000080:00200000:3.0:1787826191.368592:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x6b:0x0](ffff9181c80221c8) 00000080:00200000:3.0:1787826191.368600:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x6b:0x0](ffff9181c8053248) 00000080:00200000:3.0:1787826191.368619:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x6c:0x0](ffff9181c803aa08) 00000080:00200000:3.0:1787826191.368628:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x6c:0x0](ffff9181c803cb08) 00000080:00200000:3.0:1787826191.368636:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x6d:0x0](ffff9181c803c2c8) 00000080:00200000:3.0:1787826191.368643:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x6d:0x0](ffff9181c803d348) 00000080:00200000:3.0:1787826191.368650:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x6e:0x0](ffff9181c803ec08) 00000080:00200000:3.0:1787826191.368658:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x6e:0x0](ffff9181c80380c8) 00000080:00200000:3.0:1787826191.368665:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x6f:0x0](ffff9181c8038908) 00000080:00200000:3.0:1787826191.368673:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x6f:0x0](ffff9181c803b248) 00000080:00200000:3.0:1787826191.368682:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x70:0x0](ffff9181c8048908) 00000080:00200000:3.0:1787826191.368690:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x70:0x0](ffff9181c804c2c8) 00000080:00200000:3.0:1787826191.368697:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x71:0x0](ffff9181c8052a08) 00000080:00200000:3.0:1787826191.368717:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x71:0x0](ffff9181c8054b08) 00000080:00200000:3.0:1787826191.368724:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x72:0x0](ffff9181c804d348) 00000080:00200000:3.0:1787826191.368731:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x72:0x0](ffff9181c804ec08) 00000080:00200000:3.0:1787826191.368739:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x73:0x0](ffff9181c8050908) 00000080:00200000:3.0:1787826191.370253:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x73:0x0](ffff9181c80542c8) 00000080:00200000:3.0:1787826191.370270:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x74:0x0](ffff9181c8047448) 00000080:00200000:3.0:1787826191.370280:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x74:0x0](ffff9181c80421c8) 00000080:00200000:3.0:1787826191.370288:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x75:0x0](ffff9181c8049148) 00000080:00200000:3.0:1787826191.370297:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x75:0x0](ffff9181c804e3c8) 00000080:00200000:3.0:1787826191.370305:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x76:0x0](ffff9181c804a1c8) 00000080:00200000:3.0:1787826191.370313:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x76:0x0](ffff9181c8049988) 00000080:00200000:3.0:1787826191.370321:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x77:0x0](ffff9181c8041988) 00000080:00200000:3.0:1787826191.370328:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x77:0x0](ffff9181c80463c8) 00000080:00200000:3.0:1787826191.370335:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x78:0x0](ffff9181c80500c8) 00000080:00200000:3.0:1787826191.370344:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x78:0x0](ffff9181c8051148) 00000080:00200000:3.0:1787826191.370352:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x79:0x0](ffff9181c80400c8) 00000080:00200000:3.0:1787826191.370360:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x79:0x0](ffff9181c8037448) 00000080:00200000:3.0:1787826191.370368:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x7a:0x0](ffff9181c8041148) 00000080:00200000:3.0:1787826191.370376:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x7a:0x0](ffff9181c805a1c8) 00000080:00200000:3.0:1787826191.370383:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x7b:0x0](ffff9181c8059988) 00000080:00200000:3.0:1787826191.370392:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x7b:0x0](ffff9181c805db88) 00000080:00200000:3.0:1787826191.370399:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x7c:0x0](ffff9181c805ba88) 00000080:00200000:3.0:1787826191.370407:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x7c:0x0](ffff9181c805b248) 00000080:00200000:3.0:1787826191.370414:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x7d:0x0](ffff9181c804f448) 00000080:00200000:3.0:1787826191.370422:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x7d:0x0](ffff9181c80480c8) 00000080:00200000:3.0:1787826191.370429:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x7e:0x0](ffff9181c805aa08) 00000080:00200000:3.0:1787826191.370436:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x7e:0x0](ffff9181c805cb08) 00000080:00200000:3.0:1787826191.370443:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x7f:0x0](ffff9181c804cb08) 00000080:00200000:3.0:1787826191.370452:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x7f:0x0](ffff9181c804aa08) 00000080:00200000:3.0:1787826191.370458:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x80:0x0](ffff9181c803a1c8) 00000080:00200000:3.0:1787826191.370585:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x80:0x0](ffff9181c8039148) 00000080:00200000:3.0:1787826191.370597:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x81:0x0](ffff9181c8039988) 00000080:00200000:3.0:1787826191.370605:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x81:0x0](ffff9181c80600c8) 00000080:00200000:3.0:1787826191.370677:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x82:0x0](ffff9181c805ec08) 00000080:00200000:3.0:1787826191.370685:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x82:0x0](ffff9181c80580c8) 00000080:00200000:3.0:1787826191.370693:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x83:0x0](ffff9181c8061148) 00000080:00200000:3.0:1787826191.370703:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x83:0x0](ffff9181c80663c8) 00000080:00200000:3.0:1787826191.370710:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x84:0x0](ffff9181c80563c8) 00000080:00200000:3.0:1787826191.370718:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x84:0x0](ffff9181c8057448) 00000080:00200000:3.0:1787826191.370725:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x85:0x0](ffff9181c804b248) 00000080:00200000:3.0:1787826191.370733:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x85:0x0](ffff9181c804db88) 00000080:00200000:3.0:1787826191.370740:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x86:0x0](ffff9181c805e3c8) 00000080:00200000:3.0:1787826191.371892:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x86:0x0](ffff9181c805f448) 00000080:00200000:3.0:1787826191.372445:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x87:0x0](ffff9181c8051988) 00000080:00200000:3.0:1787826191.372462:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x87:0x0](ffff9181c8055b88) 00000080:00200000:3.0:1787826191.373002:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x88:0x0](ffff9181c804ba88) 00000080:00200000:3.0:1787826191.373015:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x88:0x0](ffff9181c806c2c8) 00000080:00200000:3.0:1787826191.373023:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x89:0x0](ffff9181c806d348) 00000080:00200000:3.0:1787826191.373031:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x89:0x0](ffff9181c806ec08) 00000080:00200000:3.0:1787826191.373040:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x8a:0x0](ffff9181c8053a88) 00000080:00200000:3.0:1787826191.373049:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x8a:0x0](ffff9181c80521c8) 00000080:00200000:3.0:1787826191.373057:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x8b:0x0](ffff9181c805c2c8) 00000080:00200000:3.0:1787826191.373067:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x8b:0x0](ffff9181c8058908) 00000080:00200000:3.0:1787826191.373075:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x8c:0x0](ffff9181c8067448) 00000080:00200000:3.0:1787826191.373086:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x8c:0x0](ffff9181c80621c8) 00000080:00200000:3.0:1787826191.373095:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x8d:0x0](ffff9181c8069148) 00000080:00200000:3.0:1787826191.373103:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x8d:0x0](ffff9181c806e3c8) 00000080:00200000:3.0:1787826191.373112:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x8e:0x0](ffff9181c8059148) 00000080:00200000:3.0:1787826191.373120:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x8e:0x0](ffff9181c805d348) 00000080:00200000:3.0:1787826191.373129:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x8f:0x0](ffff9181c80763c8) 00000080:00200000:3.0:1787826191.373137:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x8f:0x0](ffff9181c8077448) 00000080:00200000:3.0:1787826191.373154:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x90:0x0](ffff9181c806f448) 00000080:00200000:3.0:1787826191.373162:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x90:0x0](ffff9181c806a1c8) 00000080:00200000:3.0:1787826191.373169:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x91:0x0](ffff9181c806db88) 00000080:00200000:3.0:1787826191.373177:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x91:0x0](ffff9181c806ba88) 00000080:00200000:3.0:1787826191.373185:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x92:0x0](ffff9181c8069988) 00000080:00200000:3.0:1787826191.373193:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x92:0x0](ffff9181c806b248) 00000080:00200000:3.0:1787826191.373199:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x93:0x0](ffff9181c8071988) 00000080:00200000:3.0:1787826191.373208:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x93:0x0](ffff9181c8075b88) 00000080:00200000:3.0:1787826191.373215:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x94:0x0](ffff9181c80721c8) 00000080:00200000:3.0:1787826191.373223:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x94:0x0](ffff9181c8073a88) 00000080:00200000:3.0:1787826191.373231:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x95:0x0](ffff9181c8065b88) 00000080:00200000:3.0:1787826191.373246:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x95:0x0](ffff9181c8063a88) 00000080:00200000:3.0:1787826191.373254:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x96:0x0](ffff9181c8056c08) 00000080:00200000:3.0:1787826191.373262:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x96:0x0](ffff9181c807e3c8) 00000080:00200000:3.0:1787826191.373269:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x97:0x0](ffff9181c8072a08) 00000080:00200000:3.0:1787826191.373277:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x97:0x0](ffff9181c8074b08) 00000080:00200000:3.0:1787826191.373284:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x98:0x0](ffff9181c806cb08) 00000080:00200000:3.0:1787826191.373293:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x98:0x0](ffff9181c8068908) 00000080:00200000:3.0:1787826191.373300:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x99:0x0](ffff9181c80742c8) 00000080:00200000:3.0:1787826191.373308:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x99:0x0](ffff9181c807a1c8) 00000080:00200000:3.0:1787826191.373316:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x9a:0x0](ffff9181c8075348) 00000080:00200000:3.0:1787826191.373322:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x9a:0x0](ffff9181c8076c08) 00000080:00200000:3.0:1787826191.373328:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x9b:0x0](ffff9181c80680c8) 00000080:00200000:3.0:1787826191.373336:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x9b:0x0](ffff9181c8082a08) 00000080:00200000:3.0:1787826191.373344:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x9c:0x0](ffff9181c8080908) 00000080:00200000:3.0:1787826191.373351:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x9c:0x0](ffff9181c80842c8) 00000080:00200000:3.0:1787826191.373359:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x9d:0x0](ffff9181c8070908) 00000080:00200000:3.0:1787826191.373366:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x9d:0x0](ffff9181c8073248) 00000080:00200000:3.0:1787826191.373372:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x9e:0x0](ffff9181c8063248) 00000080:00200000:3.0:1787826191.373381:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x9e:0x0](ffff9181c8062a08) 00000080:00200000:3.0:1787826191.373389:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x9f:0x0](ffff9181c8071148) 00000080:00200000:3.0:1787826191.373396:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x9f:0x0](ffff9181c8055348) 00000080:00200000:3.0:1787826191.373403:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xa0:0x0](ffff9181c807ba88) 00000080:00200000:3.0:1787826191.373415:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xa0:0x0](ffff9181c807b248) 00000080:00200000:3.0:1787826191.373422:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xa1:0x0](ffff9181c80700c8) 00000080:00200000:3.0:1787826191.373429:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xa1:0x0](ffff9181c808e3c8) 00000080:00200000:3.0:1787826191.373437:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xa2:0x0](ffff9181c808a1c8) 00000080:00200000:3.0:1787826191.373444:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xa2:0x0](ffff9181c8089988) 00000080:00200000:3.0:1787826191.373451:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xa3:0x0](ffff9181c808ba88) 00000080:00200000:3.0:1787826191.373458:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xa3:0x0](ffff9181c808b248) 00000080:00200000:3.0:1787826191.373465:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xa4:0x0](ffff9181c8064b08) 00000080:00200000:3.0:1787826191.379716:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xa4:0x0](ffff9181c8060908) 00000080:00200000:3.0:1787826191.379733:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xa5:0x0](ffff9181c80642c8) 00000080:00200000:3.0:1787826191.379774:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xa5:0x0](ffff9181c8065348) 00000080:00200000:3.0:1787826191.379786:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xa6:0x0](ffff9181c8084b08) 00000080:00200000:3.0:1787826191.379798:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xa6:0x0](ffff9181c8085348) 00000080:00200000:3.0:1787826191.379807:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xa7:0x0](ffff9181c808cb08) 00000080:00200000:3.0:1787826191.379816:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xa7:0x0](ffff9181c8088908) 00000080:00200000:3.0:1787826191.379823:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xa8:0x0](ffff9181c808c2c8) 00000080:00200000:3.0:1787826191.379831:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xa8:0x0](ffff9181c808d348) 00000080:00200000:3.0:1787826191.379839:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xa9:0x0](ffff9181c8086c08) 00000080:00200000:3.0:1787826191.379861:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xa9:0x0](ffff9181c80800c8) 00000080:00200000:3.0:1787826191.379869:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xaa:0x0](ffff9181c808ec08) 00000080:00200000:3.0:1787826191.379878:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xaa:0x0](ffff9181c80880c8) 00000080:00200000:3.0:1787826191.379885:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xab:0x0](ffff9181c8089148) 00000080:00200000:3.0:1787826191.379893:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xab:0x0](ffff9181c808aa08) 00000080:00200000:3.0:1787826191.379912:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xac:0x0](ffff9181c80863c8) 00000080:00200000:3.0:1787826191.379921:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xac:0x0](ffff9181c8087448) 00000080:00200000:3.0:1787826191.379928:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xad:0x0](ffff9181c80821c8) 00000080:00200000:3.0:1787826191.379937:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xad:0x0](ffff9181c8081988) 00000080:00200000:3.0:1787826191.379945:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xae:0x0](ffff9181c8085b88) 00000080:00200000:3.0:1787826191.379953:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xae:0x0](ffff9181c8083a88) 00000080:00200000:3.0:1787826191.379960:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xaf:0x0](ffff9181c8083248) 00000080:00200000:3.0:1787826191.379968:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xaf:0x0](ffff9181c8081148) 00000080:00200000:3.0:1787826191.379976:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xb0:0x0](ffff9181c8061988) 00000080:00200000:3.0:1787826191.379984:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xb0:0x0](ffff9181c8066c08) 00000080:00200000:3.0:1787826191.379992:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xb1:0x0](ffff9181c808f448) 00000080:00200000:3.0:1787826191.380001:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xb1:0x0](ffff9181c808db88) 00000080:00200000:3.0:1787826191.380007:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xb2:0x0](ffff9181c8091148) 00000080:00200000:3.0:1787826191.380015:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xb2:0x0](ffff9181c80963c8) 00000080:00200000:3.0:1787826191.380023:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xb3:0x0](ffff9181c80980c8) 00000080:00200000:3.0:1787826191.380030:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xb3:0x0](ffff9181c8099148) 00000080:00200000:3.0:1787826191.380037:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xb4:0x0](ffff9181c80921c8) 00000080:00200000:3.0:1787826191.380044:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xb4:0x0](ffff9181c8091988) 00000080:00200000:3.0:1787826191.380051:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xb5:0x0](ffff9181c8079148) 00000080:00200000:3.0:1787826191.380059:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xb5:0x0](ffff9181c80780c8) 00000080:00200000:3.0:1787826191.380066:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xb6:0x0](ffff9181c8093a88) 00000080:00200000:3.0:1787826191.380074:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xb6:0x0](ffff9181c8093248) 00000080:00200000:3.0:1787826191.380081:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xb7:0x0](ffff9181c806aa08) 00000080:00200000:3.0:1787826191.380089:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xb7:0x0](ffff9181c80a0908) 00000080:00200000:3.0:1787826191.380096:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xb8:0x0](ffff9181c8090908) 00000080:00200000:3.0:1787826191.380104:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xb8:0x0](ffff9181c80942c8) 00000080:00200000:3.0:1787826191.380110:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xb9:0x0](ffff9181c80a42c8) 00000080:00200000:3.0:1787826191.380117:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xb9:0x0](ffff9181c80a5348) 00000080:00200000:3.0:1787826191.380124:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xba:0x0](ffff9181c8094b08) 00000080:00200000:3.0:1787826191.380132:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xba:0x0](ffff9181c8095348) 00000080:00200000:3.0:1787826191.380139:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xbb:0x0](ffff9181c80a6c08) 00000080:00200000:3.0:1787826191.380146:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xbb:0x0](ffff9181c80a00c8) 00000080:00200000:3.0:1787826191.380154:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xbc:0x0](ffff9181c80a1148) 00000080:00200000:3.0:1787826191.380161:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xbc:0x0](ffff9181c80a63c8) 00000080:00200000:3.0:1787826191.380167:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xbd:0x0](ffff9181c809a1c8) 00000080:00200000:3.0:1787826191.380175:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xbd:0x0](ffff9181c8099988) 00000080:00200000:3.0:1787826191.380181:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xbe:0x0](ffff9181c8096c08) 00000080:00200000:3.0:1787826191.380189:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xbe:0x0](ffff9181c8092a08) 00000080:00200000:3.0:1787826191.380195:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xbf:0x0](ffff9181c809f448) 00000080:00200000:3.0:1787826191.380203:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xbf:0x0](ffff9181c809db88) 00000080:00200000:3.0:1787826191.380210:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xc0:0x0](ffff9181c809ba88) 00000080:00200000:3.0:1787826191.380217:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xc0:0x0](ffff9181c809b248) 00000080:00200000:3.0:1787826191.380224:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xc1:0x0](ffff9181c80a21c8) 00000080:00200000:3.0:1787826191.380231:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xc1:0x0](ffff9181c80a1988) 00000080:00200000:3.0:1787826191.380238:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xc2:0x0](ffff9181c8095b88) 00000080:00200000:3.0:1787826191.380245:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xc2:0x0](ffff9181c80900c8) 00000080:00200000:3.0:1787826191.380253:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xc3:0x0](ffff9181c809aa08) 00000080:00200000:3.0:1787826191.380260:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xc3:0x0](ffff9181c809cb08) 00000080:00200000:3.0:1787826191.380268:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xc4:0x0](ffff9181c80a3a88) 00000080:00200000:3.0:1787826191.380276:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xc4:0x0](ffff9181c80a3248) 00000080:00200000:3.0:1787826191.380283:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xc5:0x0](ffff9181c809c2c8) 00000080:00200000:3.0:1787826191.380290:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xc5:0x0](ffff9181c809d348) 00000080:00200000:3.0:1787826191.380297:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xc6:0x0](ffff9181c8097448) 00000080:00200000:3.0:1787826191.380306:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xc6:0x0](ffff9181c80adb88) 00000080:00200000:3.0:1787826191.380312:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xc7:0x0](ffff9181c80a4b08) 00000080:00200000:3.0:1787826191.380319:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xc7:0x0](ffff9181c80a2a08) 00000080:00200000:3.0:1787826191.380325:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xc8:0x0](ffff9181c807d348) 00000080:00200000:3.0:1787826191.380332:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xc8:0x0](ffff9181c807c2c8) 00000080:00200000:3.0:1787826191.380340:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xc9:0x0](ffff9181c80a5b88) 00000080:00200000:3.0:1787826191.380347:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xc9:0x0](ffff9181c80a7448) 00000080:00200000:3.0:1787826191.380354:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xca:0x0](ffff9181c80b6c08) 00000080:00200000:3.0:1787826191.380361:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xca:0x0](ffff9181c80b00c8) 00000080:00200000:3.0:1787826191.380368:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xcb:0x0](ffff9181c809ec08) 00000080:00200000:3.0:1787826191.380374:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xcb:0x0](ffff9181c8098908) 00000080:00200000:3.0:1787826191.380380:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xcc:0x0](ffff9181c80b1148) 00000080:00200000:3.0:1787826191.380387:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xcc:0x0](ffff9181c80b63c8) 00000080:00200000:3.0:1787826191.380394:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xcd:0x0](ffff9181c80b7448) 00000080:00200000:3.0:1787826191.380400:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xcd:0x0](ffff9181c80b21c8) 00000080:00200000:3.0:1787826191.380406:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xce:0x0](ffff9181c80b1988) 00000080:00200000:3.0:1787826191.380412:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xce:0x0](ffff9181c80b5b88) 00000080:00200000:3.0:1787826191.380419:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xcf:0x0](ffff9181c80b3a88) 00000080:00200000:3.0:1787826191.380425:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xcf:0x0](ffff9181c80b3248) 00000080:00200000:3.0:1787826191.380431:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xd0:0x0](ffff9181c80aec08) 00000080:00200000:3.0:1787826191.380438:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xd0:0x0](ffff9181c80a80c8) 00000080:00200000:3.0:1787826191.380444:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xd1:0x0](ffff9181c80ae3c8) 00000080:00200000:3.0:1787826191.380451:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xd1:0x0](ffff9181c80af448) 00000080:00200000:3.0:1787826191.380457:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xd2:0x0](ffff9181c80b2a08) 00000080:00200000:3.0:1787826191.380464:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xd2:0x0](ffff9181c80b4b08) 00000080:00200000:3.0:1787826191.380471:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xd3:0x0](ffff9181c80b0908) 00000080:00200000:3.0:1787826191.380478:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xd3:0x0](ffff9181c80b42c8) 00000080:00200000:3.0:1787826191.380484:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xd4:0x0](ffff9181c80a9988) 00000080:00200000:3.0:1787826191.380492:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xd4:0x0](ffff9181c80aa1c8) 00000080:00200000:3.0:1787826191.380498:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xd5:0x0](ffff9181c80a9148) 00000080:00200000:3.0:1787826191.380505:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xd5:0x0](ffff9181c80ad348) 00000080:00200000:3.0:1787826191.380511:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xd6:0x0](ffff9181c80b5348) 00000080:00200000:3.0:1787826191.380518:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xd6:0x0](ffff9181c80bdb88) 00000080:00200000:3.0:1787826191.380525:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xd7:0x0](ffff9181c80a8908) 00000080:00200000:3.0:1787826191.380532:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xd7:0x0](ffff9181c80acb08) 00000080:00200000:3.0:1787826191.380539:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xd8:0x0](ffff9181c80aaa08) 00000080:00200000:3.0:1787826191.380546:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xd8:0x0](ffff9181c80ab248) 00000080:00200000:3.0:1787826191.380554:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xd9:0x0](ffff9181c80aba88) 00000080:00200000:3.0:1787826191.380561:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xd9:0x0](ffff9181c80ac2c8) 00000080:00200000:3.0:1787826191.380568:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xda:0x0](ffff9181c80c00c8) 00000080:00200000:3.0:1787826191.380575:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xda:0x0](ffff9181c80c1148) 00000080:00200000:3.0:1787826191.380582:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xdb:0x0](ffff9181c809e3c8) 00000080:00200000:3.0:1787826191.380589:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xdb:0x0](ffff9181c80caa08) 00000080:00200000:3.0:1787826191.380596:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xdc:0x0](ffff9181c80c8908) 00000080:00200000:3.0:1787826191.380603:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xdc:0x0](ffff9181c80cc2c8) 00000080:00200000:3.0:1787826191.380610:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xdd:0x0](ffff9181c8079988) 00000080:00200000:3.0:1787826191.380618:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xdd:0x0](ffff9181c807f448) 00000080:00200000:3.0:1787826191.380625:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xde:0x0](ffff9181c80c7448) 00000080:00200000:3.0:1787826191.380634:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xde:0x0](ffff9181c80c21c8) 00000080:00200000:3.0:1787826191.380642:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xdf:0x0](ffff9181c80c1988) 00000080:00200000:3.0:1787826191.380655:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xdf:0x0](ffff9181c80c5b88) 00000080:00200000:3.0:1787826191.380663:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xe0:0x0](ffff9181c80bcb08) 00000080:00200000:3.0:1787826191.380672:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xe0:0x0](ffff9181c80b8908) 00000080:00200000:3.0:1787826191.380679:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xe1:0x0](ffff9181c80bc2c8) 00000080:00200000:3.0:1787826191.380688:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xe1:0x0](ffff9181c80bd348) 00000080:00200000:3.0:1787826191.380695:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xe2:0x0](ffff9181c80b80c8) 00000080:00200000:3.0:1787826191.380703:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xe2:0x0](ffff9181c80b9148) 00000080:00200000:3.0:1787826191.380712:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xe3:0x0](ffff9181c80bec08) 00000080:00200000:3.0:1787826191.380720:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xe3:0x0](ffff9181c80be3c8) 00000080:00200000:3.0:1787826191.380728:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xe4:0x0](ffff9181c80cd348) 00000080:00200000:3.0:1787826191.380736:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xe4:0x0](ffff9181c80cec08) 00000080:00200000:3.0:1787826191.380769:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xe5:0x0](ffff9181c80c80c8) 00000080:00200000:3.0:1787826191.380779:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xe5:0x0](ffff9181c80c9148) 00000080:00200000:3.0:1787826191.380787:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xe6:0x0](ffff9181c80ba1c8) 00000080:00200000:3.0:1787826191.380793:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xe6:0x0](ffff9181c80b9988) 00000080:00200000:3.0:1787826191.380800:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xe7:0x0](ffff9181c80c4b08) 00000080:00200000:3.0:1787826191.380809:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xe7:0x0](ffff9181c80c0908) 00000080:00200000:3.0:1787826191.380816:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xe8:0x0](ffff9181c80bf448) 00000080:00200000:3.0:1787826191.380824:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xe8:0x0](ffff9181c80baa08) 00000080:00200000:3.0:1787826191.380832:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xe9:0x0](ffff9181c80bb248) 00000080:00200000:3.0:1787826191.380855:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xe9:0x0](ffff9181c80bba88) 00000080:00200000:3.0:1787826191.380864:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xea:0x0](ffff9181c807cb08) 00000080:00200000:3.0:1787826191.380872:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xea:0x0](ffff9181c8078908) 00000080:00200000:3.0:1787826191.380880:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xeb:0x0](ffff9181c807ec08) 00000080:00200000:3.0:1787826191.380890:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xeb:0x0](ffff9181c807aa08) 00000080:00200000:3.0:1787826191.380900:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xec:0x0](ffff9181c807db88) 00000080:00200000:3.0:1787826191.380908:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xec:0x0](ffff9181c80d2a08) 00000080:00200000:3.0:1787826191.380916:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xed:0x0](ffff9181c80df448) 00000080:00200000:3.0:1787826191.380924:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xed:0x0](ffff9181c80da1c8) 00000080:00200000:3.0:1787826191.380933:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xee:0x0](ffff9181c80c5348) 00000080:00200000:3.0:1787826191.380943:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xee:0x0](ffff9181c80c42c8) 00000080:00200000:3.0:1787826191.380950:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xef:0x0](ffff9181c80ddb88) 00000080:00200000:3.0:1787826191.380958:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xef:0x0](ffff9181c80dba88) 00000080:00200000:3.0:1787826191.380966:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xf0:0x0](ffff9181c80db248) 00000080:00200000:3.0:1787826191.380974:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xf0:0x0](ffff9181c80daa08) 00000080:00200000:3.0:1787826191.380981:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xf1:0x0](ffff9181c80ce3c8) 00000080:00200000:3.0:1787826191.380989:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xf1:0x0](ffff9181c80cf448) 00000080:00200000:3.0:1787826191.380996:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xf2:0x0](ffff9181c80dcb08) 00000080:00200000:3.0:1787826191.381004:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xf2:0x0](ffff9181c80d8908) 00000080:00200000:3.0:1787826191.381012:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xf3:0x0](ffff9181c80dc2c8) 00000080:00200000:3.0:1787826191.381019:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xf3:0x0](ffff9181c80dd348) 00000080:00200000:3.0:1787826191.381026:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xf4:0x0](ffff9181c80c63c8) 00000080:00200000:3.0:1787826191.381034:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xf4:0x0](ffff9181c80c3a88) 00000080:00200000:3.0:1787826191.381041:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xf5:0x0](ffff9181c80c3248) 00000080:00200000:3.0:1787826191.381048:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xf5:0x0](ffff9181c80c2a08) 00000080:00200000:3.0:1787826191.381055:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xf6:0x0](ffff9181c80ca1c8) 00000080:00200000:3.0:1787826191.381063:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xf6:0x0](ffff9181c80c9988) 00000080:00200000:3.0:1787826191.381071:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xf7:0x0](ffff9181c80c6c08) 00000080:00200000:3.0:1787826191.381079:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xf7:0x0](ffff9181c80e63c8) 00000080:00200000:3.0:1787826191.381086:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xf8:0x0](ffff9181c80d9148) 00000080:00200000:3.0:1787826191.381095:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xf8:0x0](ffff9181c80de3c8) 00000080:00200000:3.0:1787826191.381103:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xf9:0x0](ffff9181c80d42c8) 00000080:00200000:3.0:1787826191.381110:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xf9:0x0](ffff9181c80d5348) 00000080:00200000:3.0:1787826191.381116:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xfa:0x0](ffff9181c80d80c8) 00000080:00200000:3.0:1787826191.381123:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xfa:0x0](ffff9181c80dec08) 00000080:00200000:3.0:1787826191.381130:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xfb:0x0](ffff9181c80d9988) 00000080:00200000:3.0:1787826191.381139:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xfb:0x0](ffff9181c80e9988) 00000080:00200000:3.0:1787826191.381146:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xfc:0x0](ffff9181c80d6c08) 00000080:00200000:3.0:1787826191.381154:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xfc:0x0](ffff9181c80d00c8) 00000080:00200000:3.0:1787826191.381161:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xfd:0x0](ffff9181c80d63c8) 00000080:00200000:3.0:1787826191.381169:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xfd:0x0](ffff9181c80d7448) 00000080:00200000:3.0:1787826191.381177:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xfe:0x0](ffff9181c80e3a88) 00000080:00200000:3.0:1787826191.381185:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xfe:0x0](ffff9181c80e3248) 00000080:00200000:3.0:1787826191.381193:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0xff:0x0](ffff9181c80e4b08) 00000080:00200000:3.0:1787826191.381201:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0xff:0x0](ffff9181c80e0908) 00000080:00200000:3.0:1787826191.381209:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x100:0x0](ffff9181c80e2a08) 00000080:00200000:3.0:1787826191.381217:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x100:0x0](ffff9181c80e42c8) 00000080:00200000:3.0:1787826191.381224:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x101:0x0](ffff9181c80e6c08) 00000080:00200000:3.0:1787826191.381231:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x101:0x0](ffff9181c80e00c8) 00000080:00200000:3.0:1787826191.381237:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x102:0x0](ffff9181c80cba88) 00000080:00200000:3.0:1787826191.381246:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x102:0x0](ffff9181c80cb248) 00000080:00200000:3.0:1787826191.381253:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x103:0x0](ffff9181c80e1148) 00000080:00200000:3.0:1787826191.381261:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x103:0x0](ffff9181c80e5b88) 00000080:00200000:3.0:1787826191.381267:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x104:0x0](ffff9181c80e21c8) 00000080:00200000:3.0:1787826191.381275:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x104:0x0](ffff9181c80e7448) 00000080:00200000:3.0:1787826191.381283:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x105:0x0](ffff9181c80cdb88) 00000080:00200000:3.0:1787826191.381290:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x105:0x0](ffff9181c80ccb08) 00000080:00200000:3.0:1787826191.381297:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x106:0x0](ffff9181c80eba88) 00000080:00200000:3.0:1787826191.381305:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x106:0x0](ffff9181c80eb248) 00000080:00200000:3.0:1787826191.381312:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x107:0x0](ffff9181c80d1148) 00000080:00200000:3.0:1787826191.381320:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x107:0x0](ffff9181c80d21c8) 00000080:00200000:3.0:1787826191.381326:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x108:0x0](ffff9181c80e1988) 00000080:00200000:3.0:1787826191.381334:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x108:0x0](ffff9181c80f42c8) 00000080:00200000:3.0:1787826191.381342:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x109:0x0](ffff9181c80f5348) 00000080:00200000:3.0:1787826191.381349:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x109:0x0](ffff9181c80f6c08) 00000080:00200000:3.0:1787826191.381357:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x10a:0x0](ffff9181c80e8908) 00000080:00200000:3.0:1787826191.381365:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x10a:0x0](ffff9181c80ec2c8) 00000080:00200000:3.0:1787826191.381372:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x10b:0x0](ffff9181c80d1988) 00000080:00200000:3.0:1787826191.381380:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x10b:0x0](ffff9181c80d5b88) 00000080:00200000:3.0:1787826191.381387:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x10c:0x0](ffff9181c80ecb08) 00000080:00200000:3.0:1787826191.381394:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x10c:0x0](ffff9181c80ed348) 00000080:00200000:3.0:1787826191.381401:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x10d:0x0](ffff9181c80fd348) 00000080:00200000:3.0:1787826191.381408:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x10d:0x0](ffff9181c80fec08) 00000080:00200000:3.0:1787826191.381415:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x10e:0x0](ffff9181c80f00c8) 00000080:00200000:3.0:1787826191.381422:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x10e:0x0](ffff9181c80f1148) 00000080:00200000:3.0:1787826191.381428:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x10f:0x0](ffff9181c80fe3c8) 00000080:00200000:3.0:1787826191.381436:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x10f:0x0](ffff9181c80ff448) 00000080:00200000:3.0:1787826191.381443:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x110:0x0](ffff9181c80f63c8) 00000080:00200000:3.0:1787826191.381451:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x110:0x0](ffff9181c80f7448) 00000080:00200000:3.0:1787826191.381458:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x111:0x0](ffff9181c80e80c8) 00000080:00200000:3.0:1787826191.381465:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x111:0x0](ffff9181c80e9148) 00000080:00200000:3.0:1787826191.381472:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x112:0x0](ffff9181c80f5b88) 00000080:00200000:3.0:1787826191.381479:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x112:0x0](ffff9181c80f3a88) 00000080:00200000:3.0:1787826191.381486:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x113:0x0](ffff9181c80f2a08) 00000080:00200000:3.0:1787826191.381494:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x113:0x0](ffff9181c80f4b08) 00000080:00200000:3.0:1787826191.381501:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x114:0x0](ffff9181c80f3248) 00000080:00200000:3.0:1787826191.381507:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x114:0x0](ffff9181c80f0908) 00000080:00200000:3.0:1787826191.381514:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x115:0x0](ffff9181c80d3248) 00000080:00200000:3.0:1787826191.381520:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x115:0x0](ffff9181c80d3a88) 00000080:00200000:3.0:1787826191.381527:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x116:0x0](ffff9181c80f9148) 00000080:00200000:3.0:1787826191.381533:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x116:0x0](ffff9181c80fa1c8) 00000080:00200000:3.0:1787826191.381540:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x117:0x0](ffff9181c80f21c8) 00000080:00200000:3.0:1787826191.381546:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x117:0x0](ffff9181c80f1988) 00000080:00200000:3.0:1787826191.381553:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x118:0x0](ffff9181c8101148) 00000080:00200000:3.0:1787826191.381561:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x118:0x0](ffff9181c81063c8) 00000080:00200000:3.0:1787826191.381569:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x119:0x0](ffff9181c80d0908) 00000080:00200000:3.0:1787826191.381577:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x119:0x0](ffff9181c80d4b08) 00000080:00200000:3.0:1787826191.381584:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x11a:0x0](ffff9181c8107448) 00000080:00200000:3.0:1787826191.381592:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x11a:0x0](ffff9181c81021c8) 00000080:00200000:3.0:1787826191.381600:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x11b:0x0](ffff9181c8101988) 00000080:00200000:3.0:1787826191.381608:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x11b:0x0](ffff9181c8105b88) 00000080:00200000:3.0:1787826191.381615:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x11c:0x0](ffff9181c80fb248) 00000080:00200000:3.0:1787826191.381623:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x11c:0x0](ffff9181c80faa08) 00000080:00200000:3.0:1787826191.381630:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x11d:0x0](ffff9181c80ef448) 00000080:00200000:3.0:1787826191.381636:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x11d:0x0](ffff9181c80ee3c8) 00000080:00200000:3.0:1787826191.381644:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x11e:0x0](ffff9181c80eaa08) 00000080:00200000:3.0:1787826191.381652:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x11e:0x0](ffff9181c80edb88) 00000080:00200000:3.0:1787826191.381660:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x11f:0x0](ffff9181c80ea1c8) 00000080:00200000:3.0:1787826191.381667:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x11f:0x0](ffff9181c810b248) 00000080:00200000:3.0:1787826191.381674:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x120:0x0](ffff9181c8103a88) 00000080:00200000:3.0:1787826191.381681:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x120:0x0](ffff9181c8103248) 00000080:00200000:3.0:1787826191.381687:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x121:0x0](ffff9181c8102a08) 00000080:00200000:3.0:1787826191.381695:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x121:0x0](ffff9181c8104b08) 00000080:00200000:3.0:1787826191.381702:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x122:0x0](ffff9181c8100908) 00000080:00200000:3.0:1787826191.381711:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x122:0x0](ffff9181c81042c8) 00000080:00200000:3.0:1787826191.381718:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x123:0x0](ffff9181c8108908) 00000080:00200000:3.0:1787826191.381725:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x123:0x0](ffff9181c810c2c8) 00000080:00200000:3.0:1787826191.381732:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x124:0x0](ffff9181c8105348) 00000080:00200000:3.0:1787826191.381739:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x124:0x0](ffff9181c8106c08) 00000080:00200000:3.0:1787826191.381774:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x125:0x0](ffff9181c8114b08) 00000080:00200000:3.0:1787826191.381783:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x125:0x0](ffff9181c8110908) 00000080:00200000:3.0:1787826191.381790:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x126:0x0](ffff9181c81080c8) 00000080:00200000:3.0:1787826191.381797:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x126:0x0](ffff9181c8109148) 00000080:00200000:3.0:1787826191.381805:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x127:0x0](ffff9181c80eec08) 00000080:00200000:3.0:1787826191.381813:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x127:0x0](ffff9181c80e5348) 00000080:00200000:3.0:1787826191.381821:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x128:0x0](ffff9181c80f8908) 00000080:00200000:3.0:1787826191.381829:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x128:0x0](ffff9181c80fc2c8) 00000080:00200000:3.0:1787826191.381836:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x129:0x0](ffff9181c81000c8) 00000080:00200000:3.0:1787826191.381854:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x129:0x0](ffff9181c811a1c8) 00000080:00200000:3.0:1787826191.381863:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x12a:0x0](ffff9181c81142c8) 00000080:00200000:3.0:1787826191.381871:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x12a:0x0](ffff9181c8115348) 00000080:00200000:3.0:1787826191.381878:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x12b:0x0](ffff9181c80fba88) 00000080:00200000:3.0:1787826191.381886:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x12b:0x0](ffff9181c80fdb88) 00000080:00200000:3.0:1787826191.381894:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x12c:0x0](ffff9181c811ba88) 00000080:00200000:3.0:1787826191.381901:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x12c:0x0](ffff9181c811b248) 00000080:00200000:3.0:1787826191.381908:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x12d:0x0](ffff9181c80f9988) 00000080:00200000:3.0:1787826191.381915:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x12d:0x0](ffff9181c80f80c8) 00000080:00200000:3.0:1787826191.381921:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x12e:0x0](ffff9181c8109988) 00000080:00200000:3.0:1787826191.381929:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x12e:0x0](ffff9181c810db88) 00000080:00200000:3.0:1787826191.381937:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x12f:0x0](ffff9181c8116c08) 00000080:00200000:3.0:1787826191.381945:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x12f:0x0](ffff9181c81100c8) 00000080:00200000:3.0:1787826191.381952:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x130:0x0](ffff9181c811aa08) 00000080:00200000:3.0:1787826191.381959:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x130:0x0](ffff9181c811cb08) 00000080:00200000:3.0:1787826191.381967:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x131:0x0](ffff9181c8118908) 00000080:00200000:3.0:1787826191.381974:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x131:0x0](ffff9181c811c2c8) 00000080:00200000:3.0:1787826191.381982:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x132:0x0](ffff9181c811d348) 00000080:00200000:3.0:1787826191.381990:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x132:0x0](ffff9181c811ec08) 00000080:00200000:3.0:1787826191.381998:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x133:0x0](ffff9181c810ba88) 00000080:00200000:3.0:1787826191.382005:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x133:0x0](ffff9181c810a1c8) 00000080:00200000:3.0:1787826191.382013:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x134:0x0](ffff9181c810e3c8) 00000080:00200000:3.0:1787826191.382021:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x134:0x0](ffff9181c810ec08) 00000080:00200000:3.0:1787826191.382028:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x135:0x0](ffff9181c8111148) 00000080:00200000:3.0:1787826191.382035:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x135:0x0](ffff9181c81163c8) 00000080:00200000:3.0:1787826191.382043:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x136:0x0](ffff9181c80fcb08) 00000080:00200000:3.0:1787826191.382051:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x136:0x0](ffff9181c8121148) 00000080:00200000:3.0:1787826191.382059:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x137:0x0](ffff9181c8119148) 00000080:00200000:3.0:1787826191.382067:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x137:0x0](ffff9181c811e3c8) 00000080:00200000:3.0:1787826191.382074:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x138:0x0](ffff9181c810cb08) 00000080:00200000:3.0:1787826191.382080:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x138:0x0](ffff9181c810aa08) 00000080:00200000:3.0:1787826191.382088:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x139:0x0](ffff9181c811f448) 00000080:00200000:3.0:1787826191.382096:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x139:0x0](ffff9181c81180c8) 00000080:00200000:3.0:1787826191.382103:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x13a:0x0](ffff9181c8119988) 00000080:00200000:3.0:1787826191.382110:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x13a:0x0](ffff9181c812aa08) 00000080:00200000:3.0:1787826191.382117:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x13b:0x0](ffff9181c8128908) 00000080:00200000:3.0:1787826191.382123:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x13b:0x0](ffff9181c812c2c8) 00000080:00200000:3.0:1787826191.382130:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x13c:0x0](ffff9181c812d348) 00000080:00200000:3.0:1787826191.382137:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x13c:0x0](ffff9181c812ec08) 00000080:00200000:3.0:1787826191.382143:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x13d:0x0](ffff9181c8129148) 00000080:00200000:3.0:1787826191.382150:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x13d:0x0](ffff9181c812e3c8) 00000080:00200000:3.0:1787826191.382158:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x13e:0x0](ffff9181c810d348) 00000080:00200000:3.0:1787826191.382165:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x13e:0x0](ffff9181c810f448) 00000080:00200000:3.0:1787826191.382173:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x13f:0x0](ffff9181c8121988) 00000080:00200000:3.0:1787826191.382181:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x13f:0x0](ffff9181c8125b88) 00000080:00200000:3.0:1787826191.382189:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x140:0x0](ffff9181c8123248) 00000080:00200000:3.0:1787826191.382197:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x140:0x0](ffff9181c8122a08) 00000080:00200000:3.0:1787826191.382204:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x141:0x0](ffff9181c812a1c8) 00000080:00200000:3.0:1787826191.382213:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x141:0x0](ffff9181c8129988) 00000080:00200000:3.0:1787826191.382220:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x142:0x0](ffff9181c81321c8) 00000080:00200000:3.0:1787826191.382228:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x142:0x0](ffff9181c8131988) 00000080:00200000:3.0:1787826191.382235:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x143:0x0](ffff9181c8135b88) 00000080:00200000:3.0:1787826191.382243:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x143:0x0](ffff9181c8133a88) 00000080:00200000:3.0:1787826191.382250:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x144:0x0](ffff9181c8111988) 00000080:00200000:3.0:1787826191.382258:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x144:0x0](ffff9181c8115b88) 00000080:00200000:3.0:1787826191.382265:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x145:0x0](ffff9181c812ba88) 00000080:00200000:3.0:1787826191.382273:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x145:0x0](ffff9181c812b248) 00000080:00200000:3.0:1787826191.382279:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x146:0x0](ffff9181c812f448) 00000080:00200000:3.0:1787826191.382287:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x146:0x0](ffff9181c81280c8) 00000080:00200000:3.0:1787826191.382294:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x147:0x0](ffff9181c8120908) 00000080:00200000:3.0:1787826191.382304:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x147:0x0](ffff9181c81242c8) 00000080:00200000:3.0:1787826191.382311:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x148:0x0](ffff9181c812db88) 00000080:00200000:3.0:1787826191.382318:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x148:0x0](ffff9181c811db88) 00000080:00200000:3.0:1787826191.382326:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x149:0x0](ffff9181c812cb08) 00000080:00200000:3.0:1787826191.382333:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x149:0x0](ffff9181c813c2c8) 00000080:00200000:3.0:1787826191.382341:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x14a:0x0](ffff9181c8134b08) 00000080:00200000:3.0:1787826191.382349:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x14a:0x0](ffff9181c8130908) 00000080:00200000:3.0:1787826191.382356:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x14b:0x0](ffff9181c813ec08) 00000080:00200000:3.0:1787826191.382365:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x14b:0x0](ffff9181c81380c8) 00000080:00200000:3.0:1787826191.382372:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x14c:0x0](ffff9181c813e3c8) 00000080:00200000:3.0:1787826191.382379:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x14c:0x0](ffff9181c813f448) 00000080:00200000:3.0:1787826191.382386:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x14d:0x0](ffff9181c81342c8) 00000080:00200000:3.0:1787826191.382393:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x14d:0x0](ffff9181c8135348) 00000080:00200000:3.0:1787826191.382399:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x14e:0x0](ffff9181c8136c08) 00000080:00200000:3.0:1787826191.382405:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x14e:0x0](ffff9181c81300c8) 00000080:00200000:3.0:1787826191.382411:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x14f:0x0](ffff9181c813db88) 00000080:00200000:3.0:1787826191.382418:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x14f:0x0](ffff9181c813ba88) 00000080:00200000:3.0:1787826191.382425:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x150:0x0](ffff9181c8126c08) 00000080:00200000:3.0:1787826191.382432:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x150:0x0](ffff9181c81200c8) 00000080:00200000:3.0:1787826191.382439:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x151:0x0](ffff9181c8113a88) 00000080:00200000:3.0:1787826191.382446:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x151:0x0](ffff9181c8113248) 00000080:00200000:3.0:1787826191.382453:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x152:0x0](ffff9181c81363c8) 00000080:00200000:3.0:1787826191.382460:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x152:0x0](ffff9181c8137448) 00000080:00200000:3.0:1787826191.382466:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x153:0x0](ffff9181c813cb08) 00000080:00200000:3.0:1787826191.382476:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x153:0x0](ffff9181c8138908) 00000080:00200000:3.0:1787826191.382482:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x154:0x0](ffff9181c81121c8) 00000080:00200000:3.0:1787826191.382489:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x154:0x0](ffff9181c8117448) 00000080:00200000:3.0:1787826191.382497:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x155:0x0](ffff9181c8131148) 00000080:00200000:3.0:1787826191.382504:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x155:0x0](ffff9181c8132a08) 00000080:00200000:3.0:1787826191.382511:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x156:0x0](ffff9181c813b248) 00000080:00200000:3.0:1787826191.382518:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x156:0x0](ffff9181c8139988) 00000080:00200000:3.0:1787826191.382525:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x157:0x0](ffff9181c8133248) 00000080:00200000:3.0:1787826191.382532:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x157:0x0](ffff9181c8141988) 00000080:00200000:3.0:1787826191.382539:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x158:0x0](ffff9181c813aa08) 00000080:00200000:3.0:1787826191.382546:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x158:0x0](ffff9181c813a1c8) 00000080:00200000:3.0:1787826191.382553:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x159:0x0](ffff9181c813d348) 00000080:00200000:3.0:1787826191.382562:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x159:0x0](ffff9181c814cb08) 00000080:00200000:3.0:1787826191.382569:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x15a:0x0](ffff9181c814c2c8) 00000080:00200000:3.0:1787826191.382577:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x15a:0x0](ffff9181c814d348) 00000080:00200000:3.0:1787826191.382585:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x15b:0x0](ffff9181c8123a88) 00000080:00200000:3.0:1787826191.382594:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x15b:0x0](ffff9181c81221c8) 00000080:00200000:3.0:1787826191.382602:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x15c:0x0](ffff9181c814ec08) 00000080:00200000:3.0:1787826191.382610:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x15c:0x0](ffff9181c81480c8) 00000080:00200000:3.0:1787826191.382619:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x15d:0x0](ffff9181c8149148) 00000080:00200000:3.0:1787826191.382627:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x15d:0x0](ffff9181c814e3c8) 00000080:00200000:3.0:1787826191.382636:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x15e:0x0](ffff9181c814f448) 00000080:00200000:3.0:1787826191.382643:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x15e:0x0](ffff9181c814a1c8) 00000080:00200000:3.0:1787826191.382653:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x15f:0x0](ffff9181c81563c8) 00000080:00200000:3.0:1787826191.382663:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x15f:0x0](ffff9181c8157448) 00000080:00200000:3.0:1787826191.382672:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x160:0x0](ffff9181c814db88) 00000080:00200000:3.0:1787826191.382680:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x160:0x0](ffff9181c814ba88) 00000080:00200000:3.0:1787826191.382688:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x161:0x0](ffff9181c8145b88) 00000080:00200000:3.0:1787826191.382696:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x161:0x0](ffff9181c8143a88) 00000080:00200000:3.0:1787826191.382704:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x162:0x0](ffff9181c814b248) 00000080:00200000:3.0:1787826191.382713:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x162:0x0](ffff9181c8149988) 00000080:00200000:3.0:1787826191.382720:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x163:0x0](ffff9181c8151988) 00000080:00200000:3.0:1787826191.382729:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x163:0x0](ffff9181c8155b88) 00000080:00200000:3.0:1787826191.382737:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x164:0x0](ffff9181c8124b08) 00000080:00200000:3.0:1787826191.382770:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x164:0x0](ffff9181c8125348) 00000080:00200000:3.0:1787826191.382780:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x165:0x0](ffff9181c81263c8) 00000080:00200000:3.0:1787826191.382788:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x165:0x0](ffff9181c815d348) 00000080:00200000:3.0:1787826191.382795:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x166:0x0](ffff9181c815ec08) 00000080:00200000:3.0:1787826191.382806:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x166:0x0](ffff9181c81580c8) 00000080:00200000:3.0:1787826191.382814:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x167:0x0](ffff9181c814aa08) 00000080:00200000:3.0:1787826191.382822:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x167:0x0](ffff9181c8112a08) 00000080:00200000:3.0:1787826191.382830:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x168:0x0](ffff9181c8139148) 00000080:00200000:3.0:1787826191.382838:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x168:0x0](ffff9181c8164b08) 00000080:00200000:3.0:1787826191.389940:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x169:0x0](ffff9181c8153a88) 00000080:00200000:3.0:1787826191.389986:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x169:0x0](ffff9181c8153248) 00000080:00200000:3.0:1787826191.389996:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x16a:0x0](ffff9181c8152a08) 00000080:00200000:3.0:1787826191.390009:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x16a:0x0](ffff9181c8154b08) 00000080:00200000:3.0:1787826191.390018:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x16b:0x0](ffff9181c8144b08) 00000080:00200000:3.0:1787826191.390029:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x16b:0x0](ffff9181c8140908) 00000080:00200000:3.0:1787826191.390038:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x16c:0x0](ffff9181c81442c8) 00000080:00200000:3.0:1787826191.390046:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x16c:0x0](ffff9181c8145348) 00000080:00200000:3.0:1787826191.390054:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x16d:0x0](ffff9181c8150908) 00000080:00200000:3.0:1787826191.390064:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x16d:0x0](ffff9181c81542c8) 00000080:00200000:3.0:1787826191.390077:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x16e:0x0](ffff9181c8166c08) 00000080:00200000:3.0:1787826191.390089:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x16e:0x0](ffff9181c81600c8) 00000080:00200000:3.0:1787826191.390097:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x16f:0x0](ffff9181c81663c8) 00000080:00200000:3.0:1787826191.390105:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x16f:0x0](ffff9181c8167448) 00000080:00200000:3.0:1787826191.390113:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x170:0x0](ffff9181c8161148) 00000080:00200000:3.0:1787826191.390122:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x170:0x0](ffff9181c81621c8) 00000080:00200000:3.0:1787826191.390130:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x171:0x0](ffff9181c8165b88) 00000080:00200000:3.0:1787826191.390144:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x171:0x0](ffff9181c8163a88) 00000080:00200000:3.0:1787826191.390152:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x172:0x0](ffff9181c8161988) 00000080:00200000:3.0:1787826191.390161:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x172:0x0](ffff9181c8163248) 00000080:00200000:3.0:1787826191.390169:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x173:0x0](ffff9181c8165348) 00000080:00200000:3.0:1787826191.390178:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x173:0x0](ffff9181c81642c8) 00000080:00200000:3.0:1787826191.390186:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x174:0x0](ffff9181c8160908) 00000080:00200000:3.0:1787826191.390195:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x174:0x0](ffff9181c8162a08) 00000080:00200000:3.0:1787826191.390203:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x175:0x0](ffff9181c816ec08) 00000080:00200000:3.0:1787826191.390212:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x175:0x0](ffff9181c81680c8) 00000080:00200000:3.0:1787826191.390219:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x176:0x0](ffff9181c8141148) 00000080:00200000:3.0:1787826191.390229:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x176:0x0](ffff9181c81463c8) 00000080:00200000:3.0:1787826191.390237:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x177:0x0](ffff9181c816f448) 00000080:00200000:3.0:1787826191.390246:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x177:0x0](ffff9181c816a1c8) 00000080:00200000:3.0:1787826191.390255:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x178:0x0](ffff9181c8151148) 00000080:00200000:3.0:1787826191.390264:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x178:0x0](ffff9181c8156c08) 00000080:00200000:3.0:1787826191.390273:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x179:0x0](ffff9181c8169988) 00000080:00200000:3.0:1787826191.390281:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x179:0x0](ffff9181c816db88) 00000080:00200000:3.0:1787826191.390289:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x17a:0x0](ffff9181c816e3c8) 00000080:00200000:3.0:1787826191.390297:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x17a:0x0](ffff9181c816ba88) 00000080:00200000:3.0:1787826191.390305:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x17b:0x0](ffff9181c81521c8) 00000080:00200000:3.0:1787826191.390315:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x17b:0x0](ffff9181c81500c8) 00000080:00200000:3.0:1787826191.390333:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x17c:0x0](ffff9181c816b248) 00000080:00200000:3.0:1787826191.390343:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x17c:0x0](ffff9181c816aa08) 00000080:00200000:3.0:1787826191.390351:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x17d:0x0](ffff9181c8168908) 00000080:00200000:3.0:1787826191.390363:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x17d:0x0](ffff9181c816c2c8) 00000080:00200000:3.0:1787826191.390371:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x17e:0x0](ffff9181c816cb08) 00000080:00200000:3.0:1787826191.390380:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x17e:0x0](ffff9181c8169148) 00000080:00200000:3.0:1787826191.390388:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x17f:0x0](ffff9181c8173248) 00000080:00200000:3.0:1787826191.390397:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x17f:0x0](ffff9181c8172a08) 00000080:00200000:3.0:1787826191.390404:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x180:0x0](ffff9181c815e3c8) 00000080:00200000:3.0:1787826191.390412:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x180:0x0](ffff9181c815f448) 00000080:00200000:3.0:1787826191.390418:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x181:0x0](ffff9181c815a1c8) 00000080:00200000:3.0:1787826191.390427:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x181:0x0](ffff9181c8159988) 00000080:00200000:3.0:1787826191.390442:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x182:0x0](ffff9181c81421c8) 00000080:00200000:3.0:1787826191.390451:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x182:0x0](ffff9181c8147448) 00000080:00200000:3.0:1787826191.390458:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x183:0x0](ffff9181c81400c8) 00000080:00200000:3.0:1787826191.390466:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x183:0x0](ffff9181c8146c08) 00000080:00200000:3.0:1787826191.390473:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x184:0x0](ffff9181c8176c08) 00000080:00200000:3.0:1787826191.390481:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x184:0x0](ffff9181c81700c8) 00000080:00200000:3.0:1787826191.390488:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x185:0x0](ffff9181c8142a08) 00000080:00200000:3.0:1787826191.390497:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x185:0x0](ffff9181c8143248) 00000080:00200000:3.0:1787826191.390504:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x186:0x0](ffff9181c81763c8) 00000080:00200000:3.0:1787826191.390513:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x186:0x0](ffff9181c8177448) 00000080:00200000:3.0:1787826191.390522:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x187:0x0](ffff9181c8155348) 00000080:00200000:3.0:1787826191.390530:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x187:0x0](ffff9181c817b248) 00000080:00200000:3.0:1787826191.390537:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x188:0x0](ffff9181c8171148) 00000080:00200000:3.0:1787826191.390544:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x188:0x0](ffff9181c81721c8) 00000080:00200000:3.0:1787826191.390552:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x189:0x0](ffff9181c8148908) 00000080:00200000:3.0:1787826191.390560:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x189:0x0](ffff9181c81863c8) 00000080:00200000:3.0:1787826191.390568:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x18a:0x0](ffff9181c8187448) 00000080:00200000:3.0:1787826191.390576:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x18a:0x0](ffff9181c81821c8) 00000080:00200000:3.0:1787826191.390584:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x18b:0x0](ffff9181c8185b88) 00000080:00200000:3.0:1787826191.390591:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x18b:0x0](ffff9181c8183a88) 00000080:00200000:3.0:1787826191.390599:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x18c:0x0](ffff9181c8183248) 00000080:00200000:3.0:1787826191.390607:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x18c:0x0](ffff9181c8182a08) 00000080:00200000:3.0:1787826191.390615:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x18d:0x0](ffff9181c8184b08) 00000080:00200000:3.0:1787826191.390625:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x18d:0x0](ffff9181c8180908) 00000080:00200000:3.0:1787826191.390633:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x18e:0x0](ffff9181c8171988) 00000080:00200000:3.0:1787826191.390640:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x18e:0x0](ffff9181c8175b88) 00000080:00200000:3.0:1787826191.390648:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x18f:0x0](ffff9181c8178908) 00000080:00200000:3.0:1787826191.390657:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x18f:0x0](ffff9181c817c2c8) 00000080:00200000:3.0:1787826191.390665:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x190:0x0](ffff9181c817d348) 00000080:00200000:3.0:1787826191.390673:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x190:0x0](ffff9181c817ec08) 00000080:00200000:3.0:1787826191.390680:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x191:0x0](ffff9181c8175348) 00000080:00200000:3.0:1787826191.390687:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x191:0x0](ffff9181c81742c8) 00000080:00200000:3.0:1787826191.390695:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x192:0x0](ffff9181c8173a88) 00000080:00200000:3.0:1787826191.390718:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x192:0x0](ffff9181c8170908) 00000080:00200000:3.0:1787826191.390727:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x193:0x0](ffff9181c8179148) 00000080:00200000:3.0:1787826191.390735:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x193:0x0](ffff9181c817e3c8) 00000080:00200000:3.0:1787826191.391085:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x194:0x0](ffff9181c8186c08) 00000080:00200000:3.0:1787826191.391100:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x194:0x0](ffff9181c81800c8) 00000080:00200000:3.0:1787826191.391108:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x195:0x0](ffff9181c816d348) 00000080:00200000:3.0:1787826191.391117:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x195:0x0](ffff9181c818aa08) 00000080:00200000:3.0:1787826191.391124:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x196:0x0](ffff9181c8181148) 00000080:00200000:3.0:1787826191.391133:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x196:0x0](ffff9181c8185348) 00000080:00200000:3.0:1787826191.391141:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x197:0x0](ffff9181c815db88) 00000080:00200000:3.0:1787826191.391151:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x197:0x0](ffff9181c815ba88) 00000080:00200000:3.0:1787826191.391159:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x198:0x0](ffff9181c818d348) 00000080:00200000:3.0:1787826191.391167:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x198:0x0](ffff9181c818ec08) 00000080:00200000:3.0:1787826191.391174:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x199:0x0](ffff9181c81880c8) 00000080:00200000:3.0:1787826191.391181:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x199:0x0](ffff9181c8189148) 00000080:00200000:3.0:1787826191.391188:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x19a:0x0](ffff9181c815b248) 00000080:00200000:3.0:1787826191.391200:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x19a:0x0](ffff9181c815aa08) 00000080:00200000:3.0:1787826191.391207:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x19b:0x0](ffff9181c8181988) 00000080:00200000:3.0:1787826191.391215:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x19b:0x0](ffff9181c8174b08) 00000080:00200000:3.0:1787826191.391221:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x19c:0x0](ffff9181c818a1c8) 00000080:00200000:3.0:1787826191.391229:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x19c:0x0](ffff9181c8189988) 00000080:00200000:3.0:1787826191.391237:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x19d:0x0](ffff9181c815cb08) 00000080:00200000:3.0:1787826191.391245:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x19d:0x0](ffff9181c8158908) 00000080:00200000:3.0:1787826191.391254:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x19e:0x0](ffff9181c817f448) 00000080:00200000:3.0:1787826191.391262:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x19e:0x0](ffff9181c817a1c8) 00000080:00200000:3.0:1787826191.391269:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x19f:0x0](ffff9181c81842c8) 00000080:00200000:3.0:1787826191.391276:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x19f:0x0](ffff9181c81921c8) 00000080:00200000:3.0:1787826191.391283:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1a0:0x0](ffff9181c8159148) 00000080:00200000:3.0:1787826191.391290:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1a0:0x0](ffff9181c8127448) 00000080:00200000:3.0:1787826191.391297:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1a1:0x0](ffff9181c818db88) 00000080:00200000:3.0:1787826191.391305:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1a1:0x0](ffff9181c818ba88) 00000080:00200000:3.0:1787826191.391312:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1a2:0x0](ffff9181c8195b88) 00000080:00200000:3.0:1787826191.391321:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1a2:0x0](ffff9181c8193a88) 00000080:00200000:3.0:1787826191.391330:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1a3:0x0](ffff9181c8193248) 00000080:00200000:3.0:1787826191.391339:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1a3:0x0](ffff9181c8192a08) 00000080:00200000:3.0:1787826191.391346:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1a4:0x0](ffff9181c817ba88) 00000080:00200000:3.0:1787826191.391353:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1a4:0x0](ffff9181c8179988) 00000080:00200000:3.0:1787826191.391361:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1a5:0x0](ffff9181c8194b08) 00000080:00200000:3.0:1787826191.391369:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1a5:0x0](ffff9181c8190908) 00000080:00200000:3.0:1787826191.391376:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1a6:0x0](ffff9181c818b248) 00000080:00200000:3.0:1787826191.391384:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1a6:0x0](ffff9181c818f448) 00000080:00200000:3.0:1787826191.391391:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1a7:0x0](ffff9181c818e3c8) 00000080:00200000:3.0:1787826191.391399:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1a7:0x0](ffff9181c818c2c8) 00000080:00200000:3.0:1787826191.391419:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1a8:0x0](ffff9181c817cb08) 00000080:00200000:3.0:1787826191.391427:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1a8:0x0](ffff9181c817aa08) 00000080:00200000:3.0:1787826191.391434:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1a9:0x0](ffff9181c815c2c8) 00000080:00200000:3.0:1787826191.391442:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1a9:0x0](ffff9181c8199988) 00000080:00200000:3.0:1787826191.391449:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1aa:0x0](ffff9181c8196c08) 00000080:00200000:3.0:1787826191.391457:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1aa:0x0](ffff9181c81900c8) 00000080:00200000:3.0:1787826191.391463:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1ab:0x0](ffff9181c818cb08) 00000080:00200000:3.0:1787826191.391471:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1ab:0x0](ffff9181c8188908) 00000080:00200000:3.0:1787826191.391478:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1ac:0x0](ffff9181c8191148) 00000080:00200000:3.0:1787826191.391486:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1ac:0x0](ffff9181c81963c8) 00000080:00200000:3.0:1787826191.391493:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1ad:0x0](ffff9181c81a2a08) 00000080:00200000:3.0:1787826191.391501:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1ad:0x0](ffff9181c81a4b08) 00000080:00200000:3.0:1787826191.391508:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1ae:0x0](ffff9181c81a3248) 00000080:00200000:3.0:1787826191.391517:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1ae:0x0](ffff9181c81a0908) 00000080:00200000:3.0:1787826191.391524:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1af:0x0](ffff9181c81a42c8) 00000080:00200000:3.0:1787826191.391532:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1af:0x0](ffff9181c81a5348) 00000080:00200000:3.0:1787826191.391541:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1b0:0x0](ffff9181c81780c8) 00000080:00200000:3.0:1787826191.391549:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1b0:0x0](ffff9181c817db88) 00000080:00200000:3.0:1787826191.391556:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1b1:0x0](ffff9181c8195348) 00000080:00200000:3.0:1787826191.391565:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1b1:0x0](ffff9181c81942c8) 00000080:00200000:3.0:1787826191.391573:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1b2:0x0](ffff9181c8197448) 00000080:00200000:3.0:1787826191.391581:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1b2:0x0](ffff9181c81a9988) 00000080:00200000:3.0:1787826191.391588:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1b3:0x0](ffff9181c81a6c08) 00000080:00200000:3.0:1787826191.391597:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1b3:0x0](ffff9181c81a00c8) 00000080:00200000:3.0:1787826191.391605:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1b4:0x0](ffff9181c81b4b08) 00000080:00200000:3.0:1787826191.391613:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1b4:0x0](ffff9181c81b0908) 00000080:00200000:3.0:1787826191.391621:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1b5:0x0](ffff9181c81b2a08) 00000080:00200000:3.0:1787826191.391630:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1b5:0x0](ffff9181c81b42c8) 00000080:00200000:3.0:1787826191.391636:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1b6:0x0](ffff9181c81ab248) 00000080:00200000:3.0:1787826191.391644:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1b6:0x0](ffff9181c81aaa08) 00000080:00200000:3.0:1787826191.391652:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1b7:0x0](ffff9181c81b5348) 00000080:00200000:3.0:1787826191.391659:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1b7:0x0](ffff9181c81b6c08) 00000080:00200000:3.0:1787826191.391667:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1b8:0x0](ffff9181c81a8908) 00000080:00200000:3.0:1787826191.391675:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1b8:0x0](ffff9181c81ac2c8) 00000080:00200000:3.0:1787826191.391684:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1b9:0x0](ffff9181c81a21c8) 00000080:00200000:3.0:1787826191.391692:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1b9:0x0](ffff9181c81a1988) 00000080:00200000:3.0:1787826191.394039:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1ba:0x0](ffff9181c81ad348) 00000080:00200000:2.0:1787826191.398037:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1ba:0x0](ffff9181c81aec08) 00000080:00200000:2.0:1787826191.398063:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1bb:0x0](ffff9181c81a5b88) 00000080:00200000:2.0:1787826191.398077:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1bb:0x0](ffff9181c81a7448) 00000080:00200000:2.0:1787826191.398085:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1bc:0x0](ffff9181c81b00c8) 00000080:00200000:2.0:1787826191.398094:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1bc:0x0](ffff9181c81b1148) 00000080:00200000:2.0:1787826191.398103:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1bd:0x0](ffff9181c81a1148) 00000080:00200000:2.0:1787826191.398111:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1bd:0x0](ffff9181c81a3a88) 00000080:00200000:2.0:1787826191.398120:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1be:0x0](ffff9181c81a80c8) 00000080:00200000:2.0:1787826191.398129:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1be:0x0](ffff9181c81a9148) 00000080:00200000:2.0:1787826191.398137:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1bf:0x0](ffff9181c81ae3c8) 00000080:00200000:2.0:1787826191.398145:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1bf:0x0](ffff9181c81af448) 00000080:00200000:2.0:1787826191.398153:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1c0:0x0](ffff9181c81b63c8) 00000080:00200000:2.0:1787826191.398170:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1c0:0x0](ffff9181c81b7448) 00000080:00200000:2.0:1787826191.398179:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1c1:0x0](ffff9181c81aa1c8) 00000080:00200000:2.0:1787826191.398187:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1c1:0x0](ffff9181c81acb08) 00000080:00200000:2.0:1787826191.398196:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1c2:0x0](ffff9181c81b1988) 00000080:00200000:2.0:1787826191.398204:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1c2:0x0](ffff9181c81b5b88) 00000080:00200000:2.0:1787826191.398212:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1c3:0x0](ffff9181c8191988) 00000080:00200000:2.0:1787826191.398223:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1c3:0x0](ffff9181c81ba1c8) 00000080:00200000:2.0:1787826191.398230:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1c4:0x0](ffff9181c81b9988) 00000080:00200000:2.0:1787826191.398244:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1c4:0x0](ffff9181c81bdb88) 00000080:00200000:2.0:1787826191.398253:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1c5:0x0](ffff9181c81bb248) 00000080:00200000:2.0:1787826191.398262:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1c5:0x0](ffff9181c81baa08) 00000080:00200000:2.0:1787826191.398270:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1c6:0x0](ffff9181c81bba88) 00000080:00200000:2.0:1787826191.398279:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1c6:0x0](ffff9181c81bcb08) 00000080:00200000:2.0:1787826191.398286:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1c7:0x0](ffff9181c81bc2c8) 00000080:00200000:2.0:1787826191.398295:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1c7:0x0](ffff9181c81bd348) 00000080:00200000:2.0:1787826191.398303:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1c8:0x0](ffff9181c81adb88) 00000080:00200000:2.0:1787826191.398313:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1c8:0x0](ffff9181c81aba88) 00000080:00200000:2.0:1787826191.398321:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1c9:0x0](ffff9181c81c3a88) 00000080:00200000:2.0:1787826191.398329:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1c9:0x0](ffff9181c81c3248) 00000080:00200000:2.0:1787826191.398337:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1ca:0x0](ffff9181c81b80c8) 00000080:00200000:2.0:1787826191.398345:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1ca:0x0](ffff9181c81b9148) 00000080:00200000:2.0:1787826191.398353:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1cb:0x0](ffff9181c81be3c8) 00000080:00200000:2.0:1787826191.398373:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1cb:0x0](ffff9181c81bf448) 00000080:00200000:2.0:1787826191.398383:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1cc:0x0](ffff9181c81b8908) 00000080:00200000:2.0:1787826191.398391:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1cc:0x0](ffff9181c81a63c8) 00000080:00200000:2.0:1787826191.398399:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1cd:0x0](ffff9181c81bec08) 00000080:00200000:2.0:1787826191.398408:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1cd:0x0](ffff9181c81cb248) 00000080:00200000:2.0:1787826191.398416:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1ce:0x0](ffff9181c81c4b08) 00000080:00200000:2.0:1787826191.398426:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1ce:0x0](ffff9181c81c0908) 00000080:00200000:2.0:1787826191.398434:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1cf:0x0](ffff9181c819cb08) 00000080:00200000:2.0:1787826191.398444:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1cf:0x0](ffff9181c8198908) 00000080:00200000:2.0:1787826191.398452:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1d0:0x0](ffff9181c81c8908) 00000080:00200000:2.0:1787826191.398461:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1d0:0x0](ffff9181c81cc2c8) 00000080:00200000:2.0:1787826191.398469:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1d1:0x0](ffff9181c81cec08) 00000080:00200000:2.0:1787826191.398477:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1d1:0x0](ffff9181c81c80c8) 00000080:00200000:2.0:1787826191.398486:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1d2:0x0](ffff9181c81cd348) 00000080:00200000:2.0:1787826191.398495:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1d2:0x0](ffff9181c81c9148) 00000080:00200000:2.0:1787826191.398504:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1d3:0x0](ffff9181c81c5348) 00000080:00200000:2.0:1787826191.398515:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1d3:0x0](ffff9181c81c6c08) 00000080:00200000:2.0:1787826191.398524:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1d4:0x0](ffff9181c81c00c8) 00000080:00200000:2.0:1787826191.398534:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1d4:0x0](ffff9181c81c1148) 00000080:00200000:2.0:1787826191.398544:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1d5:0x0](ffff9181c819c2c8) 00000080:00200000:2.0:1787826191.398553:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1d5:0x0](ffff9181c819d348) 00000080:00200000:2.0:1787826191.398561:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1d6:0x0](ffff9181c81b3248) 00000080:00200000:2.0:1787826191.398570:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1d6:0x0](ffff9181c81b3a88) 00000080:00200000:2.0:1787826191.398579:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1d7:0x0](ffff9181c81cdb88) 00000080:00200000:2.0:1787826191.398588:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1d7:0x0](ffff9181c81cba88) 00000080:00200000:2.0:1787826191.398595:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1d8:0x0](ffff9181c819ec08) 00000080:00200000:2.0:1787826191.398604:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1d8:0x0](ffff9181c81980c8) 00000080:00200000:2.0:1787826191.398635:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1d9:0x0](ffff9181c81ca1c8) 00000080:00200000:2.0:1787826191.398644:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1d9:0x0](ffff9181c81cf448) 00000080:00200000:2.0:1787826191.398652:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1da:0x0](ffff9181c81c7448) 00000080:00200000:2.0:1787826191.398661:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1da:0x0](ffff9181c81c21c8) 00000080:00200000:2.0:1787826191.398669:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1db:0x0](ffff9181c81ce3c8) 00000080:00200000:2.0:1787826191.398676:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1db:0x0](ffff9181c81ccb08) 00000080:00200000:2.0:1787826191.398685:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1dc:0x0](ffff9181c81c9988) 00000080:00200000:2.0:1787826191.398694:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1dc:0x0](ffff9181c81caa08) 00000080:00200000:2.0:1787826191.398716:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1dd:0x0](ffff9181c81d7448) 00000080:00200000:2.0:1787826191.398724:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1dd:0x0](ffff9181c81d21c8) 00000080:00200000:2.0:1787826191.398733:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1de:0x0](ffff9181c81d1988) 00000080:00200000:2.0:1787826191.398741:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1de:0x0](ffff9181c81d5b88) 00000080:00200000:2.0:1787826191.398778:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1df:0x0](ffff9181c81c5b88) 00000080:00200000:2.0:1787826191.398786:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1df:0x0](ffff9181c81c1988) 00000080:00200000:2.0:1787826191.398793:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1e0:0x0](ffff9181c81c63c8) 00000080:00200000:2.0:1787826191.398801:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1e0:0x0](ffff9181c81c42c8) 00000080:00200000:2.0:1787826191.398809:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1e1:0x0](ffff9181c81b21c8) 00000080:00200000:2.0:1787826191.398817:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1e1:0x0](ffff9181c81d80c8) 00000080:00200000:2.0:1787826191.398825:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1e2:0x0](ffff9181c81c2a08) 00000080:00200000:2.0:1787826191.398833:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1e2:0x0](ffff9181c81e5b88) 00000080:00200000:2.0:1787826191.398841:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1e3:0x0](ffff9181c81de3c8) 00000080:00200000:2.0:1787826191.398859:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1e3:0x0](ffff9181c81df448) 00000080:00200000:2.0:1787826191.398867:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1e4:0x0](ffff9181c81d3248) 00000080:00200000:2.0:1787826191.398876:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1e4:0x0](ffff9181c81d2a08) 00000080:00200000:2.0:1787826191.398887:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1e5:0x0](ffff9181c819f448) 00000080:00200000:2.0:1787826191.399246:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1e5:0x0](ffff9181c819e3c8) 00000080:00200000:2.0:1787826191.399264:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1e6:0x0](ffff9181c81d0908) 00000080:00200000:2.0:1787826191.399274:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1e6:0x0](ffff9181c81d42c8) 00000080:00200000:2.0:1787826191.399282:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1e7:0x0](ffff9181c81d4b08) 00000080:00200000:2.0:1787826191.399291:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1e7:0x0](ffff9181c81d5348) 00000080:00200000:2.0:1787826191.399299:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1e8:0x0](ffff9181c81e2a08) 00000080:00200000:2.0:1787826191.399309:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1e8:0x0](ffff9181c81e4b08) 00000080:00200000:2.0:1787826191.399316:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1e9:0x0](ffff9181c81d00c8) 00000080:00200000:2.0:1787826191.399324:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1e9:0x0](ffff9181c81d1148) 00000080:00200000:2.0:1787826191.399332:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1ea:0x0](ffff9181c819aa08) 00000080:00200000:2.0:1787826191.399340:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1ea:0x0](ffff9181c819b248) 00000080:00200000:2.0:1787826191.399347:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1eb:0x0](ffff9181c819ba88) 00000080:00200000:2.0:1787826191.399355:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1eb:0x0](ffff9181c819db88) 00000080:00200000:2.0:1787826191.399363:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1ec:0x0](ffff9181c81d63c8) 00000080:00200000:2.0:1787826191.399372:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1ec:0x0](ffff9181c81d3a88) 00000080:00200000:2.0:1787826191.399379:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1ed:0x0](ffff9181c81e42c8) 00000080:00200000:2.0:1787826191.399387:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1ed:0x0](ffff9181c81e5348) 00000080:00200000:2.0:1787826191.399395:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1ee:0x0](ffff9181c81e6c08) 00000080:00200000:2.0:1787826191.399402:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1ee:0x0](ffff9181c81e00c8) 00000080:00200000:2.0:1787826191.399410:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1ef:0x0](ffff9181c81d6c08) 00000080:00200000:2.0:1787826191.399419:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1ef:0x0](ffff9181c81ef448) 00000080:00200000:2.0:1787826191.399426:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1f0:0x0](ffff9181c81e63c8) 00000080:00200000:2.0:1787826191.399434:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1f0:0x0](ffff9181c81e7448) 00000080:00200000:2.0:1787826191.399442:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1f1:0x0](ffff9181c81edb88) 00000080:00200000:2.0:1787826191.399451:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1f1:0x0](ffff9181c81eba88) 00000080:00200000:2.0:1787826191.399460:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1f2:0x0](ffff9181c81e21c8) 00000080:00200000:2.0:1787826191.399468:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1f2:0x0](ffff9181c81e1988) 00000080:00200000:2.0:1787826191.399475:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1f3:0x0](ffff9181c81e1148) 00000080:00200000:2.0:1787826191.399483:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1f3:0x0](ffff9181c81e0908) 00000080:00200000:2.0:1787826191.399490:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1f4:0x0](ffff9181c81e3a88) 00000080:00200000:2.0:1787826191.399498:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1f4:0x0](ffff9181c81f1148) 00000080:00200000:2.0:1787826191.399506:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1f5:0x0](ffff9181c81f63c8) 00000080:00200000:2.0:1787826191.399513:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1f5:0x0](ffff9181c81f7448) 00000080:00200000:2.0:1787826191.399521:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1f6:0x0](ffff9181c81f21c8) 00000080:00200000:2.0:1787826191.399528:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1f6:0x0](ffff9181c81f1988) 00000080:00200000:2.0:1787826191.399536:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1f7:0x0](ffff9181c8199148) 00000080:00200000:2.0:1787826191.399544:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1f7:0x0](ffff9181c819a1c8) 00000080:00200000:2.0:1787826191.399552:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1f8:0x0](ffff9181c81e8908) 00000080:00200000:2.0:1787826191.399562:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1f8:0x0](ffff9181c81ec2c8) 00000080:00200000:2.0:1787826191.399571:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1f9:0x0](ffff9181c81f3248) 00000080:00200000:2.0:1787826191.399579:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1f9:0x0](ffff9181c81f2a08) 00000080:00200000:2.0:1787826191.399586:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1fa:0x0](ffff9181c81f80c8) 00000080:00200000:2.0:1787826191.399594:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1fa:0x0](ffff9181c81f9148) 00000080:00200000:2.0:1787826191.399602:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1fb:0x0](ffff9181c81f4b08) 00000080:00200000:2.0:1787826191.399611:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1fb:0x0](ffff9181c81f0908) 00000080:00200000:2.0:1787826191.399620:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1fc:0x0](ffff9181c81e80c8) 00000080:00200000:2.0:1787826191.399627:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1fc:0x0](ffff9181c81e9148) 00000080:00200000:2.0:1787826191.399636:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1fd:0x0](ffff9181c81f5348) 00000080:00200000:2.0:1787826191.399646:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1fd:0x0](ffff9181c81f6c08) 00000080:00200000:2.0:1787826191.399654:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1fe:0x0](ffff9181c81eec08) 00000080:00200000:2.0:1787826191.399663:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1fe:0x0](ffff9181c81ee3c8) 00000080:00200000:2.0:1787826191.399670:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x1ff:0x0](ffff9181c81ecb08) 00000080:00200000:2.0:1787826191.399678:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x1ff:0x0](ffff9181c81eaa08) 00000080:00200000:2.0:1787826191.399686:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x200:0x0](ffff9181c81e9988) 00000080:00200000:2.0:1787826191.399694:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x200:0x0](ffff9181c81ea1c8) 00000080:00200000:2.0:1787826191.400134:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x201:0x0](ffff9181c81ed348) 00000080:00200000:2.0:1787826191.400149:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x201:0x0](ffff9181c82000c8) 00000080:00200000:2.0:1787826191.400227:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x202:0x0](ffff9181c81f00c8) 00000080:00200000:2.0:1787826191.400255:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x202:0x0](ffff9181c81f42c8) 00000080:00200000:2.0:1787826191.400263:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x203:0x0](ffff9181c81f3a88) 00000080:00200000:2.0:1787826191.400270:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x203:0x0](ffff9181c81f5b88) 00000080:00200000:2.0:1787826191.400278:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x204:0x0](ffff9181c81eb248) 00000080:00200000:2.0:1787826191.400285:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x204:0x0](ffff9181c81e3248) 00000080:00200000:2.0:1787826191.400292:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x205:0x0](ffff9181c81fe3c8) 00000080:00200000:2.0:1787826191.400301:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x205:0x0](ffff9181c81ff448) 00000080:00200000:2.0:1787826191.400308:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x206:0x0](ffff9181c82063c8) 00000080:00200000:2.0:1787826191.400316:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x206:0x0](ffff9181c8207448) 00000080:00200000:2.0:1787826191.400324:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x207:0x0](ffff9181c820cb08) 00000080:00200000:2.0:1787826191.400332:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x207:0x0](ffff9181c8208908) 00000080:00200000:2.0:1787826191.400339:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x208:0x0](ffff9181c8201988) 00000080:00200000:2.0:1787826191.400348:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x208:0x0](ffff9181c8205b88) 00000080:00200000:2.0:1787826191.400355:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x209:0x0](ffff9181c8203a88) 00000080:00200000:2.0:1787826191.400364:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x209:0x0](ffff9181c8203248) 00000080:00200000:2.0:1787826191.400372:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x20a:0x0](ffff9181c81dba88) 00000080:00200000:2.0:1787826191.400381:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x20a:0x0](ffff9181c81db248) 00000080:00200000:2.0:1787826191.400389:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x20b:0x0](ffff9181c81daa08) 00000080:00200000:2.0:1787826191.400397:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x20b:0x0](ffff9181c81dcb08) 00000080:00200000:2.0:1787826191.400404:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x20c:0x0](ffff9181c82042c8) 00000080:00200000:2.0:1787826191.400411:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x20c:0x0](ffff9181c8205348) 00000080:00200000:2.0:1787826191.400419:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x20d:0x0](ffff9181c820d348) 00000080:00200000:2.0:1787826191.400427:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x20d:0x0](ffff9181c820ec08) 00000080:00200000:2.0:1787826191.400434:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x20e:0x0](ffff9181c81dd348) 00000080:00200000:2.0:1787826191.400443:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x20e:0x0](ffff9181c81dec08) 00000080:00200000:2.0:1787826191.400451:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x20f:0x0](ffff9181c82080c8) 00000080:00200000:2.0:1787826191.400459:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x20f:0x0](ffff9181c8209148) 00000080:00200000:2.0:1787826191.400477:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x210:0x0](ffff9181c8206c08) 00000080:00200000:2.0:1787826191.400486:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x210:0x0](ffff9181c8200908) 00000080:00200000:2.0:1787826191.400494:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x211:0x0](ffff9181c81f9988) 00000080:00200000:2.0:1787826191.400503:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x211:0x0](ffff9181c81fdb88) 00000080:00200000:2.0:1787826191.400510:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x212:0x0](ffff9181c820e3c8) 00000080:00200000:2.0:1787826191.400518:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x212:0x0](ffff9181c820f448) 00000080:00200000:2.0:1787826191.400526:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x213:0x0](ffff9181c82021c8) 00000080:00200000:2.0:1787826191.400535:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x213:0x0](ffff9181c8201148) 00000080:00200000:2.0:1787826191.400543:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x214:0x0](ffff9181c820db88) 00000080:00200000:2.0:1787826191.400552:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x214:0x0](ffff9181c820ba88) 00000080:00200000:2.0:1787826191.400561:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x215:0x0](ffff9181c8209988) 00000080:00200000:2.0:1787826191.400569:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x215:0x0](ffff9181c820b248) 00000080:00200000:2.0:1787826191.400579:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x216:0x0](ffff9181c8204b08) 00000080:00200000:2.0:1787826191.400588:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x216:0x0](ffff9181c8213248) 00000080:00200000:2.0:1787826191.400596:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x217:0x0](ffff9181c8212a08) 00000080:00200000:2.0:1787826191.400604:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x217:0x0](ffff9181c8214b08) 00000080:00200000:2.0:1787826191.400630:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x218:0x0](ffff9181c820a1c8) 00000080:00200000:2.0:1787826191.400641:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x218:0x0](ffff9181c820c2c8) 00000080:00200000:2.0:1787826191.400650:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x219:0x0](ffff9181c820aa08) 00000080:00200000:2.0:1787826191.400659:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x219:0x0](ffff9181c821a1c8) 00000080:00200000:2.0:1787826191.400667:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x21a:0x0](ffff9181c8219988) 00000080:00200000:2.0:1787826191.400676:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x21a:0x0](ffff9181c821db88) 00000080:00200000:2.0:1787826191.400684:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x21b:0x0](ffff9181c81d9988) 00000080:00200000:2.0:1787826191.400692:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x21b:0x0](ffff9181c81da1c8) 00000080:00200000:2.0:1787826191.402547:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x21c:0x0](ffff9181c821b248) 00000080:00200000:2.0:1787826191.402590:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x21c:0x0](ffff9181c821aa08) 00000080:00200000:2.0:1787826191.402601:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x21d:0x0](ffff9181c8215348) 00000080:00200000:2.0:1787826191.402669:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x21d:0x0](ffff9181c8216c08) 00000080:00200000:2.0:1787826191.402680:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x21e:0x0](ffff9181c821cb08) 00000080:00200000:2.0:1787826191.402690:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x21e:0x0](ffff9181c8218908) 00000080:00200000:2.0:1787826191.402698:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x21f:0x0](ffff9181c8211148) 00000080:00200000:2.0:1787826191.403121:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x21f:0x0](ffff9181c82163c8) 00000080:00200000:2.0:1787826191.403129:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x220:0x0](ffff9181c821d348) 00000080:00200000:2.0:1787826191.403139:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x220:0x0](ffff9181c821ec08) 00000080:00200000:2.0:1787826191.403147:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x221:0x0](ffff9181c8217448) 00000080:00200000:2.0:1787826191.403165:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x221:0x0](ffff9181c82121c8) 00000080:00200000:2.0:1787826191.403174:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x222:0x0](ffff9181c8215b88) 00000080:00200000:2.0:1787826191.403183:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x222:0x0](ffff9181c8213a88) 00000080:00200000:2.0:1787826191.403191:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x223:0x0](ffff9181c81ddb88) 00000080:00200000:2.0:1787826191.403201:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x223:0x0](ffff9181c81d9148) 00000080:00200000:2.0:1787826191.403208:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x224:0x0](ffff9181c81fba88) 00000080:00200000:2.0:1787826191.403219:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x224:0x0](ffff9181c81fb248) 00000080:00200000:2.0:1787826191.403226:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x225:0x0](ffff9181c8219148) 00000080:00200000:2.0:1787826191.403235:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x225:0x0](ffff9181c82142c8) 00000080:00200000:2.0:1787826191.403244:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x226:0x0](ffff9181c821f448) 00000080:00200000:2.0:1787826191.403253:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x226:0x0](ffff9181c82180c8) 00000080:00200000:2.0:1787826191.403260:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x227:0x0](ffff9181c8210908) 00000080:00200000:2.0:1787826191.403267:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x227:0x0](ffff9181c82100c8) 00000080:00200000:2.0:1787826191.403275:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x228:0x0](ffff9181c8211988) 00000080:00200000:2.0:1787826191.403283:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x228:0x0](ffff9181c8202a08) 00000080:00200000:2.0:1787826191.403291:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x229:0x0](ffff9181c8224b08) 00000080:00200000:2.0:1787826191.403309:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x229:0x0](ffff9181c8220908) 00000080:00200000:2.0:1787826191.403318:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x22a:0x0](ffff9181c81d8908) 00000080:00200000:2.0:1787826191.403328:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x22a:0x0](ffff9181c81dc2c8) 00000080:00200000:2.0:1787826191.403335:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x22b:0x0](ffff9181c821ba88) 00000080:00200000:2.0:1787826191.403343:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x22b:0x0](ffff9181c821c2c8) 00000080:00200000:2.0:1787826191.403351:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x22c:0x0](ffff9181c821e3c8) 00000080:00200000:2.0:1787826191.403358:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x22c:0x0](ffff9181c822b248) 00000080:00200000:2.0:1787826191.403367:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x22d:0x0](ffff9181c822aa08) 00000080:00200000:2.0:1787826191.403375:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x22d:0x0](ffff9181c822cb08) 00000080:00200000:2.0:1787826191.403383:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x22e:0x0](ffff9181c8228908) 00000080:00200000:2.0:1787826191.403392:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x22e:0x0](ffff9181c822c2c8) 00000080:00200000:2.0:1787826191.403399:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x22f:0x0](ffff9181c822d348) 00000080:00200000:2.0:1787826191.403408:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x22f:0x0](ffff9181c822ec08) 00000080:00200000:2.0:1787826191.403417:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x230:0x0](ffff9181c82200c8) 00000080:00200000:2.0:1787826191.403425:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x230:0x0](ffff9181c8221148) 00000080:00200000:2.0:1787826191.403433:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x231:0x0](ffff9181c8229148) 00000080:00200000:2.0:1787826191.403441:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x231:0x0](ffff9181c822e3c8) 00000080:00200000:2.0:1787826191.403450:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x232:0x0](ffff9181c82221c8) 00000080:00200000:2.0:1787826191.403461:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x232:0x0](ffff9181c8221988) 00000080:00200000:2.0:1787826191.403480:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x233:0x0](ffff9181c8225b88) 00000080:00200000:2.0:1787826191.403491:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x233:0x0](ffff9181c8223a88) 00000080:00200000:2.0:1787826191.403499:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x234:0x0](ffff9181c8230908) 00000080:00200000:2.0:1787826191.403507:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x234:0x0](ffff9181c82342c8) 00000080:00200000:2.0:1787826191.403515:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x235:0x0](ffff9181c822f448) 00000080:00200000:2.0:1787826191.403523:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x235:0x0](ffff9181c822a1c8) 00000080:00200000:2.0:1787826191.403531:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x236:0x0](ffff9181c81f8908) 00000080:00200000:2.0:1787826191.403541:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x236:0x0](ffff9181c81fc2c8) 00000080:00200000:2.0:1787826191.403549:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x237:0x0](ffff9181c82263c8) 00000080:00200000:2.0:1787826191.403559:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x237:0x0](ffff9181c8226c08) 00000080:00200000:2.0:1787826191.403568:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x238:0x0](ffff9181c8229988) 00000080:00200000:2.0:1787826191.403576:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x238:0x0](ffff9181c8235348) 00000080:00200000:2.0:1787826191.403584:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x239:0x0](ffff9181c822db88) 00000080:00200000:2.0:1787826191.403592:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x239:0x0](ffff9181c822ba88) 00000080:00200000:2.0:1787826191.403599:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x23a:0x0](ffff9181c8236c08) 00000080:00200000:2.0:1787826191.403617:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x23a:0x0](ffff9181c82300c8) 00000080:00200000:2.0:1787826191.403626:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x23b:0x0](ffff9181c8225348) 00000080:00200000:2.0:1787826191.403634:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x23b:0x0](ffff9181c82242c8) 00000080:00200000:2.0:1787826191.403641:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x23c:0x0](ffff9181c82280c8) 00000080:00200000:2.0:1787826191.403650:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x23c:0x0](ffff9181c8239148) 00000080:00200000:2.0:1787826191.403658:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x23d:0x0](ffff9181c8222a08) 00000080:00200000:2.0:1787826191.403666:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x23d:0x0](ffff9181c8227448) 00000080:00200000:2.0:1787826191.403674:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x23e:0x0](ffff9181c8241988) 00000080:00200000:2.0:1787826191.403682:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x23e:0x0](ffff9181c8245b88) 00000080:00200000:2.0:1787826191.403690:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x23f:0x0](ffff9181c8243a88) 00000080:00200000:2.0:1787826191.403698:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x23f:0x0](ffff9181c8243248) 00000080:00200000:2.0:1787826191.403719:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x240:0x0](ffff9181c8244b08) 00000080:00200000:2.0:1787826191.403727:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x240:0x0](ffff9181c8240908) 00000080:00200000:2.0:1787826191.403735:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x241:0x0](ffff9181c82363c8) 00000080:00200000:2.0:1787826191.404187:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x241:0x0](ffff9181c8237448) 00000080:00200000:2.0:1787826191.404203:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x242:0x0](ffff9181c823a1c8) 00000080:00200000:2.0:1787826191.404213:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x242:0x0](ffff9181c8239988) 00000080:00200000:2.0:1787826191.404222:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x243:0x0](ffff9181c8246c08) 00000080:00200000:2.0:1787826191.404231:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x243:0x0](ffff9181c82400c8) 00000080:00200000:2.0:1787826191.404238:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x244:0x0](ffff9181c823ba88) 00000080:00200000:2.0:1787826191.404246:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x244:0x0](ffff9181c823b248) 00000080:00200000:2.0:1787826191.404254:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x245:0x0](ffff9181c823aa08) 00000080:00200000:2.0:1787826191.404262:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x245:0x0](ffff9181c823cb08) 00000080:00200000:2.0:1787826191.404269:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x246:0x0](ffff9181c823c2c8) 00000080:00200000:2.0:1787826191.404277:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x246:0x0](ffff9181c823d348) 00000080:00200000:2.0:1787826191.404285:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x247:0x0](ffff9181c8231988) 00000080:00200000:2.0:1787826191.404294:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x247:0x0](ffff9181c8235b88) 00000080:00200000:2.0:1787826191.404302:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x248:0x0](ffff9181c82463c8) 00000080:00200000:2.0:1787826191.404310:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x248:0x0](ffff9181c8247448) 00000080:00200000:2.0:1787826191.404318:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x249:0x0](ffff9181c8233a88) 00000080:00200000:2.0:1787826191.404325:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x249:0x0](ffff9181c8233248) 00000080:00200000:2.0:1787826191.404332:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x24a:0x0](ffff9181c823ec08) 00000080:00200000:2.0:1787826191.404341:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x24a:0x0](ffff9181c8238908) 00000080:00200000:2.0:1787826191.404349:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x24b:0x0](ffff9181c823db88) 00000080:00200000:2.0:1787826191.404357:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x24b:0x0](ffff9181c823f448) 00000080:00200000:2.0:1787826191.404367:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x24c:0x0](ffff9181c8232a08) 00000080:00200000:2.0:1787826191.404376:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x24c:0x0](ffff9181c8234b08) 00000080:00200000:2.0:1787826191.404383:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x24d:0x0](ffff9181c82421c8) 00000080:00200000:2.0:1787826191.404391:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x24d:0x0](ffff9181c8241148) 00000080:00200000:2.0:1787826191.404398:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x24e:0x0](ffff9181c8245348) 00000080:00200000:2.0:1787826191.404406:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x24e:0x0](ffff9181c82442c8) 00000080:00200000:2.0:1787826191.404414:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x24f:0x0](ffff9181c81fec08) 00000080:00200000:2.0:1787826191.404423:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x24f:0x0](ffff9181c81fd348) 00000080:00200000:2.0:1787826191.404432:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x250:0x0](ffff9181c823e3c8) 00000080:00200000:2.0:1787826191.404440:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x250:0x0](ffff9181c82380c8) 00000080:00200000:2.0:1787826191.404447:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x251:0x0](ffff9181c81fcb08) 00000080:00200000:2.0:1787826191.404455:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x251:0x0](ffff9181c81faa08) 00000080:00200000:2.0:1787826191.408584:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x252:0x0](ffff9181c8242a08) 00000080:00200000:2.0:1787826191.408646:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x252:0x0](ffff9181c824d348) 00000080:00200000:2.0:1787826191.408657:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x253:0x0](ffff9181c82480c8) 00000080:00200000:2.0:1787826191.408666:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x253:0x0](ffff9181c8249148) 00000080:00200000:2.0:1787826191.408674:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x254:0x0](ffff9181c824e3c8) 00000080:00200000:2.0:1787826191.408683:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x254:0x0](ffff9181c824f448) 00000080:00200000:2.0:1787826191.408691:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x255:0x0](ffff9181c8223248) 00000080:00200000:2.0:1787826191.408715:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x255:0x0](ffff9181c82563c8) 00000080:00200000:2.0:1787826191.408725:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x256:0x0](ffff9181c8249988) 00000080:00200000:2.0:1787826191.408733:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x256:0x0](ffff9181c824db88) 00000080:00200000:2.0:1787826191.408741:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x257:0x0](ffff9181c824ba88) 00000080:00200000:2.0:1787826191.408913:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x257:0x0](ffff9181c824b248) 00000080:00200000:2.0:1787826191.408923:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x258:0x0](ffff9181c8231148) 00000080:00200000:2.0:1787826191.408932:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x258:0x0](ffff9181c82321c8) 00000080:00200000:2.0:1787826191.408940:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x259:0x0](ffff9181c8251988) 00000080:00200000:2.0:1787826191.408949:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x259:0x0](ffff9181c8255b88) 00000080:00200000:2.0:1787826191.408956:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x25a:0x0](ffff9181c824aa08) 00000080:00200000:2.0:1787826191.408965:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x25a:0x0](ffff9181c824cb08) 00000080:00200000:2.0:1787826191.408972:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x25b:0x0](ffff9181c8248908) 00000080:00200000:2.0:1787826191.408980:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x25b:0x0](ffff9181c824c2c8) 00000080:00200000:2.0:1787826191.408987:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x25c:0x0](ffff9181c824ec08) 00000080:00200000:2.0:1787826191.408995:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x25c:0x0](ffff9181c8263248) 00000080:00200000:2.0:1787826191.409003:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x25d:0x0](ffff9181c8264b08) 00000080:00200000:2.0:1787826191.409015:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x25d:0x0](ffff9181c8260908) 00000080:00200000:2.0:1787826191.409024:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x25e:0x0](ffff9181c8259988) 00000080:00200000:2.0:1787826191.409034:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x25e:0x0](ffff9181c825db88) 00000080:00200000:2.0:1787826191.409042:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x25f:0x0](ffff9181c825b248) 00000080:00200000:2.0:1787826191.409050:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x25f:0x0](ffff9181c825aa08) 00000080:00200000:2.0:1787826191.409059:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x260:0x0](ffff9181c8265348) 00000080:00200000:2.0:1787826191.409068:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x260:0x0](ffff9181c8266c08) 00000080:00200000:2.0:1787826191.409075:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x261:0x0](ffff9181c8261148) 00000080:00200000:2.0:1787826191.409084:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x261:0x0](ffff9181c82663c8) 00000080:00200000:2.0:1787826191.409092:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x262:0x0](ffff9181c8267448) 00000080:00200000:2.0:1787826191.409101:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x262:0x0](ffff9181c82621c8) 00000080:00200000:2.0:1787826191.409109:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x263:0x0](ffff9181c8258908) 00000080:00200000:2.0:1787826191.409118:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x263:0x0](ffff9181c825c2c8) 00000080:00200000:2.0:1787826191.409126:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x264:0x0](ffff9181c82600c8) 00000080:00200000:2.0:1787826191.409133:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x264:0x0](ffff9181c8261988) 00000080:00200000:2.0:1787826191.409140:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x265:0x0](ffff9181c8263a88) 00000080:00200000:2.0:1787826191.409148:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x265:0x0](ffff9181c82642c8) 00000080:00200000:2.0:1787826191.409157:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x266:0x0](ffff9181c81fa1c8) 00000080:00200000:2.0:1787826191.409170:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x266:0x0](ffff9181c826db88) 00000080:00200000:2.0:1787826191.409178:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x267:0x0](ffff9181c826ba88) 00000080:00200000:2.0:1787826191.409187:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x267:0x0](ffff9181c826b248) 00000080:00200000:2.0:1787826191.409195:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x268:0x0](ffff9181c8254b08) 00000080:00200000:2.0:1787826191.409204:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x268:0x0](ffff9181c8250908) 00000080:00200000:2.0:1787826191.409211:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x269:0x0](ffff9181c825cb08) 00000080:00200000:2.0:1787826191.409220:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x269:0x0](ffff9181c825d348) 00000080:00200000:2.0:1787826191.409227:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x26a:0x0](ffff9181c82542c8) 00000080:00200000:2.0:1787826191.409236:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x26a:0x0](ffff9181c8255348) 00000080:00200000:2.0:1787826191.409244:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x26b:0x0](ffff9181c82580c8) 00000080:00200000:2.0:1787826191.409252:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x26b:0x0](ffff9181c8259148) 00000080:00200000:2.0:1787826191.409259:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x26c:0x0](ffff9181c82500c8) 00000080:00200000:2.0:1787826191.409267:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x26c:0x0](ffff9181c8251148) 00000080:00200000:2.0:1787826191.409274:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x26d:0x0](ffff9181c8252a08) 00000080:00200000:2.0:1787826191.409282:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x26d:0x0](ffff9181c8253248) 00000080:00200000:2.0:1787826191.409290:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x26e:0x0](ffff9181c82521c8) 00000080:00200000:2.0:1787826191.409307:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x26e:0x0](ffff9181c8257448) 00000080:00200000:2.0:1787826191.409317:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x26f:0x0](ffff9181c8262a08) 00000080:00200000:2.0:1787826191.409325:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x26f:0x0](ffff9181c8265b88) 00000080:00200000:2.0:1787826191.409332:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x270:0x0](ffff9181c8274b08) 00000080:00200000:2.0:1787826191.409344:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x270:0x0](ffff9181c8270908) 00000080:00200000:2.0:1787826191.409351:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x271:0x0](ffff9181c8275348) 00000080:00200000:2.0:1787826191.409359:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x271:0x0](ffff9181c8276c08) 00000080:00200000:2.0:1787826191.409367:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x272:0x0](ffff9181c82700c8) 00000080:00200000:2.0:1787826191.409374:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x272:0x0](ffff9181c8271148) 00000080:00200000:2.0:1787826191.409380:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x273:0x0](ffff9181c8277448) 00000080:00200000:2.0:1787826191.409387:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x273:0x0](ffff9181c82721c8) 00000080:00200000:2.0:1787826191.409394:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x274:0x0](ffff9181c8271988) 00000080:00200000:2.0:1787826191.409402:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x274:0x0](ffff9181c8275b88) 00000080:00200000:2.0:1787826191.409410:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x275:0x0](ffff9181c8273a88) 00000080:00200000:2.0:1787826191.409418:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x275:0x0](ffff9181c8273248) 00000080:00200000:2.0:1787826191.409425:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x276:0x0](ffff9181c826c2c8) 00000080:00200000:2.0:1787826191.409433:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x276:0x0](ffff9181c826d348) 00000080:00200000:2.0:1787826191.409442:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x277:0x0](ffff9181c8272a08) 00000080:00200000:2.0:1787826191.409450:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x277:0x0](ffff9181c82763c8) 00000080:00200000:2.0:1787826191.409458:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x278:0x0](ffff9181c825ba88) 00000080:00200000:2.0:1787826191.409686:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x278:0x0](ffff9181c825a1c8) 00000080:00200000:2.0:1787826191.409858:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x279:0x0](ffff9181c827c2c8) 00000080:00200000:2.0:1787826191.409870:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x279:0x0](ffff9181c827d348) 00000080:00200000:2.0:1787826191.409879:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x27a:0x0](ffff9181c827ec08) 00000080:00200000:2.0:1787826191.409886:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x27a:0x0](ffff9181c82780c8) 00000080:00200000:2.0:1787826191.409949:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x27b:0x0](ffff9181c827e3c8) 00000080:00200000:2.0:1787826191.409962:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x27c:0x0](ffff9181c827f448) 00000080:00200000:2.0:1787826191.409971:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x27c:0x0](ffff9181c8279148) 00000080:00200000:2.0:1787826191.409979:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x27d:0x0](ffff9181c827a1c8) 00000080:00200000:2.0:1787826191.409986:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x27d:0x0](ffff9181c827db88) 00000080:00200000:2.0:1787826191.409995:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x27e:0x0](ffff9181c827ba88) 00000080:00200000:2.0:1787826191.410002:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x27e:0x0](ffff9181c825e3c8) 00000080:00200000:2.0:1787826191.410012:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x27f:0x0](ffff9181c825f448) 00000080:00200000:2.0:1787826191.410020:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x27f:0x0](ffff9181c827aa08) 00000080:00200000:2.0:1787826191.410028:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x280:0x0](ffff9181c827cb08) 00000080:00200000:2.0:1787826191.410037:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x280:0x0](ffff9181c827b248) 00000080:00200000:2.0:1787826191.410044:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x281:0x0](ffff9181c8278908) 00000080:00200000:2.0:1787826191.410052:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x281:0x0](ffff9181c82842c8) 00000080:00200000:2.0:1787826191.410060:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x282:0x0](ffff9181c8285348) 00000080:00200000:2.0:1787826191.410068:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x282:0x0](ffff9181c82742c8) 00000080:00200000:2.0:1787826191.410076:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x283:0x0](ffff9181c828ba88) 00000080:00200000:2.0:1787826191.410083:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x283:0x0](ffff9181c828b248) 00000080:00200000:2.0:1787826191.410092:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x284:0x0](ffff9181c828aa08) 00000080:00200000:2.0:1787826191.410099:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x284:0x0](ffff9181c8256c08) 00000080:00200000:2.0:1787826191.410107:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x285:0x0](ffff9181c8253a88) 00000080:00200000:2.0:1787826191.410115:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x285:0x0](ffff9181c8286c08) 00000080:00200000:2.0:1787826191.410123:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x286:0x0](ffff9181c82800c8) 00000080:00200000:2.0:1787826191.410131:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x286:0x0](ffff9181c8281148) 00000080:00200000:2.0:1787826191.410139:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x287:0x0](ffff9181c82863c8) 00000080:00200000:2.0:1787826191.410146:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x287:0x0](ffff9181c828c2c8) 00000080:00200000:2.0:1787826191.410155:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x288:0x0](ffff9181c828d348) 00000080:00200000:2.0:1787826191.410164:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x288:0x0](ffff9181c824a1c8) 00000080:00200000:2.0:1787826191.410171:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x289:0x0](ffff9181c8296c08) 00000080:00200000:2.0:1787826191.410179:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x289:0x0](ffff9181c826ec08) 00000080:00200000:2.0:1787826191.410188:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x28a:0x0](ffff9181c82680c8) 00000080:00200000:2.0:1787826191.410196:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x28a:0x0](ffff9181c82900c8) 00000080:00200000:2.0:1787826191.410203:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x28b:0x0](ffff9181c8291148) 00000080:00200000:2.0:1787826191.410211:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x28b:0x0](ffff9181c8287448) 00000080:00200000:2.0:1787826191.410220:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x28c:0x0](ffff9181c82821c8) 00000080:00200000:2.0:1787826191.410228:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x28c:0x0](ffff9181c8289148) 00000080:00200000:2.0:1787826191.410237:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x28d:0x0](ffff9181c828e3c8) 00000080:00200000:2.0:1787826191.410245:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x28d:0x0](ffff9181c8269148) 00000080:00200000:2.0:1787826191.410256:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x28e:0x0](ffff9181c826e3c8) 00000080:00200000:2.0:1787826191.410265:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x28e:0x0](ffff9181c82963c8) 00000080:00200000:2.0:1787826191.410272:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x28f:0x0](ffff9181c8297448) 00000080:00200000:2.0:1787826191.410280:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x28f:0x0](ffff9181c828a1c8) 00000080:00200000:2.0:1787826191.410287:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x290:0x0](ffff9181c8289988) 00000080:00200000:2.0:1787826191.410294:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x290:0x0](ffff9181c828db88) 00000080:00200000:2.0:1787826191.410397:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x291:0x0](ffff9181c828f448) 00000080:00200000:2.0:1787826191.410408:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x291:0x0](ffff9181c8295b88) 00000080:00200000:2.0:1787826191.410416:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x292:0x0](ffff9181c8293a88) 00000080:00200000:2.0:1787826191.410424:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x292:0x0](ffff9181c82880c8) 00000080:00200000:2.0:1787826191.410432:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x293:0x0](ffff9181c828ec08) 00000080:00200000:2.0:1787826191.410441:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x293:0x0](ffff9181c8269988) 00000080:00200000:2.0:1787826191.410450:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x294:0x0](ffff9181c826a1c8) 00000080:00200000:2.0:1787826191.410459:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x294:0x0](ffff9181c8291988) 00000080:00200000:2.0:1787826191.410532:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x295:0x0](ffff9181c8293248) 00000080:00200000:2.0:1787826191.410545:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x295:0x0](ffff9181c8292a08) 00000080:00200000:2.0:1787826191.410554:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x296:0x0](ffff9181c8294b08) 00000080:00200000:2.0:1787826191.410562:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x296:0x0](ffff9181c8288908) 00000080:00200000:2.0:1787826191.410570:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x297:0x0](ffff9181c8279988) 00000080:00200000:2.0:1787826191.410578:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x297:0x0](ffff9181c828cb08) 00000080:00200000:2.0:1787826191.410585:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x298:0x0](ffff9181c8299148) 00000080:00200000:2.0:1787826191.410594:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x298:0x0](ffff9181c8285b88) 00000080:00200000:2.0:1787826191.410603:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x299:0x0](ffff9181c8283a88) 00000080:00200000:2.0:1787826191.411127:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x299:0x0](ffff9181c8290908) 00000080:00200000:2.0:1787826191.411141:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x29a:0x0](ffff9181c82942c8) 00000080:00200000:2.0:1787826191.411149:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x29a:0x0](ffff9181c829f448) 00000080:00200000:2.0:1787826191.411159:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x29b:0x0](ffff9181c829a1c8) 00000080:00200000:2.0:1787826191.411167:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x29b:0x0](ffff9181c8295348) 00000080:00200000:2.0:1787826191.411176:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x29c:0x0](ffff9181c82921c8) 00000080:00200000:2.0:1787826191.411183:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x29c:0x0](ffff9181c82a3a88) 00000080:00200000:2.0:1787826191.411192:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x29d:0x0](ffff9181c82a3248) 00000080:00200000:2.0:1787826191.411200:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x29d:0x0](ffff9181c8283248) 00000080:00200000:2.0:1787826191.411208:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x29e:0x0](ffff9181c8282a08) 00000080:00200000:2.0:1787826191.411216:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x29e:0x0](ffff9181c8284b08) 00000080:00200000:2.0:1787826191.411225:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x29f:0x0](ffff9181c8280908) 00000080:00200000:2.0:1787826191.411233:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x29f:0x0](ffff9181c82a2a08) 00000080:00200000:2.0:1787826191.411243:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2a0:0x0](ffff9181c82a4b08) 00000080:00200000:2.0:1787826191.411252:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2a0:0x0](ffff9181c826cb08) 00000080:00200000:2.0:1787826191.411263:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2a1:0x0](ffff9181c8268908) 00000080:00200000:2.0:1787826191.411272:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2a1:0x0](ffff9181c829ba88) 00000080:00200000:2.0:1787826191.411281:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2a2:0x0](ffff9181c829b248) 00000080:00200000:2.0:1787826191.411291:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2a2:0x0](ffff9181c829aa08) 00000080:00200000:2.0:1787826191.413116:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2a3:0x0](ffff9181c829cb08) 00000080:00200000:2.0:1787826191.413143:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2a3:0x0](ffff9181c8298908) 00000080:00200000:2.0:1787826191.413152:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2a4:0x0](ffff9181c829c2c8) 00000080:00200000:2.0:1787826191.413161:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2a4:0x0](ffff9181c829ec08) 00000080:00200000:2.0:1787826191.413169:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2a5:0x0](ffff9181c82980c8) 00000080:00200000:2.0:1787826191.413176:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2a5:0x0](ffff9181c829d348) 00000080:00200000:2.0:1787826191.413184:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2a6:0x0](ffff9181c829db88) 00000080:00200000:2.0:1787826191.413191:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2a6:0x0](ffff9181c82a5348) 00000080:00200000:2.0:1787826191.413200:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2a7:0x0](ffff9181c82a6c08) 00000080:00200000:2.0:1787826191.413208:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2a7:0x0](ffff9181c826f448) 00000080:00200000:2.0:1787826191.413221:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2a8:0x0](ffff9181c826aa08) 00000080:00200000:2.0:1787826191.413229:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2a8:0x0](ffff9181c825ec08) 00000080:00200000:2.0:1787826191.413237:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2a9:0x0](ffff9181c82ab248) 00000080:00200000:2.0:1787826191.413245:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2a9:0x0](ffff9181c829e3c8) 00000080:00200000:2.0:1787826191.413252:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2aa:0x0](ffff9181c82b1988) 00000080:00200000:2.0:1787826191.413260:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2aa:0x0](ffff9181c82a63c8) 00000080:00200000:2.0:1787826191.413267:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2ab:0x0](ffff9181c82a7448) 00000080:00200000:2.0:1787826191.413274:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2ab:0x0](ffff9181c82a21c8) 00000080:00200000:2.0:1787826191.413282:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2ac:0x0](ffff9181c82a1988) 00000080:00200000:2.0:1787826191.413290:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2ac:0x0](ffff9181c8281988) 00000080:00200000:2.0:1787826191.413410:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2ad:0x0](ffff9181c82bdb88) 00000080:00200000:2.0:1787826191.413421:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2ad:0x0](ffff9181c82bba88) 00000080:00200000:2.0:1787826191.413429:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2ae:0x0](ffff9181c82bb248) 00000080:00200000:2.0:1787826191.413436:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2ae:0x0](ffff9181c82aaa08) 00000080:00200000:2.0:1787826191.413446:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2af:0x0](ffff9181c82acb08) 00000080:00200000:2.0:1787826191.413457:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2af:0x0](ffff9181c82a8908) 00000080:00200000:2.0:1787826191.413513:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2b0:0x0](ffff9181c82ac2c8) 00000080:00200000:2.0:1787826191.413525:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2b0:0x0](ffff9181c82b0908) 00000080:00200000:2.0:1787826191.413537:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2b1:0x0](ffff9181c82b42c8) 00000080:00200000:2.0:1787826191.413545:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2b1:0x0](ffff9181c82b6c08) 00000080:00200000:2.0:1787826191.413552:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2b2:0x0](ffff9181c82b00c8) 00000080:00200000:2.0:1787826191.413560:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2b2:0x0](ffff9181c82ad348) 00000080:00200000:2.0:1787826191.413571:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2b3:0x0](ffff9181c82aec08) 00000080:00200000:2.0:1787826191.413583:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2b3:0x0](ffff9181c82a9148) 00000080:00200000:2.0:1787826191.413595:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2b4:0x0](ffff9181c82ae3c8) 00000080:00200000:2.0:1787826191.413602:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2b4:0x0](ffff9181c82a1148) 00000080:00200000:2.0:1787826191.414426:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2b5:0x0](ffff9181c82a00c8) 00000080:00200000:2.0:1787826191.414444:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2b5:0x0](ffff9181c82a42c8) 00000080:00200000:2.0:1787826191.414454:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2b6:0x0](ffff9181c82a0908) 00000080:00200000:2.0:1787826191.414461:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2b6:0x0](ffff9181c82b63c8) 00000080:00200000:2.0:1787826191.414680:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2b7:0x0](ffff9181c82b7448) 00000080:00200000:2.0:1787826191.414693:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2b7:0x0](ffff9181c82b1148) 00000080:00200000:2.0:1787826191.414983:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2b8:0x0](ffff9181c82b5348) 00000080:00200000:2.0:1787826191.414997:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2b8:0x0](ffff9181c82a5b88) 00000080:00200000:2.0:1787826191.415007:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2b9:0x0](ffff9181c82c5348) 00000080:00200000:2.0:1787826191.415017:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2b9:0x0](ffff9181c82baa08) 00000080:00200000:2.0:1787826191.415031:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2ba:0x0](ffff9181c82bcb08) 00000080:00200000:2.0:1787826191.415040:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2bb:0x0](ffff9181c82b21c8) 00000080:00200000:2.0:1787826191.415049:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2bb:0x0](ffff9181c82b4b08) 00000080:00200000:2.0:1787826191.415059:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2bc:0x0](ffff9181c82b3248) 00000080:00200000:2.0:1787826191.415070:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2bc:0x0](ffff9181c82b3a88) 00000080:00200000:2.0:1787826191.415078:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2bd:0x0](ffff9181c82b2a08) 00000080:00200000:2.0:1787826191.415086:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2bd:0x0](ffff9181c82b5b88) 00000080:00200000:2.0:1787826191.415094:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2be:0x0](ffff9181c82cb248) 00000080:00200000:2.0:1787826191.415105:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2be:0x0](ffff9181c82caa08) 00000080:00200000:2.0:1787826191.415113:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2bf:0x0](ffff9181c82c1148) 00000080:00200000:2.0:1787826191.415123:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2bf:0x0](ffff9181c82c63c8) 00000080:00200000:2.0:1787826191.415132:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2c0:0x0](ffff9181c82cc2c8) 00000080:00200000:2.0:1787826191.415142:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2c0:0x0](ffff9181c82cd348) 00000080:00200000:2.0:1787826191.415150:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2c1:0x0](ffff9181c82c7448) 00000080:00200000:2.0:1787826191.415159:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2c1:0x0](ffff9181c82c21c8) 00000080:00200000:2.0:1787826191.415168:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2c2:0x0](ffff9181c82af448) 00000080:00200000:2.0:1787826191.415182:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2c2:0x0](ffff9181c82aa1c8) 00000080:00200000:2.0:1787826191.415190:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2c3:0x0](ffff9181c82a9988) 00000080:00200000:2.0:1787826191.415199:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2c3:0x0](ffff9181c82adb88) 00000080:00200000:2.0:1787826191.415209:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2c4:0x0](ffff9181c82c1988) 00000080:00200000:2.0:1787826191.415219:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2c4:0x0](ffff9181c82c5b88) 00000080:00200000:2.0:1787826191.415228:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2c5:0x0](ffff9181c82bc2c8) 00000080:00200000:2.0:1787826191.415237:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2c5:0x0](ffff9181c82bd348) 00000080:00200000:2.0:1787826191.415246:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2c6:0x0](ffff9181c82b80c8) 00000080:00200000:2.0:1787826191.415254:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2c6:0x0](ffff9181c82b9148) 00000080:00200000:2.0:1787826191.415263:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2c7:0x0](ffff9181c82c3248) 00000080:00200000:2.0:1787826191.415272:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2c7:0x0](ffff9181c82c2a08) 00000080:00200000:2.0:1787826191.415280:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2c8:0x0](ffff9181c82c0908) 00000080:00200000:2.0:1787826191.415290:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2c8:0x0](ffff9181c82c42c8) 00000080:00200000:2.0:1787826191.420563:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2c9:0x0](ffff9181c82bec08) 00000080:00200000:2.0:1787826191.421162:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2c9:0x0](ffff9181c82be3c8) 00000080:00200000:2.0:1787826191.421182:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2ca:0x0](ffff9181c82cec08) 00000080:00200000:2.0:1787826191.421203:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2ca:0x0](ffff9181c82c80c8) 00000080:00200000:2.0:1787826191.421213:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2cb:0x0](ffff9181c82c4b08) 00000080:00200000:2.0:1787826191.421229:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2cb:0x0](ffff9181c82c3a88) 00000080:00200000:2.0:1787826191.421239:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2cc:0x0](ffff9181c82c00c8) 00000080:00200000:2.0:1787826191.421248:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2cc:0x0](ffff9181c82c6c08) 00000080:00200000:2.0:1787826191.421258:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2cd:0x0](ffff9181c82b9988) 00000080:00200000:2.0:1787826191.421267:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2cd:0x0](ffff9181c82bf448) 00000080:00200000:2.0:1787826191.421275:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2ce:0x0](ffff9181c82ce3c8) 00000080:00200000:2.0:1787826191.421285:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2ce:0x0](ffff9181c82cf448) 00000080:00200000:2.0:1787826191.421296:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2cf:0x0](ffff9181c82ca1c8) 00000080:00200000:2.0:1787826191.421306:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2cf:0x0](ffff9181c82c9988) 00000080:00200000:2.0:1787826191.421317:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2d0:0x0](ffff9181c82a80c8) 00000080:00200000:2.0:1787826191.421332:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2d0:0x0](ffff9181c82aba88) 00000080:00200000:2.0:1787826191.421343:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2d1:0x0](ffff9181c82d6c08) 00000080:00200000:2.0:1787826191.421354:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2d1:0x0](ffff9181c82d00c8) 00000080:00200000:2.0:1787826191.421362:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2d2:0x0](ffff9181c82cdb88) 00000080:00200000:2.0:1787826191.421373:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2d2:0x0](ffff9181c82cba88) 00000080:00200000:2.0:1787826191.421381:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2d3:0x0](ffff9181c82d63c8) 00000080:00200000:2.0:1787826191.421390:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2d3:0x0](ffff9181c82d7448) 00000080:00200000:2.0:1787826191.421399:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2d4:0x0](ffff9181c82ccb08) 00000080:00200000:2.0:1787826191.421408:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2d4:0x0](ffff9181c82c9148) 00000080:00200000:2.0:1787826191.421416:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2d5:0x0](ffff9181c82ddb88) 00000080:00200000:2.0:1787826191.421424:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2d5:0x0](ffff9181c82dba88) 00000080:00200000:2.0:1787826191.421433:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2d6:0x0](ffff9181c82db248) 00000080:00200000:2.0:1787826191.421442:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2d6:0x0](ffff9181c82daa08) 00000080:00200000:2.0:1787826191.421451:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2d7:0x0](ffff9181c82dcb08) 00000080:00200000:2.0:1787826191.421461:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2d7:0x0](ffff9181c82d8908) 00000080:00200000:2.0:1787826191.424947:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2d8:0x0](ffff9181c82d1988) 00000080:00200000:2.0:1787826191.424986:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2d8:0x0](ffff9181c82d5b88) 00000080:00200000:2.0:1787826191.424996:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2d9:0x0](ffff9181c82dc2c8) 00000080:00200000:2.0:1787826191.425006:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2d9:0x0](ffff9181c82dd348) 00000080:00200000:2.0:1787826191.425018:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2da:0x0](ffff9181c82d80c8) 00000080:00200000:2.0:1787826191.425027:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2da:0x0](ffff9181c82d9148) 00000080:00200000:2.0:1787826191.425036:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2db:0x0](ffff9181c8299988) 00000080:00200000:2.0:1787826191.425054:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2db:0x0](ffff9181c82c8908) 00000080:00200000:2.0:1787826191.425064:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2dc:0x0](ffff9181c82de3c8) 00000080:00200000:2.0:1787826191.425073:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2dc:0x0](ffff9181c82df448) 00000080:00200000:2.0:1787826191.425083:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2dd:0x0](ffff9181c82d4b08) 00000080:00200000:2.0:1787826191.425092:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2dd:0x0](ffff9181c82d0908) 00000080:00200000:2.0:1787826191.425102:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2de:0x0](ffff9181c82e0908) 00000080:00200000:2.0:1787826191.425118:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2de:0x0](ffff9181c82e42c8) 00000080:00200000:2.0:1787826191.425126:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2df:0x0](ffff9181c82d42c8) 00000080:00200000:2.0:1787826191.425135:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2df:0x0](ffff9181c82d5348) 00000080:00200000:2.0:1787826191.425142:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2e0:0x0](ffff9181c82d2a08) 00000080:00200000:2.0:1787826191.425150:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2e0:0x0](ffff9181c82d3248) 00000080:00200000:2.0:1787826191.425159:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2e1:0x0](ffff9181c82d3a88) 00000080:00200000:2.0:1787826191.425167:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2e1:0x0](ffff9181c82d21c8) 00000080:00200000:2.0:1787826191.425175:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2e2:0x0](ffff9181c82d9988) 00000080:00200000:2.0:1787826191.425184:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2e2:0x0](ffff9181c82da1c8) 00000080:00200000:2.0:1787826191.425193:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2e3:0x0](ffff9181c82dec08) 00000080:00200000:2.0:1787826191.425203:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2e3:0x0](ffff9181c82e9988) 00000080:00200000:2.0:1787826191.425211:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2e4:0x0](ffff9181c82eba88) 00000080:00200000:2.0:1787826191.425219:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2e4:0x0](ffff9181c82eb248) 00000080:00200000:2.0:1787826191.425229:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2e5:0x0](ffff9181c82eaa08) 00000080:00200000:2.0:1787826191.425237:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2e5:0x0](ffff9181c82ecb08) 00000080:00200000:2.0:1787826191.425244:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2e6:0x0](ffff9181c82f5b88) 00000080:00200000:2.0:1787826191.425253:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2e6:0x0](ffff9181c82f3a88) 00000080:00200000:2.0:1787826191.425261:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2e7:0x0](ffff9181c82ba1c8) 00000080:00200000:2.0:1787826191.425270:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2e7:0x0](ffff9181c82b8908) 00000080:00200000:2.0:1787826191.425278:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2e8:0x0](ffff9181c82e8908) 00000080:00200000:2.0:1787826191.425286:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2e8:0x0](ffff9181c82ec2c8) 00000080:00200000:2.0:1787826191.425294:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2e9:0x0](ffff9181c82e6c08) 00000080:00200000:2.0:1787826191.425312:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2e9:0x0](ffff9181c82e00c8) 00000080:00200000:2.0:1787826191.425320:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2ea:0x0](ffff9181c82f2a08) 00000080:00200000:2.0:1787826191.425329:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2ea:0x0](ffff9181c82f4b08) 00000080:00200000:2.0:1787826191.425337:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2eb:0x0](ffff9181c82eec08) 00000080:00200000:2.0:1787826191.425345:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2eb:0x0](ffff9181c82e80c8) 00000080:00200000:2.0:1787826191.425352:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2ec:0x0](ffff9181c82e9148) 00000080:00200000:2.0:1787826191.425362:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2ec:0x0](ffff9181c82ee3c8) 00000080:00200000:2.0:1787826191.425370:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2ed:0x0](ffff9181c82f5348) 00000080:00200000:2.0:1787826191.425380:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2ed:0x0](ffff9181c82f6c08) 00000080:00200000:2.0:1787826191.425388:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2ee:0x0](ffff9181c82f1148) 00000080:00200000:2.0:1787826191.425396:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2ee:0x0](ffff9181c82f63c8) 00000080:00200000:2.0:1787826191.425404:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2ef:0x0](ffff9181c82f7448) 00000080:00200000:2.0:1787826191.425414:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2ef:0x0](ffff9181c82fa1c8) 00000080:00200000:2.0:1787826191.425423:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2f0:0x0](ffff9181c82ea1c8) 00000080:00200000:2.0:1787826191.425431:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2f0:0x0](ffff9181c82ef448) 00000080:00200000:2.0:1787826191.425438:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2f1:0x0](ffff9181c82f9988) 00000080:00200000:2.0:1787826191.425447:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2f1:0x0](ffff9181c82fdb88) 00000080:00200000:2.0:1787826191.425455:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2f2:0x0](ffff9181c82ed348) 00000080:00200000:2.0:1787826191.425464:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2f2:0x0](ffff9181c82d1148) 00000080:00200000:2.0:1787826191.425482:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2f3:0x0](ffff9181c82fba88) 00000080:00200000:2.0:1787826191.425491:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2f3:0x0](ffff9181c82fb248) 00000080:00200000:2.0:1787826191.425500:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2f4:0x0](ffff9181c83021c8) 00000080:00200000:2.0:1787826191.425509:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2f4:0x0](ffff9181c8301988) 00000080:00200000:2.0:1787826191.425517:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2f5:0x0](ffff9181c8305b88) 00000080:00200000:2.0:1787826191.425524:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2f5:0x0](ffff9181c8303a88) 00000080:00200000:2.0:1787826191.425532:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2f6:0x0](ffff9181c8302a08) 00000080:00200000:2.0:1787826191.425539:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2f6:0x0](ffff9181c8304b08) 00000080:00200000:2.0:1787826191.425547:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2f7:0x0](ffff9181c82f1988) 00000080:00200000:2.0:1787826191.425556:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2f7:0x0](ffff9181c82f21c8) 00000080:00200000:2.0:1787826191.425564:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2f8:0x0](ffff9181c82fcb08) 00000080:00200000:2.0:1787826191.425572:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2f8:0x0](ffff9181c82f8908) 00000080:00200000:2.0:1787826191.425580:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2f9:0x0](ffff9181c82f00c8) 00000080:00200000:2.0:1787826191.425589:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2f9:0x0](ffff9181c82f42c8) 00000080:00200000:2.0:1787826191.425596:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2fa:0x0](ffff9181c8305348) 00000080:00200000:2.0:1787826191.425606:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2fa:0x0](ffff9181c8306c08) 00000080:00200000:2.0:1787826191.425624:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2fb:0x0](ffff9181c82e1148) 00000080:00200000:2.0:1787826191.425633:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2fb:0x0](ffff9181c82e63c8) 00000080:00200000:2.0:1787826191.425641:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2fc:0x0](ffff9181c82fc2c8) 00000080:00200000:2.0:1787826191.425649:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2fc:0x0](ffff9181c82fd348) 00000080:00200000:2.0:1787826191.425657:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2fd:0x0](ffff9181c83063c8) 00000080:00200000:2.0:1787826191.425665:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2fd:0x0](ffff9181c83000c8) 00000080:00200000:2.0:1787826191.425672:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2fe:0x0](ffff9181c82f3248) 00000080:00200000:2.0:1787826191.425680:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2fe:0x0](ffff9181c82f0908) 00000080:00200000:2.0:1787826191.425687:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x2ff:0x0](ffff9181c82fec08) 00000080:00200000:2.0:1787826191.425695:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x2ff:0x0](ffff9181c82f80c8) 00000080:00200000:2.0:1787826191.425719:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x300:0x0](ffff9181c82edb88) 00000080:00200000:2.0:1787826191.425728:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x300:0x0](ffff9181c8308908) 00000080:00200000:2.0:1787826191.425736:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x301:0x0](ffff9181c8303248) 00000080:00200000:3.0:1787826191.436112:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x301:0x0](ffff9181c8307448) 00000080:00200000:3.0:1787826191.436166:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x302:0x0](ffff9181c83042c8) 00000080:00200000:3.0:1787826191.436176:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x302:0x0](ffff9181c8301148) 00000080:00200000:3.0:1787826191.436190:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x303:0x0](ffff9181c82f9148) 00000080:00200000:3.0:1787826191.436202:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x303:0x0](ffff9181c82faa08) 00000080:00200000:3.0:1787826191.436212:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x304:0x0](ffff9181c830c2c8) 00000080:00200000:3.0:1787826191.436224:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x304:0x0](ffff9181c830d348) 00000080:00200000:3.0:1787826191.436232:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x305:0x0](ffff9181c82e7448) 00000080:00200000:3.0:1787826191.436242:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x305:0x0](ffff9181c82e21c8) 00000080:00200000:3.0:1787826191.436250:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x306:0x0](ffff9181c8315b88) 00000080:00200000:3.0:1787826191.436258:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x306:0x0](ffff9181c8313a88) 00000080:00200000:3.0:1787826191.436267:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x307:0x0](ffff9181c82e5b88) 00000080:00200000:3.0:1787826191.436276:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x307:0x0](ffff9181c82e3a88) 00000080:00200000:3.0:1787826191.436284:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x308:0x0](ffff9181c83080c8) 00000080:00200000:3.0:1787826191.436293:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x308:0x0](ffff9181c8309148) 00000080:00200000:3.0:1787826191.436302:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x309:0x0](ffff9181c830e3c8) 00000080:00200000:3.0:1787826191.436311:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x309:0x0](ffff9181c830f448) 00000080:00200000:3.0:1787826191.436321:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x30a:0x0](ffff9181c8314b08) 00000080:00200000:3.0:1787826191.436330:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x30a:0x0](ffff9181c8310908) 00000080:00200000:3.0:1787826191.436338:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x30b:0x0](ffff9181c8315348) 00000080:00200000:3.0:1787826191.436347:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x30b:0x0](ffff9181c8316c08) 00000080:00200000:3.0:1787826191.436356:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x30c:0x0](ffff9181c830db88) 00000080:00200000:3.0:1787826191.436365:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x30c:0x0](ffff9181c830ba88) 00000080:00200000:3.0:1787826191.436373:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x30d:0x0](ffff9181c83100c8) 00000080:00200000:3.0:1787826191.436382:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x30d:0x0](ffff9181c8311148) 00000080:00200000:3.0:1787826191.436391:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x30e:0x0](ffff9181c8309988) 00000080:00200000:3.0:1787826191.436399:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x30e:0x0](ffff9181c830b248) 00000080:00200000:3.0:1787826191.436406:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x30f:0x0](ffff9181c830cb08) 00000080:00200000:3.0:1787826191.436415:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x30f:0x0](ffff9181c830a1c8) 00000080:00200000:3.0:1787826191.436423:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x310:0x0](ffff9181c8317448) 00000080:00200000:3.0:1787826191.436432:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x310:0x0](ffff9181c83121c8) 00000080:00200000:3.0:1787826191.436440:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x311:0x0](ffff9181c82fe3c8) 00000080:00200000:3.0:1787826191.436450:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x311:0x0](ffff9181c831aa08) 00000080:00200000:3.0:1787826191.436458:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x312:0x0](ffff9181c8311988) 00000080:00200000:3.0:1787826191.436466:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x312:0x0](ffff9181c83163c8) 00000080:00200000:3.0:1787826191.436983:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x313:0x0](ffff9181c83142c8) 00000080:00200000:3.0:1787826191.436994:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x313:0x0](ffff9181c8312a08) 00000080:00200000:3.0:1787826191.437002:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x314:0x0](ffff9181c831cb08) 00000080:00200000:3.0:1787826191.437012:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x314:0x0](ffff9181c8318908) 00000080:00200000:3.0:1787826191.437020:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x315:0x0](ffff9181c831c2c8) 00000080:00200000:3.0:1787826191.437028:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x315:0x0](ffff9181c831d348) 00000080:00200000:3.0:1787826191.437036:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x316:0x0](ffff9181c830ec08) 00000080:00200000:3.0:1787826191.437046:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x316:0x0](ffff9181c830aa08) 00000080:00200000:3.0:1787826191.437055:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x317:0x0](ffff9181c8313248) 00000080:00200000:3.0:1787826191.437066:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x317:0x0](ffff9181c82ff448) 00000080:00200000:3.0:1787826191.437074:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x318:0x0](ffff9181c831ec08) 00000080:00200000:3.0:1787826191.437083:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x318:0x0](ffff9181c83180c8) 00000080:00200000:3.0:1787826191.437090:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x319:0x0](ffff9181c8323248) 00000080:00200000:3.0:1787826191.437099:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x319:0x0](ffff9181c8322a08) 00000080:00200000:3.0:1787826191.437107:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x31a:0x0](ffff9181c82e4b08) 00000080:00200000:3.0:1787826191.437117:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x31a:0x0](ffff9181c82e2a08) 00000080:00200000:3.0:1787826191.437124:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x31b:0x0](ffff9181c8300908) 00000080:00200000:3.0:1787826191.437133:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x31b:0x0](ffff9181c83280c8) 00000080:00200000:3.0:1787826191.437141:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x31c:0x0](ffff9181c8329148) 00000080:00200000:3.0:1787826191.437149:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x31c:0x0](ffff9181c832e3c8) 00000080:00200000:3.0:1787826191.437157:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x31d:0x0](ffff9181c832a1c8) 00000080:00200000:3.0:1787826191.437166:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x31d:0x0](ffff9181c8329988) 00000080:00200000:3.0:1787826191.437174:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x31e:0x0](ffff9181c8320908) 00000080:00200000:3.0:1787826191.437181:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x31e:0x0](ffff9181c83242c8) 00000080:00200000:3.0:1787826191.437189:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x31f:0x0](ffff9181c82e3248) 00000080:00200000:3.0:1787826191.437197:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x31f:0x0](ffff9181c82e1988) 00000080:00200000:3.0:1787826191.437205:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x320:0x0](ffff9181c8319148) 00000080:00200000:3.0:1787826191.437214:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x320:0x0](ffff9181c831e3c8) 00000080:00200000:3.0:1787826191.437223:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x321:0x0](ffff9181c832ba88) 00000080:00200000:3.0:1787826191.437231:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x321:0x0](ffff9181c832b248) 00000080:00200000:3.0:1787826191.437238:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x322:0x0](ffff9181c831f448) 00000080:00200000:3.0:1787826191.437246:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x322:0x0](ffff9181c831a1c8) 00000080:00200000:3.0:1787826191.437255:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x323:0x0](ffff9181c832aa08) 00000080:00200000:3.0:1787826191.437263:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x323:0x0](ffff9181c832cb08) 00000080:00200000:3.0:1787826191.437270:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x324:0x0](ffff9181c83263c8) 00000080:00200000:3.0:1787826191.437279:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x324:0x0](ffff9181c8327448) 00000080:00200000:3.0:1787826191.437287:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x325:0x0](ffff9181c83221c8) 00000080:00200000:3.0:1787826191.437295:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x325:0x0](ffff9181c8321988) 00000080:00200000:3.0:1787826191.437303:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x326:0x0](ffff9181c8328908) 00000080:00200000:3.0:1787826191.437311:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x326:0x0](ffff9181c832c2c8) 00000080:00200000:3.0:1787826191.437318:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x327:0x0](ffff9181c8326c08) 00000080:00200000:3.0:1787826191.437326:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x327:0x0](ffff9181c8325348) 00000080:00200000:3.0:1787826191.437334:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x328:0x0](ffff9181c82e5348) 00000080:00200000:3.0:1787826191.437343:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x328:0x0](ffff9181c83300c8) 00000080:00200000:3.0:1787826191.437350:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x329:0x0](ffff9181c83363c8) 00000080:00200000:3.0:1787826191.437358:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x329:0x0](ffff9181c8337448) 00000080:00200000:3.0:1787826191.437366:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x32a:0x0](ffff9181c83200c8) 00000080:00200000:3.0:1787826191.437374:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x32a:0x0](ffff9181c8324b08) 00000080:00200000:3.0:1787826191.437382:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x32b:0x0](ffff9181c8323a88) 00000080:00200000:3.0:1787826191.437390:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x32b:0x0](ffff9181c8325b88) 00000080:00200000:3.0:1787826191.437398:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x32c:0x0](ffff9181c8321148) 00000080:00200000:3.0:1787826191.437405:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x32c:0x0](ffff9181c833a1c8) 00000080:00200000:3.0:1787826191.437414:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x32d:0x0](ffff9181c831b248) 00000080:00200000:3.0:1787826191.437422:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x32d:0x0](ffff9181c831db88) 00000080:00200000:3.0:1787826191.437429:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x32e:0x0](ffff9181c832ec08) 00000080:00200000:3.0:1787826191.437438:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x32e:0x0](ffff9181c832db88) 00000080:00200000:3.0:1787826191.437445:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x32f:0x0](ffff9181c832d348) 00000080:00200000:3.0:1787826191.437454:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x32f:0x0](ffff9181c8345348) 00000080:00200000:3.0:1787826191.437462:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x330:0x0](ffff9181c8339988) 00000080:00200000:0.0:1787826191.443110:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x330:0x0](ffff9181c833db88) 00000080:00200000:0.0:1787826191.443148:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x331:0x0](ffff9181c83400c8) 00000080:00200000:0.0:1787826191.443158:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x331:0x0](ffff9181c8341148) 00000080:00200000:0.0:1787826191.443166:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x332:0x0](ffff9181c8347448) 00000080:00200000:0.0:1787826191.443175:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x332:0x0](ffff9181c83421c8) 00000080:00200000:0.0:1787826191.443183:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x333:0x0](ffff9181c833b248) 00000080:00200000:0.0:1787826191.443191:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x333:0x0](ffff9181c833aa08) 00000080:00200000:0.0:1787826191.443198:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x334:0x0](ffff9181c8338908) 00000080:00200000:0.0:1787826191.443207:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x334:0x0](ffff9181c833c2c8) 00000080:00200000:0.0:1787826191.443215:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x335:0x0](ffff9181c8333a88) 00000080:00200000:0.0:1787826191.443227:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x335:0x0](ffff9181c8333248) 00000080:00200000:0.0:1787826191.443236:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x336:0x0](ffff9181c8345b88) 00000080:00200000:0.0:1787826191.443246:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x336:0x0](ffff9181c8343a88) 00000080:00200000:0.0:1787826191.443257:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x337:0x0](ffff9181c8342a08) 00000080:00200000:0.0:1787826191.443267:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x337:0x0](ffff9181c8344b08) 00000080:00200000:0.0:1787826191.443276:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x338:0x0](ffff9181c833ec08) 00000080:00200000:0.0:1787826191.443286:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x338:0x0](ffff9181c83380c8) 00000080:00200000:0.0:1787826191.443296:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x339:0x0](ffff9181c8339148) 00000080:00200000:0.0:1787826191.443307:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x339:0x0](ffff9181c833e3c8) 00000080:00200000:0.0:1787826191.443317:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x33a:0x0](ffff9181c8335b88) 00000080:00200000:0.0:1787826191.443327:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x33a:0x0](ffff9181c8332a08) 00000080:00200000:0.0:1787826191.443334:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x33b:0x0](ffff9181c8334b08) 00000080:00200000:0.0:1787826191.443344:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x33b:0x0](ffff9181c8330908) 00000080:00200000:0.0:1787826191.443352:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x33c:0x0](ffff9181c833d348) 00000080:00200000:0.0:1787826191.443362:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x33c:0x0](ffff9181c833cb08) 00000080:00200000:0.0:1787826191.443370:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x33d:0x0](ffff9181c83342c8) 00000080:00200000:0.0:1787826191.443378:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x33d:0x0](ffff9181c8335348) 00000080:00200000:0.0:1787826191.443385:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x33e:0x0](ffff9181c83442c8) 00000080:00200000:0.0:1787826191.443393:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x33e:0x0](ffff9181c8340908) 00000080:00200000:0.0:1787826191.443399:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x33f:0x0](ffff9181c8343248) 00000080:00200000:0.0:1787826191.443407:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x33f:0x0](ffff9181c8341988) 00000080:00200000:0.0:1787826191.443413:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x340:0x0](ffff9181c833ba88) 00000080:00200000:0.0:1787826191.443422:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x340:0x0](ffff9181c833f448) 00000080:00200000:0.0:1787826191.443429:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x341:0x0](ffff9181c8346c08) 00000080:00200000:0.0:1787826191.443437:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x341:0x0](ffff9181c83463c8) 00000080:00200000:0.0:1787826191.443443:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x342:0x0](ffff9181c834aa08) 00000080:00200000:0.0:1787826191.443452:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x342:0x0](ffff9181c834cb08) 00000080:00200000:0.0:1787826191.443459:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x343:0x0](ffff9181c8319988) 00000080:00200000:0.0:1787826191.443482:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x343:0x0](ffff9181c831ba88) 00000080:00200000:0.0:1787826191.443491:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x344:0x0](ffff9181c8348908) 00000080:00200000:0.0:1787826191.443499:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x344:0x0](ffff9181c834c2c8) 00000080:00200000:0.0:1787826191.443506:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x345:0x0](ffff9181c834d348) 00000080:00200000:0.0:1787826191.443515:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x345:0x0](ffff9181c834ec08) 00000080:00200000:0.0:1787826191.443522:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x346:0x0](ffff9181c83480c8) 00000080:00200000:0.0:1787826191.443553:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x346:0x0](ffff9181c8349148) 00000080:00200000:0.0:1787826191.443562:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x347:0x0](ffff9181c8336c08) 00000080:00200000:0.0:1787826191.443571:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x347:0x0](ffff9181c8331988) 00000080:00200000:0.0:1787826191.443579:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x348:0x0](ffff9181c832f448) 00000080:00200000:0.0:1787826191.443588:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x348:0x0](ffff9181c8351148) 00000080:00200000:0.0:1787826191.443595:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x349:0x0](ffff9181c8349988) 00000080:00200000:0.0:1787826191.443604:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x349:0x0](ffff9181c834db88) 00000080:00200000:0.0:1787826191.443612:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x34a:0x0](ffff9181c83563c8) 00000080:00200000:0.0:1787826191.443620:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x34a:0x0](ffff9181c8357448) 00000080:00200000:0.0:1787826191.443627:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x34b:0x0](ffff9181c8351988) 00000080:00200000:0.0:1787826191.443635:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x34b:0x0](ffff9181c8355b88) 00000080:00200000:0.0:1787826191.443642:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x34c:0x0](ffff9181c83521c8) 00000080:00200000:0.0:1787826191.443650:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x34c:0x0](ffff9181c8353a88) 00000080:00200000:0.0:1787826191.443658:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x34d:0x0](ffff9181c834ba88) 00000080:00200000:0.0:1787826191.443665:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x34d:0x0](ffff9181c834a1c8) 00000080:00200000:0.0:1787826191.443672:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x34e:0x0](ffff9181c835f448) 00000080:00200000:0.0:1787826191.443682:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x34e:0x0](ffff9181c835a1c8) 00000080:00200000:0.0:1787826191.443688:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x34f:0x0](ffff9181c835e3c8) 00000080:00200000:0.0:1787826191.443696:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x34f:0x0](ffff9181c8359988) 00000080:00200000:0.0:1787826191.443703:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x350:0x0](ffff9181c834f448) 00000080:00200000:0.0:1787826191.443711:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x350:0x0](ffff9181c834e3c8) 00000080:00200000:0.0:1787826191.443728:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x351:0x0](ffff9181c834b248) 00000080:00200000:0.0:1787826191.443736:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x351:0x0](ffff9181c83600c8) 00000080:00200000:1.0:1787826191.450593:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x352:0x0](ffff9181c8350908) 00000080:00200000:1.0:1787826191.450641:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x352:0x0](ffff9181c83542c8) 00000080:00200000:1.0:1787826191.450651:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x353:0x0](ffff9181c83663c8) 00000080:00200000:1.0:1787826191.450660:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x353:0x0](ffff9181c8367448) 00000080:00200000:1.0:1787826191.450669:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x354:0x0](ffff9181c835ba88) 00000080:00200000:1.0:1787826191.450682:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x354:0x0](ffff9181c835b248) 00000080:00200000:1.0:1787826191.450691:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x355:0x0](ffff9181c8354b08) 00000080:00200000:1.0:1787826191.450713:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x355:0x0](ffff9181c8355348) 00000080:00200000:1.0:1787826191.450723:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x356:0x0](ffff9181c8331148) 00000080:00200000:1.0:1787826191.450735:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x356:0x0](ffff9181c836c2c8) 00000080:00200000:1.0:1787826191.451070:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x357:0x0](ffff9181c8361988) 00000080:00200000:1.0:1787826191.451083:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x357:0x0](ffff9181c8365b88) 00000080:00200000:1.0:1787826191.451092:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x358:0x0](ffff9181c835aa08) 00000080:00200000:1.0:1787826191.451101:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x358:0x0](ffff9181c835cb08) 00000080:00200000:1.0:1787826191.451110:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x359:0x0](ffff9181c8356c08) 00000080:00200000:1.0:1787826191.451119:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x359:0x0](ffff9181c83500c8) 00000080:00200000:1.0:1787826191.451126:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x35a:0x0](ffff9181c8362a08) 00000080:00200000:1.0:1787826191.451135:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x35a:0x0](ffff9181c8364b08) 00000080:00200000:1.0:1787826191.451141:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x35b:0x0](ffff9181c8352a08) 00000080:00200000:1.0:1787826191.451149:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x35b:0x0](ffff9181c8353248) 00000080:00200000:1.0:1787826191.451165:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x35c:0x0](ffff9181c835c2c8) 00000080:00200000:1.0:1787826191.451181:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x35c:0x0](ffff9181c835d348) 00000080:00200000:1.0:1787826191.451190:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x35d:0x0](ffff9181c8360908) 00000080:00200000:1.0:1787826191.451198:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x35d:0x0](ffff9181c83642c8) 00000080:00200000:1.0:1787826191.451205:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x35e:0x0](ffff9181c835ec08) 00000080:00200000:1.0:1787826191.451212:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x35e:0x0](ffff9181c83580c8) 00000080:00200000:1.0:1787826191.451219:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x35f:0x0](ffff9181c8366c08) 00000080:00200000:1.0:1787826191.451228:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x35f:0x0](ffff9181c8363248) 00000080:00200000:1.0:1787826191.451235:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x360:0x0](ffff9181c8374b08) 00000080:00200000:1.0:1787826191.451243:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x360:0x0](ffff9181c8370908) 00000080:00200000:1.0:1787826191.451251:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x361:0x0](ffff9181c836ec08) 00000080:00200000:1.0:1787826191.451259:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x361:0x0](ffff9181c83680c8) 00000080:00200000:1.0:1787826191.451265:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x362:0x0](ffff9181c8363a88) 00000080:00200000:1.0:1787826191.451273:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x362:0x0](ffff9181c83621c8) 00000080:00200000:1.0:1787826191.451280:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x363:0x0](ffff9181c836d348) 00000080:00200000:1.0:1787826191.451287:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x363:0x0](ffff9181c8369148) 00000080:00200000:1.0:1787826191.451295:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x364:0x0](ffff9181c8365348) 00000080:00200000:1.0:1787826191.451303:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x364:0x0](ffff9181c8361148) 00000080:00200000:1.0:1787826191.451311:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x365:0x0](ffff9181c83321c8) 00000080:00200000:1.0:1787826191.451319:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x365:0x0](ffff9181c8378908) 00000080:00200000:1.0:1787826191.451326:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x366:0x0](ffff9181c835db88) 00000080:00200000:1.0:1787826191.451333:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x366:0x0](ffff9181c8359148) 00000080:00200000:1.0:1787826191.451341:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x367:0x0](ffff9181c837d348) 00000080:00200000:1.0:1787826191.451348:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x367:0x0](ffff9181c837ec08) 00000080:00200000:1.0:1787826191.451356:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x368:0x0](ffff9181c837c2c8) 00000080:00200000:1.0:1787826191.451365:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x368:0x0](ffff9181c83780c8) 00000080:00200000:1.0:1787826191.451372:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x369:0x0](ffff9181c8358908) 00000080:00200000:1.0:1787826191.451380:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x369:0x0](ffff9181c83800c8) 00000080:00200000:1.0:1787826191.451388:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x36a:0x0](ffff9181c837f448) 00000080:00200000:1.0:1787826191.451396:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x36a:0x0](ffff9181c837a1c8) 00000080:00200000:1.0:1787826191.451404:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x36b:0x0](ffff9181c8381148) 00000080:00200000:1.0:1787826191.451413:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x36b:0x0](ffff9181c83863c8) 00000080:00200000:1.0:1787826191.451421:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x36c:0x0](ffff9181c8376c08) 00000080:00200000:1.0:1787826191.451429:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x36c:0x0](ffff9181c83700c8) 00000080:00200000:1.0:1787826191.451437:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x36d:0x0](ffff9181c8371148) 00000080:00200000:1.0:1787826191.451445:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x36d:0x0](ffff9181c83763c8) 00000080:00200000:1.0:1787826191.451453:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x36e:0x0](ffff9181c83821c8) 00000080:00200000:1.0:1787826191.451462:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x36e:0x0](ffff9181c8381988) 00000080:00200000:1.0:1787826191.454969:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x36f:0x0](ffff9181c837ba88) 00000080:00200000:1.0:1787826191.455005:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x36f:0x0](ffff9181c837b248) 00000080:00200000:1.0:1787826191.455015:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x370:0x0](ffff9181c8377448) 00000080:00200000:1.0:1787826191.455029:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x370:0x0](ffff9181c83721c8) 00000080:00200000:1.0:1787826191.455039:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x371:0x0](ffff9181c8385b88) 00000080:00200000:1.0:1787826191.455048:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x371:0x0](ffff9181c8383a88) 00000080:00200000:1.0:1787826191.455055:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x372:0x0](ffff9181c836f448) 00000080:00200000:1.0:1787826191.455064:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x372:0x0](ffff9181c836a1c8) 00000080:00200000:1.0:1787826191.455071:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x373:0x0](ffff9181c837cb08) 00000080:00200000:1.0:1787826191.455079:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x373:0x0](ffff9181c837db88) 00000080:00200000:1.0:1787826191.455085:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x374:0x0](ffff9181c8379988) 00000080:00200000:1.0:1787826191.455093:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x374:0x0](ffff9181c837e3c8) 00000080:00200000:1.0:1787826191.455101:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x375:0x0](ffff9181c8371988) 00000080:00200000:1.0:1787826191.455109:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x375:0x0](ffff9181c8375b88) 00000080:00200000:1.0:1787826191.455115:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x376:0x0](ffff9181c8369988) 00000080:00200000:1.0:1787826191.455123:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x376:0x0](ffff9181c836db88) 00000080:00200000:1.0:1787826191.455130:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x377:0x0](ffff9181c836ba88) 00000080:00200000:1.0:1787826191.455137:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x377:0x0](ffff9181c836b248) 00000080:00200000:1.0:1787826191.455144:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x378:0x0](ffff9181c8383248) 00000080:00200000:1.0:1787826191.455153:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x378:0x0](ffff9181c8382a08) 00000080:00200000:1.0:1787826191.455162:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x379:0x0](ffff9181c8373248) 00000080:00200000:1.0:1787826191.455171:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x379:0x0](ffff9181c8373a88) 00000080:00200000:1.0:1787826191.455178:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x37a:0x0](ffff9181c8379148) 00000080:00200000:1.0:1787826191.455187:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x37a:0x0](ffff9181c838f448) 00000080:00200000:1.0:1787826191.455195:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x37b:0x0](ffff9181c8375348) 00000080:00200000:1.0:1787826191.455202:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x37b:0x0](ffff9181c83742c8) 00000080:00200000:1.0:1787826191.455209:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x37c:0x0](ffff9181c838db88) 00000080:00200000:1.0:1787826191.455218:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x37c:0x0](ffff9181c838ba88) 00000080:00200000:1.0:1787826191.455226:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x37d:0x0](ffff9181c836aa08) 00000080:00200000:1.0:1787826191.455234:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x37d:0x0](ffff9181c836cb08) 00000080:00200000:1.0:1787826191.455241:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x37e:0x0](ffff9181c8391988) 00000080:00200000:1.0:1787826191.455248:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x37e:0x0](ffff9181c8395b88) 00000080:00200000:1.0:1787826191.455255:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x37f:0x0](ffff9181c8389988) 00000080:00200000:1.0:1787826191.455264:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x37f:0x0](ffff9181c838b248) 00000080:00200000:1.0:1787826191.455271:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x380:0x0](ffff9181c8380908) 00000080:00200000:1.0:1787826191.455279:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x380:0x0](ffff9181c83842c8) 00000080:00200000:1.0:1787826191.455285:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x381:0x0](ffff9181c8392a08) 00000080:00200000:1.0:1787826191.455292:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x381:0x0](ffff9181c8394b08) 00000080:00200000:1.0:1787826191.455300:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x382:0x0](ffff9181c8384b08) 00000080:00200000:1.0:1787826191.455307:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x382:0x0](ffff9181c8385348) 00000080:00200000:1.0:1787826191.455313:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x383:0x0](ffff9181c8390908) 00000080:00200000:1.0:1787826191.455321:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x383:0x0](ffff9181c83942c8) 00000080:00200000:1.0:1787826191.455328:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x384:0x0](ffff9181c8395348) 00000080:00200000:1.0:1787826191.455335:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x384:0x0](ffff9181c8396c08) 00000080:00200000:1.0:1787826191.455344:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x385:0x0](ffff9181c83900c8) 00000080:00200000:1.0:1787826191.455351:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x385:0x0](ffff9181c8391148) 00000080:00200000:1.0:1787826191.455357:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x386:0x0](ffff9181c8386c08) 00000080:00200000:1.0:1787826191.455366:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x386:0x0](ffff9181c8387448) 00000080:00200000:1.0:1787826191.455373:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x387:0x0](ffff9181c838c2c8) 00000080:00200000:1.0:1787826191.455381:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x387:0x0](ffff9181c838d348) 00000080:00200000:1.0:1787826191.455389:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x388:0x0](ffff9181c8368908) 00000080:00200000:1.0:1787826191.455398:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x388:0x0](ffff9181c836e3c8) 00000080:00200000:1.0:1787826191.455406:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x389:0x0](ffff9181c839aa08) 00000080:00200000:1.0:1787826191.455415:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x389:0x0](ffff9181c839cb08) 00000080:00200000:1.0:1787826191.455424:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x38a:0x0](ffff9181c8389148) 00000080:00200000:1.0:1787826191.455432:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x38a:0x0](ffff9181c838e3c8) 00000080:00200000:1.0:1787826191.455441:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x38b:0x0](ffff9181c8397448) 00000080:00200000:1.0:1787826191.455450:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x38b:0x0](ffff9181c83921c8) 00000080:00200000:1.0:1787826191.455458:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x38c:0x0](ffff9181c83963c8) 00000080:00200000:1.0:1787826191.455465:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x38c:0x0](ffff9181c8393248) 00000080:00200000:1.0:1787826191.456305:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x38d:0x0](ffff9181c8393a88) 00000080:00200000:1.0:1787826191.456315:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x38d:0x0](ffff9181c8372a08) 00000080:00200000:1.0:1787826191.456323:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x38e:0x0](ffff9181c837aa08) 00000080:00200000:1.0:1787826191.456331:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x38e:0x0](ffff9181c83a2a08) 00000080:00200000:1.0:1787826191.456339:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x38f:0x0](ffff9181c83aa1c8) 00000080:00200000:1.0:1787826191.456347:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x38f:0x0](ffff9181c83a9988) 00000080:00200000:1.0:1787826191.456355:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x390:0x0](ffff9181c838aa08) 00000080:00200000:1.0:1787826191.456367:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x390:0x0](ffff9181c838a1c8) 00000080:00200000:1.0:1787826191.456375:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x391:0x0](ffff9181c8388908) 00000080:00200000:1.0:1787826191.456383:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x391:0x0](ffff9181c838ec08) 00000080:00200000:1.0:1787826191.456391:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x392:0x0](ffff9181c83aba88) 00000080:00200000:1.0:1787826191.456398:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x392:0x0](ffff9181c83ab248) 00000080:00200000:1.0:1787826191.456406:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x393:0x0](ffff9181c838cb08) 00000080:00200000:1.0:1787826191.456414:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x393:0x0](ffff9181c83880c8) 00000080:00200000:1.0:1787826191.456421:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x394:0x0](ffff9181c83a42c8) 00000080:00200000:1.0:1787826191.456430:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x394:0x0](ffff9181c83a5348) 00000080:00200000:1.0:1787826191.456436:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x395:0x0](ffff9181c83acb08) 00000080:00200000:1.0:1787826191.456444:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x395:0x0](ffff9181c83a8908) 00000080:00200000:1.0:1787826191.456451:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x396:0x0](ffff9181c83b63c8) 00000080:00200000:1.0:1787826191.456464:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x396:0x0](ffff9181c83b7448) 00000080:00200000:1.0:1787826191.456584:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x397:0x0](ffff9181c83ac2c8) 00000080:00200000:1.0:1787826191.456594:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x397:0x0](ffff9181c83ad348) 00000080:00200000:1.0:1787826191.456602:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x398:0x0](ffff9181c83a80c8) 00000080:00200000:1.0:1787826191.456667:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x398:0x0](ffff9181c83a9148) 00000080:00200000:1.0:1787826191.456676:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x399:0x0](ffff9181c83b21c8) 00000080:00200000:1.0:1787826191.456685:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x399:0x0](ffff9181c83b1988) 00000080:00200000:1.0:1787826191.456692:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x39a:0x0](ffff9181c83b5b88) 00000080:00200000:1.0:1787826191.457048:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x39a:0x0](ffff9181c83b3a88) 00000080:00200000:1.0:1787826191.457062:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x39b:0x0](ffff9181c83ae3c8) 00000080:00200000:1.0:1787826191.457071:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x39b:0x0](ffff9181c83af448) 00000080:00200000:1.0:1787826191.457078:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x39c:0x0](ffff9181c83a6c08) 00000080:00200000:1.0:1787826191.457087:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x39c:0x0](ffff9181c83a00c8) 00000080:00200000:1.0:1787826191.457095:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x39d:0x0](ffff9181c83aec08) 00000080:00200000:1.0:1787826191.457103:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x39d:0x0](ffff9181c83aaa08) 00000080:00200000:1.0:1787826191.457109:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x39e:0x0](ffff9181c83adb88) 00000080:00200000:1.0:1787826191.457118:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x39e:0x0](ffff9181c83ba1c8) 00000080:00200000:1.0:1787826191.457126:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x39f:0x0](ffff9181c83a63c8) 00000080:00200000:1.0:1787826191.457134:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x39f:0x0](ffff9181c83a7448) 00000080:00200000:1.0:1787826191.457141:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x3a0:0x0](ffff9181c83bba88) 00000080:00200000:1.0:1787826191.457149:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x3a0:0x0](ffff9181c83bb248) 00000080:00200000:1.0:1787826191.459395:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x3a1:0x0](ffff9181c83b4b08) 00000080:00200000:1.0:1787826191.459424:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x3a1:0x0](ffff9181c83b0908) 00000080:00200000:1.0:1787826191.459433:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x3a2:0x0](ffff9181c83b5348) 00000080:00200000:1.0:1787826191.459442:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x3a2:0x0](ffff9181c83b6c08) 00000080:00200000:1.0:1787826191.459451:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x3a3:0x0](ffff9181c83baa08) 00000080:00200000:1.0:1787826191.459458:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x3a3:0x0](ffff9181c83bcb08) 00000080:00200000:1.0:1787826191.460279:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x3a4:0x0](ffff9181c83bdb88) 00000080:00200000:1.0:1787826191.460299:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x3a4:0x0](ffff9181c83b8908) 00000080:00200000:1.0:1787826191.460308:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x3a5:0x0](ffff9181c83b3248) 00000080:00200000:1.0:1787826191.460316:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x3a5:0x0](ffff9181c83b1148) 00000080:00200000:1.0:1787826191.460324:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x3a6:0x0](ffff9181c8398908) 00000080:00200000:1.0:1787826191.460333:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x3a6:0x0](ffff9181c839c2c8) 00000080:00200000:1.0:1787826191.460341:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x3a7:0x0](ffff9181c83bd348) 00000080:00200000:1.0:1787826191.460350:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x3a7:0x0](ffff9181c83bec08) 00000080:00200000:1.0:1787826191.460358:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x3a8:0x0](ffff9181c83b80c8) 00000080:00200000:1.0:1787826191.460366:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x3a8:0x0](ffff9181c83b9148) 00000080:00200000:1.0:1787826191.460373:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x3a9:0x0](ffff9181c83b00c8) 00000080:00200000:1.0:1787826191.460380:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x3a9:0x0](ffff9181c83b2a08) 00000080:00200000:1.0:1787826191.460388:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x3aa:0x0](ffff9181c83bf448) 00000080:00200000:1.0:1787826191.460396:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x3aa:0x0](ffff9181c83bc2c8) 00000080:00200000:1.0:1787826191.460404:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x3ab:0x0](ffff9181c839d348) 00000080:00200000:1.0:1787826191.460412:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x3ab:0x0](ffff9181c839ec08) 00000080:00200000:1.0:1787826191.460420:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x3ac:0x0](ffff9181c83be3c8) 00000080:00200000:1.0:1787826191.460428:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x3ac:0x0](ffff9181c83b9988) 00000080:00200000:1.0:1787826191.460436:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x3ad:0x0](ffff9181c83c2a08) 00000080:00200000:1.0:1787826191.460445:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x3ad:0x0](ffff9181c83c4b08) 00000080:00200000:1.0:1787826191.460453:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x3ae:0x0](ffff9181c83980c8) 00000080:00200000:1.0:1787826191.460462:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x3ae:0x0](ffff9181c8399148) 00000080:00200000:1.0:1787826191.461889:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200025d90:0x3af:0x0](ffff9181c83c0908) 00000080:00200000:1.0:1787826191.462017:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x24000178a:0x3af:0x0](ffff9181c83c42c8) 00000080:00200000:1.0:1787826191.462027:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x3:0x0](ffff9181c839e3c8) 00000080:00200000:1.0:1787826191.462036:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x3:0x0](ffff9181c839f448) 00000080:00200000:1.0:1787826191.462045:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x4:0x0](ffff9181c83b42c8) 00000080:00200000:1.0:1787826191.462054:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x4:0x0](ffff9181c83cba88) 00000080:00200000:1.0:1787826191.462064:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x5:0x0](ffff9181c83caa08) 00000080:00200000:1.0:1787826191.462072:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x5:0x0](ffff9181c83ccb08) 00000080:00200000:1.0:1787826191.462081:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x6:0x0](ffff9181c83c8908) 00000080:00200000:1.0:1787826191.462091:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x6:0x0](ffff9181c83cc2c8) 00000080:00200000:1.0:1787826191.462100:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x7:0x0](ffff9181c83c5348) 00000080:00200000:1.0:1787826191.462109:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x7:0x0](ffff9181c83c6c08) 00000080:00200000:1.0:1787826191.462117:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x8:0x0](ffff9181c83a5b88) 00000080:00200000:1.0:1787826191.462127:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x8:0x0](ffff9181c83a3a88) 00000080:00200000:1.0:1787826191.462137:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x9:0x0](ffff9181c83cec08) 00000080:00200000:1.0:1787826191.462146:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x9:0x0](ffff9181c83c80c8) 00000080:00200000:1.0:1787826191.462154:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0xa:0x0](ffff9181c83cd348) 00000080:00200000:1.0:1787826191.462163:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0xa:0x0](ffff9181c83c9148) 00000080:00200000:1.0:1787826191.462171:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0xb:0x0](ffff9181c83c1148) 00000080:00200000:1.0:1787826191.462179:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0xb:0x0](ffff9181c83c63c8) 00000080:00200000:1.0:1787826191.462187:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0xc:0x0](ffff9181c83cf448) 00000080:00200000:1.0:1787826191.462197:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0xc:0x0](ffff9181c83ca1c8) 00000080:00200000:1.0:1787826191.462204:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0xd:0x0](ffff9181c83c9988) 00000080:00200000:1.0:1787826191.462212:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0xd:0x0](ffff9181c83cdb88) 00000080:00200000:1.0:1787826191.462220:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0xe:0x0](ffff9181c83c21c8) 00000080:00200000:1.0:1787826191.462228:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0xe:0x0](ffff9181c83c1988) 00000080:00200000:1.0:1787826191.462236:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0xf:0x0](ffff9181c83ce3c8) 00000080:00200000:1.0:1787826191.462245:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0xf:0x0](ffff9181c83d21c8) 00000080:00200000:1.0:1787826191.462252:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x10:0x0](ffff9181c83d5b88) 00000080:00200000:1.0:1787826191.462262:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x10:0x0](ffff9181c83d3a88) 00000080:00200000:1.0:1787826191.462271:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x11:0x0](ffff9181c83d3248) 00000080:00200000:1.0:1787826191.462279:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x11:0x0](ffff9181c83d2a08) 00000080:00200000:1.0:1787826191.462287:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x12:0x0](ffff9181c83c3248) 00000080:00200000:1.0:1787826191.462296:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x12:0x0](ffff9181c83c5b88) 00000080:00200000:1.0:1787826191.462305:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x13:0x0](ffff9181c83a3248) 00000080:00200000:1.0:1787826191.462313:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x13:0x0](ffff9181c83a1988) 00000080:00200000:1.0:1787826191.462320:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x14:0x0](ffff9181c83d4b08) 00000080:00200000:1.0:1787826191.462329:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x14:0x0](ffff9181c83d0908) 00000080:00200000:1.0:1787826191.462338:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x15:0x0](ffff9181c83c00c8) 00000080:00200000:1.0:1787826191.462347:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x15:0x0](ffff9181c83c7448) 00000080:00200000:1.0:1787826191.462355:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x16:0x0](ffff9181c839b248) 00000080:00200000:1.0:1787826191.462364:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x16:0x0](ffff9181c839db88) 00000080:00200000:1.0:1787826191.462372:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x17:0x0](ffff9181c83d5348) 00000080:00200000:1.0:1787826191.462385:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x17:0x0](ffff9181c83d6c08) 00000080:00200000:1.0:1787826191.462393:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x18:0x0](ffff9181c839ba88) 00000080:00200000:1.0:1787826191.462402:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x18:0x0](ffff9181c8399988) 00000080:00200000:1.0:1787826191.462410:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x19:0x0](ffff9181c83c3a88) 00000080:00200000:1.0:1787826191.462417:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x19:0x0](ffff9181c83d9988) 00000080:00200000:1.0:1787826191.462424:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x1a:0x0](ffff9181c83a21c8) 00000080:00200000:1.0:1787826191.462433:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x1a:0x0](ffff9181c83a1148) 00000080:00200000:1.0:1787826191.462440:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x1b:0x0](ffff9181c83d00c8) 00000080:00200000:1.0:1787826191.462449:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x1b:0x0](ffff9181c83d1148) 00000080:00200000:1.0:1787826191.462457:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x1c:0x0](ffff9181c83dba88) 00000080:00200000:1.0:1787826191.462464:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x1c:0x0](ffff9181c83db248) 00000080:00200000:1.0:1787826191.463016:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x1d:0x0](ffff9181c83daa08) 00000080:00200000:1.0:1787826191.463026:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x1d:0x0](ffff9181c83dcb08) 00000080:00200000:1.0:1787826191.463033:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x1e:0x0](ffff9181c839a1c8) 00000080:00200000:1.0:1787826191.463041:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x1e:0x0](ffff9181c83e0908) 00000080:00200000:1.0:1787826191.463049:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x1f:0x0](ffff9181c83dc2c8) 00000080:00200000:1.0:1787826191.463056:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x1f:0x0](ffff9181c83dd348) 00000080:00200000:1.0:1787826191.463064:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x20:0x0](ffff9181c83e42c8) 00000080:00200000:1.0:1787826191.463071:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x20:0x0](ffff9181c83e5348) 00000080:00200000:1.0:1787826191.463077:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x21:0x0](ffff9181c83dec08) 00000080:00200000:1.0:1787826191.463085:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x21:0x0](ffff9181c83d80c8) 00000080:00200000:1.0:1787826191.463092:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x22:0x0](ffff9181c83d7448) 00000080:00200000:1.0:1787826191.463101:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x22:0x0](ffff9181c83d63c8) 00000080:00200000:1.0:1787826191.463108:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x23:0x0](ffff9181c83d1988) 00000080:00200000:1.0:1787826191.463115:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x23:0x0](ffff9181c83e80c8) 00000080:00200000:1.0:1787826191.463122:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x24:0x0](ffff9181c83d9148) 00000080:00200000:1.0:1787826191.463129:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x24:0x0](ffff9181c83de3c8) 00000080:00200000:1.0:1787826191.463137:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x25:0x0](ffff9181c83df448) 00000080:00200000:1.0:1787826191.463144:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x25:0x0](ffff9181c83da1c8) 00000080:00200000:1.0:1787826191.466207:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x26:0x0](ffff9181c83a4b08) 00000080:00200000:1.0:1787826191.466253:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x26:0x0](ffff9181c83a0908) 00000080:00200000:1.0:1787826191.466262:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x27:0x0](ffff9181c83d8908) 00000080:00200000:1.0:1787826191.466272:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x27:0x0](ffff9181c83ddb88) 00000080:00200000:1.0:1787826191.466281:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x28:0x0](ffff9181c83f6c08) 00000080:00200000:1.0:1787826191.466290:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x28:0x0](ffff9181c83f00c8) 00000080:00200000:1.0:1787826191.466297:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x29:0x0](ffff9181c83e6c08) 00000080:00200000:1.0:1787826191.466307:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x29:0x0](ffff9181c83e00c8) 00000080:00200000:1.0:1787826191.466314:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x2a:0x0](ffff9181c83f9988) 00000080:00200000:1.0:1787826191.466322:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x2a:0x0](ffff9181c83fdb88) 00000080:00200000:1.0:1787826191.466331:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x2b:0x0](ffff9181c83e1148) 00000080:00200000:1.0:1787826191.466340:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x2b:0x0](ffff9181c83e63c8) 00000080:00200000:1.0:1787826191.466349:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x2c:0x0](ffff9181c83fba88) 00000080:00200000:1.0:1787826191.466356:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x2c:0x0](ffff9181c83fb248) 00000080:00200000:1.0:1787826191.466363:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x2d:0x0](ffff9181c83f1148) 00000080:00200000:1.0:1787826191.466370:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x2d:0x0](ffff9181c83f63c8) 00000080:00200000:1.0:1787826191.466377:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x2e:0x0](ffff9181c83faa08) 00000080:00200000:1.0:1787826191.466388:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x2e:0x0](ffff9181c83fcb08) 00000080:00200000:1.0:1787826191.466395:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x2f:0x0](ffff9181c83f7448) 00000080:00200000:1.0:1787826191.466402:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x2f:0x0](ffff9181c83f21c8) 00000080:00200000:1.0:1787826191.466410:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x30:0x0](ffff9181c83e9988) 00000080:00200000:1.0:1787826191.466418:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x30:0x0](ffff9181c83edb88) 00000080:00200000:1.0:1787826191.466424:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x31:0x0](ffff9181c83eba88) 00000080:00200000:1.0:1787826191.466431:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x31:0x0](ffff9181c83eb248) 00000080:00200000:1.0:1787826191.466437:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x32:0x0](ffff9181c83eaa08) 00000080:00200000:1.0:1787826191.466444:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x32:0x0](ffff9181c83ecb08) 00000080:00200000:1.0:1787826191.466451:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x33:0x0](ffff9181c83e8908) 00000080:00200000:1.0:1787826191.466458:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x33:0x0](ffff9181c83ec2c8) 00000080:00200000:1.0:1787826191.466477:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x34:0x0](ffff9181c83fd348) 00000080:00200000:1.0:1787826191.466486:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x34:0x0](ffff9181c83fec08) 00000080:00200000:1.0:1787826191.466494:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x35:0x0](ffff9181c83e5b88) 00000080:00200000:1.0:1787826191.466505:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x35:0x0](ffff9181c83e3a88) 00000080:00200000:1.0:1787826191.466513:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x36:0x0](ffff9181c83eec08) 00000080:00200000:1.0:1787826191.466521:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x36:0x0](ffff9181c83ed348) 00000080:00200000:1.0:1787826191.466528:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x37:0x0](ffff9181c83e3248) 00000080:00200000:1.0:1787826191.466536:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x37:0x0](ffff9181c83e2a08) 00000080:00200000:1.0:1787826191.466544:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x38:0x0](ffff9181c83ef448) 00000080:00200000:1.0:1787826191.466552:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x38:0x0](ffff9181c83ee3c8) 00000080:00200000:1.0:1787826191.466559:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x39:0x0](ffff9181c83f1988) 00000080:00200000:1.0:1787826191.466566:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x39:0x0](ffff9181c83f5b88) 00000080:00200000:1.0:1787826191.466573:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x3a:0x0](ffff9181c83fe3c8) 00000080:00200000:1.0:1787826191.466582:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x3a:0x0](ffff9181c83ff448) 00000080:00200000:1.0:1787826191.466589:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x3b:0x0](ffff9181c83e9148) 00000080:00200000:1.0:1787826191.466597:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x3b:0x0](ffff9181c83ea1c8) 00000080:00200000:1.0:1787826191.466603:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x3c:0x0](ffff9181d0805348) 00000080:00200000:1.0:1787826191.466610:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x3c:0x0](ffff9181d0806c08) 00000080:00200000:1.0:1787826191.466628:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x3d:0x0](ffff9181d08000c8) 00000080:00200000:1.0:1787826191.466635:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x3d:0x0](ffff9181d0801148) 00000080:00200000:1.0:1787826191.466642:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x3e:0x0](ffff9181c83e4b08) 00000080:00200000:1.0:1787826191.466649:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x3e:0x0](ffff9181c83e1988) 00000080:00200000:1.0:1787826191.466657:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x3f:0x0](ffff9181c83f8908) 00000080:00200000:1.0:1787826191.466665:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x3f:0x0](ffff9181c83fa1c8) 00000080:00200000:1.0:1787826191.466672:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x40:0x0](ffff9181c83e21c8) 00000080:00200000:1.0:1787826191.466679:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x40:0x0](ffff9181c83e7448) 00000080:00200000:1.0:1787826191.466686:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x41:0x0](ffff9181c83fc2c8) 00000080:00200000:1.0:1787826191.467001:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x41:0x0](ffff9181c83f80c8) 00000080:00200000:1.0:1787826191.467014:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200026560:0x42:0x0](ffff9181d080ba88) 00000080:00200000:1.0:1787826191.467021:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f58:0x42:0x0](ffff9181d080b248) 00000080:00200000:1.0:1787826191.467028:0:159403:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20001a9e3:0x1bcf:0x0](ffff9181d080aa08) 00000080:00200000:1.0:1787826191.467038:0:159403:0:(llite_lib.c:1688:ll_put_super()) VFS Op: cfg_instance lustre-client-ffff9181c4d2b000 (sb ffff9181c4d2b000) 00010000:00010000:1.0:1787826191.467084:0:159403:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f108 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00ced17c expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826191.467096:0:159403:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f108 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x878a5eea00ced17c expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826191.467108:0:159403:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f108 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000400000000 nid: local remote: 0x878a5eea00ced17c expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826191.467116:0:159403:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f108 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000400000000 nid: local remote: 0x878a5eea00ced17c expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826191.467184:0:159403:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1787826191.467191:0:159403:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0513c00/0xbef42c0221d6f101 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00ced175 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826191.467198:0:159403:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0513c00/0xbef42c0221d6f101 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x878a5eea00ced175 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826191.467204:0:159403:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0513c00/0xbef42c0221d6f101 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000400000000 nid: local remote: 0x878a5eea00ced175 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826191.467209:0:159403:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e0513c00/0xbef42c0221d6f101 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000400000000 nid: local remote: 0x878a5eea00ced175 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826191.471604:0:33350:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1787826191.471630:0:33350:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f108 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000400000000 nid: local remote: 0x878a5eea00ced17c expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826191.471643:0:33350:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b5800 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:0.0:1787826191.471648:0:33350:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f108 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000400000000 nid: local remote: 0x878a5eea00ced17c expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826191.471658:0:33350:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f108 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1008400000000 nid: local remote: 0x878a5eea00ced17c expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1787826191.471665:0:33350:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f108 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1009400000000 nid: local remote: 0x878a5eea00ced17c expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787826191.471671:0:33350:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 10000000:01000000:0.0:1787826191.471673:0:33350:0:(mgc_request.c:944:mgc_blocking_ast()) log lustre-client: stopping, won't requeue 00010000:00010000:0.0:1787826191.471730:0:33350:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f108 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x5809400000020 nid: local remote: 0x878a5eea00ced17c expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826191.471736:0:33350:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826191.472102:0:33350:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f108 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x5809400000020 nid: local remote: 0x878a5eea00ced17c expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826191.472111:0:33350:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f108 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x5809400000020 nid: local remote: 0x878a5eea00ced17c expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826191.472129:0:33350:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1787826191.472133:0:33350:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e0513c00/0xbef42c0221d6f101 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000400000000 nid: local remote: 0x878a5eea00ced175 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826191.472138:0:33350:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0513c00 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:0.0:1787826191.472141:0:33350:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e0513c00/0xbef42c0221d6f101 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000400000000 nid: local remote: 0x878a5eea00ced175 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826191.472149:0:33350:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e0513c00/0xbef42c0221d6f101 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1008400000000 nid: local remote: 0x878a5eea00ced175 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1787826191.472164:0:33350:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e0513c00/0xbef42c0221d6f101 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1009400000000 nid: local remote: 0x878a5eea00ced175 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787826191.472170:0:33350:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 10000000:01000000:0.0:1787826191.472173:0:33350:0:(mgc_request.c:944:mgc_blocking_ast()) log lustre-cliir: stopping, won't requeue 00010000:00010000:0.0:1787826191.472187:0:33350:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.147@tcp lock: ffff9181e0513c00/0xbef42c0221d6f101 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5809400000020 nid: local remote: 0x878a5eea00ced175 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826191.472195:0:33350:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826191.472201:0:33350:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e0513c00/0xbef42c0221d6f101 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5809400000020 nid: local remote: 0x878a5eea00ced175 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826191.472208:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826191.472210:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826191.472212:0:33350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826191.472233:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7528148b-9bee-4d31-b597-8aae3b0b4982:2389:1874663042981504:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1787826191.472332:0:2389:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1787826191.472339:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7528148b-9bee-4d31-b597-8aae3b0b4982:2389:1874663042981632:192.168.203.147@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826191.478295:0:159403:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to regular blp list, 0 blwis in pool 00010000:00010000:1.0:1787826191.478330:0:159403:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e0513c00/0xbef42c0221d6f101 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5809400000020 nid: local remote: 0x878a5eea00ced175 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826191.478355:0:159403:0:(mgc_request.c:130:config_log_put()) dropping config log lustre-cliir 00010000:00010000:1.0:1787826191.478370:0:159403:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512600/0xbef42c0221d6f10f lrc: 3/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00ced18a expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826191.478377:0:159403:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512600/0xbef42c0221d6f10f lrc: 3/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x878a5eea00ced18a expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826191.478384:0:159403:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512600/0xbef42c0221d6f10f lrc: 3/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000400000000 nid: local remote: 0x878a5eea00ced18a expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826191.478390:0:159403:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e0512600/0xbef42c0221d6f10f lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000400000000 nid: local remote: 0x878a5eea00ced18a expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826191.478411:0:159403:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 10000000:01000000:1.0:1787826191.478414:0:159403:0:(mgc_request.c:130:config_log_put()) dropping config log lustre-client 10000000:01000000:1.0:1787826191.478417:0:159403:0:(mgc_request.c:547:config_log_end()) end config log lustre-client (0) 00010000:00010000:2.0:1787826191.479451:0:15161:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826191.479461:0:15161:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e0512600/0xbef42c0221d6f10f lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000400000000 nid: local remote: 0x878a5eea00ced18a expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826191.479486:0:15161:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0512600 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:2.0:1787826191.479490:0:15161:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512600/0xbef42c0221d6f10f lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000400000000 nid: local remote: 0x878a5eea00ced18a expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826191.479499:0:15161:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e0512600/0xbef42c0221d6f10f lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1008400000000 nid: local remote: 0x878a5eea00ced18a expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1787826191.479523:0:15161:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512600/0xbef42c0221d6f10f lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1009400000000 nid: local remote: 0x878a5eea00ced18a expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787826191.479529:0:15161:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 10000000:01000000:2.0:1787826191.479532:0:15161:0:(mgc_request.c:944:mgc_blocking_ast()) log params: stopping, won't requeue 10000000:01000000:2.0:1787826191.479534:0:15161:0:(mgc_request.c:130:config_log_put()) dropping config log params 00010000:00010000:2.0:1787826191.479573:0:15161:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.147@tcp lock: ffff9181e0512600/0xbef42c0221d6f10f lrc: 2/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5809400000020 nid: local remote: 0x878a5eea00ced18a expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826191.479580:0:15161:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826191.479618:0:15161:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e0512600/0xbef42c0221d6f10f lrc: 1/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5809400000020 nid: local remote: 0x878a5eea00ced18a expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826191.479628:0:15161:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e0512600/0xbef42c0221d6f10f lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5809400000020 nid: local remote: 0x878a5eea00ced18a expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826191.479659:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826191.479661:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826191.479663:0:15161:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826191.479683:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7528148b-9bee-4d31-b597-8aae3b0b4982:2392:1874663042981760:192.168.203.147@tcp:103:ldlm_bl.0 00020000:01000000:1.0:1787826191.480577:0:159403:0:(lov_obd.c:601:lov_del_target()) uuid: lustre-OST0000_UUID idx: 0 gen: 1 exp: ffff9181c5a68800 active: 1 00020000:01000000:1.0:1787826191.480589:0:159403:0:(lov_obd.c:601:lov_del_target()) uuid: lustre-OST0001_UUID idx: 1 gen: 1 exp: ffff9181c5a6d800 active: 1 00020000:01000000:1.0:1787826191.480592:0:159403:0:(lov_obd.c:64:lov_tgts_putref()) destroying 2 lov targets 00020000:01000000:1.0:1787826191.480597:0:159403:0:(lov_obd.c:623:__lov_del_obd()) Removing tgt lustre-OST0001_UUID : lustre-OST0001-osc-ffff9181c4d2b000 00020000:01000000:1.0:1787826191.480599:0:159403:0:(lov_obd.c:248:lov_disconnect_obd()) lustre-clilov-ffff9181c4d2b000: disconnecting target lustre-OST0001-osc-ffff9181c4d2b000 00000100:00080000:1.0:1787826191.480626:0:159403:0:(pinger.c:398:ptlrpc_pinger_del_import()) removing pingable import a9ce6693-0634-46c5-bf21-5ca7d026d44e->lustre-OST0001_UUID 00000100:00080000:1.0:1787826191.483098:0:159403:0:(import.c:65:import_set_state_nolock()) ffff9181e0e59800 lustre-OST0001_UUID: changing import state from FULL to CONNECTING 00000100:00100000:0.0:1787826191.483258:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042981888:192.168.203.147@tcp:9:umount.0 00010000:00010000:3.0:1787826191.491452:0:33329:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826191.492223:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7528148b-9bee-4d31-b597-8aae3b0b4982:2389:1874663042981632:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1787826191.492236:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7528148b-9bee-4d31-b597-8aae3b0b4982:2389:1874663042981504:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00080000:0.0:1787826191.515301:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181e0e59800 lustre-OST0001_UUID: changing import state from CONNECTING to CLOSED 00000100:00100000:0.0:1787826191.515333:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042981888:192.168.203.147@tcp:9:umount.0 00000100:00080000:1.0:1787826191.516010:0:159403:0:(import.c:138:ptlrpc_deactivate_import_nolock()) setting import lustre-OST0001_UUID INVALID 00000100:00100000:1.0:1787826191.516015:0:159403:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-ffff9181c4d2b000: committing for last_committed 4294975530 gen 35 00000100:00100000:1.0:1787826191.516021:0:159403:0:(import.c:323:ptlrpc_invalidate_import()) Sleeping 20s for inflight to error out 00000020:00000080:1.0:1787826191.516156:0:159403:0:(genops.c:1468:class_disconnect()) disconnect: cookie 0xbef42c0221d10916 00000020:00000080:1.0:1787826191.516173:0:159403:0:(genops.c:885:class_export_put()) final put ffff9181c5a6d800/LOV_OSC_UUID 00000020:01000000:1.0:1787826191.516223:0:159403:0:(obd_config.c:2477:class_manual_cleanup()) Manual cleanup of lustre-OST0001-osc-ffff9181c4d2b000 (flags='') 00000020:00000080:1.0:1787826191.516228:0:159403:0:(obd_config.c:1472:class_process_config()) processing cmd: cf004 00000020:00000080:1.0:1787826191.516237:0:159403:0:(obd_config.c:956:class_cleanup()) lustre-OST0001-osc-ffff9181c4d2b000: forcing exports to disconnect: 0/3 00000020:00080000:1.0:1787826191.516242:0:159403:0:(genops.c:1558:class_disconnect_exports()) OBD device 6 (ffff9181c87b4d28) has no exports 00000100:00100000:2.0:1787826191.517458:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7528148b-9bee-4d31-b597-8aae3b0b4982:2392:1874663042981760:192.168.203.147@tcp:103:ldlm_bl.0 00000020:00000080:3.0:1787826191.518230:0:152015:0:(genops.c:844:class_export_destroy()) destroying export ffff9181c5a6d800/LOV_OSC_UUID for lustre-OST0001-osc-ffff9181c4d2b000 00000008:01000000:1.0:1787826191.518304:0:159403:0:(obd_class.h:670:obd_cleanup_client_import()) lustre-OST0001-osc-ffff9181c4d2b000: client import never connected 00000100:00080000:1.0:1787826191.518309:0:159403:0:(import.c:138:ptlrpc_deactivate_import_nolock()) setting import lustre-OST0001_UUID INVALID 00000100:00100000:1.0:1787826191.518313:0:159403:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-ffff9181c4d2b000: committing for last_committed 4294975530 gen 36 00000100:00100000:1.0:1787826191.518319:0:159403:0:(import.c:323:ptlrpc_invalidate_import()) Sleeping 20s for inflight to error out 00000020:00000080:1.0:1787826191.518507:0:159403:0:(obd_config.c:1472:class_process_config()) processing cmd: cf002 00000020:00000080:1.0:1787826191.518518:0:159403:0:(obd_config.c:883:class_detach()) detach on obd lustre-OST0001-osc-ffff9181c4d2b000 (uuid a9ce6693-0634-46c5-bf21-5ca7d026d44e) 00020000:01000000:1.0:1787826191.518522:0:159403:0:(lov_obd.c:623:__lov_del_obd()) Removing tgt lustre-OST0000_UUID : lustre-OST0000-osc-ffff9181c4d2b000 00020000:01000000:1.0:1787826191.518524:0:159403:0:(lov_obd.c:248:lov_disconnect_obd()) lustre-clilov-ffff9181c4d2b000: disconnecting target lustre-OST0000-osc-ffff9181c4d2b000 00000100:00080000:1.0:1787826191.518539:0:159403:0:(pinger.c:398:ptlrpc_pinger_del_import()) removing pingable import a9ce6693-0634-46c5-bf21-5ca7d026d44e->lustre-OST0000_UUID 00000100:00080000:1.0:1787826191.518634:0:159403:0:(import.c:65:import_set_state_nolock()) ffff9181c2af3000 lustre-OST0000_UUID: changing import state from FULL to CONNECTING 00000100:00100000:0.0:1787826191.519062:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042982016:192.168.203.147@tcp:9:umount.0 00000020:00000080:3.0:1787826191.519244:0:152015:0:(genops.c:1183:obd_zombie_import_free()) destroying import ffff9181e0e59800 for lustre-OST0001-osc-ffff9181c4d2b000 00000020:00000080:3.0:1787826191.519253:0:152015:0:(genops.c:885:class_export_put()) final put ffff9181e0e58800/a9ce6693-0634-46c5-bf21-5ca7d026d44e 00000020:00000080:3.0:1787826191.519256:0:152015:0:(genops.c:844:class_export_destroy()) destroying export ffff9181e0e58800/a9ce6693-0634-46c5-bf21-5ca7d026d44e for lustre-OST0001-osc-ffff9181c4d2b000 00000020:01000000:3.0:1787826191.519262:0:152015:0:(genops.c:441:class_free_dev()) finishing cleanup of obd lustre-OST0001-osc-ffff9181c4d2b000 (a9ce6693-0634-46c5-bf21-5ca7d026d44e) 00000100:00080000:1.0:1787826191.523464:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c2af3000 lustre-OST0000_UUID: changing import state from CONNECTING to CLOSED 00000100:00100000:1.0:1787826191.523506:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042982016:192.168.203.147@tcp:9:umount.0 00000100:00080000:2.0:1787826191.523581:0:159403:0:(import.c:138:ptlrpc_deactivate_import_nolock()) setting import lustre-OST0000_UUID INVALID 00000100:00100000:2.0:1787826191.523586:0:159403:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff9181c4d2b000: committing for last_committed 34359738465 gen 40 00000100:00100000:2.0:1787826191.523591:0:159403:0:(import.c:323:ptlrpc_invalidate_import()) Sleeping 20s for inflight to error out 00000020:00000080:2.0:1787826191.523726:0:159403:0:(genops.c:1468:class_disconnect()) disconnect: cookie 0xbef42c0221d1090f 00000020:00000080:2.0:1787826191.523786:0:159403:0:(genops.c:885:class_export_put()) final put ffff9181c5a68800/LOV_OSC_UUID 00000020:01000000:2.0:1787826191.523837:0:159403:0:(obd_config.c:2477:class_manual_cleanup()) Manual cleanup of lustre-OST0000-osc-ffff9181c4d2b000 (flags='') 00000020:00000080:2.0:1787826191.523855:0:159403:0:(obd_config.c:1472:class_process_config()) processing cmd: cf004 00000020:00000080:2.0:1787826191.523865:0:159403:0:(obd_config.c:956:class_cleanup()) lustre-OST0000-osc-ffff9181c4d2b000: forcing exports to disconnect: 0/3 00000020:00080000:2.0:1787826191.523870:0:159403:0:(genops.c:1558:class_disconnect_exports()) OBD device 5 (ffff9181c87b3370) has no exports 00000020:00000080:3.0:1787826191.524271:0:152015:0:(genops.c:844:class_export_destroy()) destroying export ffff9181c5a68800/LOV_OSC_UUID for lustre-OST0000-osc-ffff9181c4d2b000 00000008:01000000:2.0:1787826191.524796:0:159403:0:(obd_class.h:670:obd_cleanup_client_import()) lustre-OST0000-osc-ffff9181c4d2b000: client import never connected 00000100:00080000:2.0:1787826191.524801:0:159403:0:(import.c:138:ptlrpc_deactivate_import_nolock()) setting import lustre-OST0000_UUID INVALID 00000100:00100000:2.0:1787826191.524803:0:159403:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff9181c4d2b000: committing for last_committed 34359738465 gen 41 00000100:00100000:2.0:1787826191.524809:0:159403:0:(import.c:323:ptlrpc_invalidate_import()) Sleeping 20s for inflight to error out 00000020:00000080:2.0:1787826191.525012:0:159403:0:(obd_config.c:1472:class_process_config()) processing cmd: cf002 00000020:00000080:2.0:1787826191.525020:0:159403:0:(obd_config.c:883:class_detach()) detach on obd lustre-OST0000-osc-ffff9181c4d2b000 (uuid a9ce6693-0634-46c5-bf21-5ca7d026d44e) 00000020:00000080:2.0:1787826191.525024:0:159403:0:(genops.c:1468:class_disconnect()) disconnect: cookie 0xbef42c0221d10908 00000020:00000080:2.0:1787826191.525030:0:159403:0:(genops.c:885:class_export_put()) final put ffff9181c5a6a000/a9ce6693-0634-46c5-bf21-5ca7d026d44e 00000100:00080000:2.0:1787826191.525098:0:159403:0:(pinger.c:398:ptlrpc_pinger_del_import()) removing pingable import a9ce6693-0634-46c5-bf21-5ca7d026d44e->lustre-MDT0000_UUID 00010000:00010000:2.0:1787826191.525154:0:159403:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513000/0xbef42c0221d64bee lrc: 2/0,0 mode: CR/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ced1f3 expref: -99 pid: 152210 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787826191.525175:0:159403:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6800/0xbef42c0221d64b3f lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ced1fa expref: -99 pid: 152202 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787826191.525241:0:159403:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513000/0xbef42c0221d64bee lrc: 1/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ced1f3 expref: -99 pid: 152210 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787826191.525249:0:159403:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6800/0xbef42c0221d64b3f lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ced1fa expref: -99 pid: 152202 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787826191.525255:0:159403:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 2 locks packed 00000100:00100000:2.0:1787826191.525270:0:159403:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c4a736c0 going to sleep for 0 seconds 00000100:00100000:2.0:1787826191.525279:0:159403:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f3480 pname:cluuid:pid:xid:nid:opc:job umount:a9ce6693-0634-46c5-bf21-5ca7d026d44e:159403:1874663042982144:192.168.203.147@tcp:103:umount.0 00010000:00010000:2.0:1787826191.525289:0:159403:0:(ldlm_request.c:1625:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway 00010000:00010000:2.0:1787826191.525295:0:159403:0:(ldlm_request.c:2491:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 00010000:00010000:2.0:1787826191.525299:0:159403:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513000/0xbef42c0221d64bee lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ced1f3 expref: -99 pid: 152210 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787826191.525308:0:159403:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f6800/0xbef42c0221d64b3f lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x17e0:0x0].0x0 bits 0x11/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ced1fa expref: -99 pid: 152202 timeout: 0 lvb_type: 0 lru_score: 254 lru_type: 1 00010000:00010000:2.0:1787826191.525375:0:159403:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6f11d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ced208 expref: -99 pid: 158321 timeout: 0 lvb_type: 0 lru_score: 11 lru_type: 1 00010000:00010000:2.0:1787826191.525396:0:159403:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6f11d lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ced208 expref: -99 pid: 158321 timeout: 0 lvb_type: 0 lru_score: 11 lru_type: 1 00010000:00010000:2.0:1787826191.525401:0:159403:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:2.0:1787826191.525406:0:159403:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c4a736c0 going to sleep for 0 seconds 00000100:00100000:2.0:1787826191.525410:0:159403:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f3480 pname:cluuid:pid:xid:nid:opc:job umount:a9ce6693-0634-46c5-bf21-5ca7d026d44e:159403:1874663042982272:192.168.203.147@tcp:103:umount.0 00010000:00010000:2.0:1787826191.525414:0:159403:0:(ldlm_request.c:1625:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway 00010000:00010000:2.0:1787826191.525418:0:159403:0:(ldlm_request.c:2491:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 00010000:00010000:2.0:1787826191.525420:0:159403:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e01b4800/0xbef42c0221d6f11d lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ced208 expref: -99 pid: 158321 timeout: 0 lvb_type: 0 lru_score: 11 lru_type: 1 00010000:00010000:2.0:1787826191.525604:0:159403:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7200/0xbef42c0221d6f12b lrc: 2/0,0 mode: PR/PR res: [0x20001a9e3:0x1bd2:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x878a5eea00ced3e4 expref: -99 pid: 157888 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826191.525615:0:159403:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6f124 lrc: 2/0,0 mode: CR/CR res: [0x20001a9e3:0x1bd2:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x878a5eea00ced3d6 expref: -99 pid: 157888 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787826191.525630:0:159403:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7200/0xbef42c0221d6f12b lrc: 1/0,0 mode: --/PR res: [0x20001a9e3:0x1bd2:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00ced3e4 expref: -99 pid: 157888 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826191.525636:0:159403:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6f124 lrc: 1/0,0 mode: --/CR res: [0x20001a9e3:0x1bd2:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00ced3d6 expref: -99 pid: 157888 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787826191.525641:0:159403:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 2 locks packed 00000100:00100000:2.0:1787826191.525647:0:159403:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c4a736c0 going to sleep for 0 seconds 00000100:00100000:2.0:1787826191.525651:0:159403:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f3480 pname:cluuid:pid:xid:nid:opc:job umount:a9ce6693-0634-46c5-bf21-5ca7d026d44e:159403:1874663042982400:192.168.203.147@tcp:103:umount.0 00010000:00010000:2.0:1787826191.525656:0:159403:0:(ldlm_request.c:1625:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway 00010000:00010000:2.0:1787826191.525660:0:159403:0:(ldlm_request.c:2491:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 00010000:00010000:2.0:1787826191.525663:0:159403:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e06f7200/0xbef42c0221d6f12b lrc: 0/0,0 mode: --/PR res: [0x20001a9e3:0x1bd2:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00ced3e4 expref: -99 pid: 157888 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826191.525674:0:159403:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c4d2b000 lock: ffff9181e0513200/0xbef42c0221d6f124 lrc: 0/0,0 mode: --/CR res: [0x20001a9e3:0x1bd2:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00ced3d6 expref: -99 pid: 157888 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000100:00080000:2.0:1787826191.526502:0:159403:0:(import.c:65:import_set_state_nolock()) ffff9181c39f6800 lustre-MDT0000_UUID: changing import state from FULL to CONNECTING 00000020:00000080:3.0:1787826191.526787:0:152015:0:(genops.c:1183:obd_zombie_import_free()) destroying import ffff9181c2af3000 for lustre-OST0000-osc-ffff9181c4d2b000 00000020:00000080:3.0:1787826191.526796:0:152015:0:(genops.c:885:class_export_put()) final put ffff9181c39f5800/a9ce6693-0634-46c5-bf21-5ca7d026d44e 00000020:00000080:3.0:1787826191.526799:0:152015:0:(genops.c:844:class_export_destroy()) destroying export ffff9181c39f5800/a9ce6693-0634-46c5-bf21-5ca7d026d44e for lustre-OST0000-osc-ffff9181c4d2b000 00000020:01000000:3.0:1787826191.526805:0:152015:0:(genops.c:441:class_free_dev()) finishing cleanup of obd lustre-OST0000-osc-ffff9181c4d2b000 (a9ce6693-0634-46c5-bf21-5ca7d026d44e) 00000020:00000080:3.0:1787826191.527170:0:152015:0:(genops.c:844:class_export_destroy()) destroying export ffff9181c5a6a000/a9ce6693-0634-46c5-bf21-5ca7d026d44e for lustre-clilov-ffff9181c4d2b000 00000100:00100000:1.0:1787826191.527352:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042982528:192.168.203.147@tcp:39:umount.0 00000100:00080000:2.0:1787826191.535043:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c39f6800 lustre-MDT0000_UUID: changing import state from CONNECTING to CLOSED 00000100:00100000:2.0:1787826191.535075:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042982528:192.168.203.147@tcp:39:umount.0 00000100:00080000:1.0:1787826191.535137:0:159403:0:(import.c:138:ptlrpc_deactivate_import_nolock()) setting import lustre-MDT0000_UUID INVALID 00000100:00100000:1.0:1787826191.535141:0:159403:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9181c4d2b000: committing for last_committed 343597383682 gen 12 00000100:00100000:1.0:1787826191.535150:0:159403:0:(client.c:3163:ptlrpc_free_committed()) @@@ free request with old gen req@ffff9181c71f3480 x1874663042982528/t343597383684(343597383684) o39->lustre-MDT0000-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 224/224 e 0 to 0 dl 1787826201 ref 1 fl Complete:RNQU/204/0 rc 0/0 job:'umount.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787826191.535171:0:159403:0:(import.c:323:ptlrpc_invalidate_import()) Sleeping 20s for inflight to error out 00000020:00000080:1.0:1787826191.541533:0:159403:0:(genops.c:1468:class_disconnect()) disconnect: cookie 0xbef42c0221d108fa 00000020:00000080:1.0:1787826191.541549:0:159403:0:(genops.c:885:class_export_put()) final put ffff9181c58ce000/a9ce6693-0634-46c5-bf21-5ca7d026d44e 00000100:00080000:0.0:1787826191.542082:0:159403:0:(pinger.c:398:ptlrpc_pinger_del_import()) removing pingable import a9ce6693-0634-46c5-bf21-5ca7d026d44e->lustre-MDT0001_UUID 00000020:00000080:2.0:1787826191.542996:0:111835:0:(genops.c:844:class_export_destroy()) destroying export ffff9181c58ce000/a9ce6693-0634-46c5-bf21-5ca7d026d44e for lustre-MDT0000-mdc-ffff9181c4d2b000 00000100:00080000:0.0:1787826191.545292:0:159403:0:(import.c:65:import_set_state_nolock()) ffff9181c39f4800 lustre-MDT0001_UUID: changing import state from FULL to CONNECTING 00000100:00100000:2.0:1787826191.546048:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042982656:192.168.203.147@tcp:39:umount.0 00000100:00080000:2.0:1787826191.571422:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c39f4800 lustre-MDT0001_UUID: changing import state from CONNECTING to CLOSED 00000100:00100000:2.0:1787826191.571446:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:a9ce6693-0634-46c5-bf21-5ca7d026d44e:2388:1874663042982656:192.168.203.147@tcp:39:umount.0 00000100:00080000:0.0:1787826191.571607:0:159403:0:(import.c:138:ptlrpc_deactivate_import_nolock()) setting import lustre-MDT0001_UUID INVALID 00000100:00100000:0.0:1787826191.571612:0:159403:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff9181c4d2b000: committing for last_committed 12884907729 gen 2 00000100:00100000:0.0:1787826191.571621:0:159403:0:(client.c:3163:ptlrpc_free_committed()) @@@ free request with old gen req@ffff9181c55a4000 x1874663042982656/t17179869982(17179869982) o39->lustre-MDT0001-mdc-ffff9181c4d2b000@192.168.203.147@tcp:12/10 lens 224/224 e 0 to 0 dl 1787826201 ref 1 fl Complete:RNQU/204/0 rc 0/0 job:'umount.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826191.571643:0:159403:0:(import.c:323:ptlrpc_invalidate_import()) Sleeping 20s for inflight to error out 00000020:00000080:0.0:1787826191.574581:0:159403:0:(genops.c:1468:class_disconnect()) disconnect: cookie 0xbef42c0221d10901 00000020:00000080:0.0:1787826191.574598:0:159403:0:(genops.c:885:class_export_put()) final put ffff9181c58cd000/a9ce6693-0634-46c5-bf21-5ca7d026d44e 00000020:00000080:0.0:1787826191.574771:0:159403:0:(genops.c:1468:class_disconnect()) disconnect: cookie 0xbef42c0221d108f3 00000020:00000080:0.0:1787826191.574778:0:159403:0:(genops.c:885:class_export_put()) final put ffff9181c58cb000/a9ce6693-0634-46c5-bf21-5ca7d026d44e 00000020:00000080:2.0:1787826191.575242:0:111835:0:(genops.c:844:class_export_destroy()) destroying export ffff9181c58cd000/a9ce6693-0634-46c5-bf21-5ca7d026d44e for lustre-MDT0001-mdc-ffff9181c4d2b000 00000020:01000000:0.0:1787826191.575394:0:159403:0:(obd_config.c:2477:class_manual_cleanup()) Manual cleanup of lustre-clilov-ffff9181c4d2b000 (flags='') 00000020:00000080:0.0:1787826191.575405:0:159403:0:(obd_config.c:1472:class_process_config()) processing cmd: cf004 00000020:00000080:0.0:1787826191.575412:0:159403:0:(obd_config.c:956:class_cleanup()) lustre-clilov-ffff9181c4d2b000: forcing exports to disconnect: 0/1 00000020:00080000:0.0:1787826191.575417:0:159403:0:(genops.c:1558:class_disconnect_exports()) OBD device 1 (ffff9181c77c99b8) has no exports 00000020:00000080:0.0:1787826191.575599:0:159403:0:(obd_config.c:1472:class_process_config()) processing cmd: cf002 00000020:00000080:0.0:1787826191.575614:0:159403:0:(obd_config.c:883:class_detach()) detach on obd lustre-clilov-ffff9181c4d2b000 (uuid a9ce6693-0634-46c5-bf21-5ca7d026d44e) 00000020:00000080:0.0:1787826191.575617:0:159403:0:(genops.c:885:class_export_put()) final put ffff9181c5976800/a9ce6693-0634-46c5-bf21-5ca7d026d44e 00000020:00000080:0.0:1787826191.575620:0:159403:0:(genops.c:844:class_export_destroy()) destroying export ffff9181c5976800/a9ce6693-0634-46c5-bf21-5ca7d026d44e for lustre-clilov-ffff9181c4d2b000 00000020:01000000:0.0:1787826191.575625:0:159403:0:(genops.c:441:class_free_dev()) finishing cleanup of obd lustre-clilov-ffff9181c4d2b000 (a9ce6693-0634-46c5-bf21-5ca7d026d44e) 00000020:01000000:0.0:1787826191.576017:0:159403:0:(obd_config.c:2477:class_manual_cleanup()) Manual cleanup of lustre-clilmv-ffff9181c4d2b000 (flags='') 00000020:00000080:0.0:1787826191.576023:0:159403:0:(obd_config.c:1472:class_process_config()) processing cmd: cf004 00000020:00000080:0.0:1787826191.576026:0:159403:0:(obd_config.c:956:class_cleanup()) lustre-clilmv-ffff9181c4d2b000: forcing exports to disconnect: 0/2 00000020:00080000:0.0:1787826191.576029:0:159403:0:(genops.c:1558:class_disconnect_exports()) OBD device 2 (ffff9181c77cb370) has no exports 00000020:00000080:0.0:1787826191.576064:0:159403:0:(obd_config.c:1472:class_process_config()) processing cmd: cf002 00000020:00000080:0.0:1787826191.576068:0:159403:0:(obd_config.c:883:class_detach()) detach on obd lustre-clilmv-ffff9181c4d2b000 (uuid a9ce6693-0634-46c5-bf21-5ca7d026d44e) 00000020:01000000:0.0:1787826191.576071:0:159403:0:(obd_config.c:2477:class_manual_cleanup()) Manual cleanup of lustre-MDT0000-mdc-ffff9181c4d2b000 (flags='') 00000020:00000080:0.0:1787826191.576073:0:159403:0:(obd_config.c:1472:class_process_config()) processing cmd: cf004 00000020:00000080:0.0:1787826191.576076:0:159403:0:(obd_config.c:956:class_cleanup()) lustre-MDT0000-mdc-ffff9181c4d2b000: forcing exports to disconnect: 0/2 00000020:00080000:0.0:1787826191.576078:0:159403:0:(genops.c:1558:class_disconnect_exports()) OBD device 3 (ffff9181c87b0000) has no exports 00000020:00000080:2.0:1787826191.576425:0:111835:0:(genops.c:844:class_export_destroy()) destroying export ffff9181c58cb000/a9ce6693-0634-46c5-bf21-5ca7d026d44e for lustre-clilmv-ffff9181c4d2b000 00000020:00000080:2.0:1787826191.576431:0:111835:0:(genops.c:885:class_export_put()) final put ffff9181c5972000/a9ce6693-0634-46c5-bf21-5ca7d026d44e 00000020:00000080:2.0:1787826191.576434:0:111835:0:(genops.c:844:class_export_destroy()) destroying export ffff9181c5972000/a9ce6693-0634-46c5-bf21-5ca7d026d44e for lustre-clilmv-ffff9181c4d2b000 00000020:01000000:2.0:1787826191.576438:0:111835:0:(genops.c:441:class_free_dev()) finishing cleanup of obd lustre-clilmv-ffff9181c4d2b000 (a9ce6693-0634-46c5-bf21-5ca7d026d44e) 00000008:01000000:0.0:1787826191.577071:0:159403:0:(obd_class.h:670:obd_cleanup_client_import()) lustre-MDT0000-mdc-ffff9181c4d2b000: client import never connected 00000100:00080000:0.0:1787826191.577077:0:159403:0:(import.c:138:ptlrpc_deactivate_import_nolock()) setting import lustre-MDT0000_UUID INVALID 00000100:00100000:0.0:1787826191.577081:0:159403:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9181c4d2b000: committing for last_committed 343597383682 gen 13 00000100:00100000:0.0:1787826191.577089:0:159403:0:(import.c:323:ptlrpc_invalidate_import()) Sleeping 20s for inflight to error out 00000020:00000080:0.0:1787826191.586994:0:159403:0:(obd_config.c:1472:class_process_config()) processing cmd: cf002 00000020:00000080:0.0:1787826191.587008:0:159403:0:(obd_config.c:883:class_detach()) detach on obd lustre-MDT0000-mdc-ffff9181c4d2b000 (uuid a9ce6693-0634-46c5-bf21-5ca7d026d44e) 00000020:01000000:0.0:1787826191.587014:0:159403:0:(obd_config.c:2477:class_manual_cleanup()) Manual cleanup of lustre-MDT0001-mdc-ffff9181c4d2b000 (flags='') 00000020:00000080:0.0:1787826191.587017:0:159403:0:(obd_config.c:1472:class_process_config()) processing cmd: cf004 00000020:00000080:0.0:1787826191.587022:0:159403:0:(obd_config.c:956:class_cleanup()) lustre-MDT0001-mdc-ffff9181c4d2b000: forcing exports to disconnect: 0/2 00000020:00080000:0.0:1787826191.587027:0:159403:0:(genops.c:1558:class_disconnect_exports()) OBD device 4 (ffff9181c87b19b8) has no exports 00000020:00000080:2.0:1787826191.587430:0:111835:0:(genops.c:1183:obd_zombie_import_free()) destroying import ffff9181c39f6800 for lustre-MDT0000-mdc-ffff9181c4d2b000 00000020:00000080:2.0:1787826191.587439:0:111835:0:(genops.c:885:class_export_put()) final put ffff9181c39f3800/a9ce6693-0634-46c5-bf21-5ca7d026d44e 00000020:00000080:2.0:1787826191.587442:0:111835:0:(genops.c:844:class_export_destroy()) destroying export ffff9181c39f3800/a9ce6693-0634-46c5-bf21-5ca7d026d44e for lustre-MDT0000-mdc-ffff9181c4d2b000 00000020:01000000:2.0:1787826191.587447:0:111835:0:(genops.c:441:class_free_dev()) finishing cleanup of obd lustre-MDT0000-mdc-ffff9181c4d2b000 (a9ce6693-0634-46c5-bf21-5ca7d026d44e) 00000008:01000000:3.0:1787826191.588179:0:159403:0:(obd_class.h:670:obd_cleanup_client_import()) lustre-MDT0001-mdc-ffff9181c4d2b000: client import never connected 00000100:00080000:3.0:1787826191.588184:0:159403:0:(import.c:138:ptlrpc_deactivate_import_nolock()) setting import lustre-MDT0001_UUID INVALID 00000100:00100000:3.0:1787826191.588189:0:159403:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff9181c4d2b000: committing for last_committed 12884907729 gen 3 00000100:00100000:3.0:1787826191.588196:0:159403:0:(import.c:323:ptlrpc_invalidate_import()) Sleeping 20s for inflight to error out 00000020:00000080:3.0:1787826191.603769:0:159403:0:(obd_config.c:1472:class_process_config()) processing cmd: cf002 00000020:00000080:3.0:1787826191.603785:0:159403:0:(obd_config.c:883:class_detach()) detach on obd lustre-MDT0001-mdc-ffff9181c4d2b000 (uuid a9ce6693-0634-46c5-bf21-5ca7d026d44e) 00000080:02000400:3.0:1787826191.603789:0:159403:0:(llite_lib.c:1739:ll_put_super()) Unmounted lustre-client 00000020:01000000:3.0:1787826191.606375:0:159403:0:(obd_config.c:1198:class_del_profile()) Del profile lustre-client 00000020:00000080:1.0:1787826191.609243:0:111835:0:(genops.c:1183:obd_zombie_import_free()) destroying import ffff9181c39f4800 for lustre-MDT0001-mdc-ffff9181c4d2b000 00000020:00000080:1.0:1787826191.609260:0:111835:0:(genops.c:885:class_export_put()) final put ffff9181c39f7800/a9ce6693-0634-46c5-bf21-5ca7d026d44e 00000020:00000080:1.0:1787826191.609263:0:111835:0:(genops.c:844:class_export_destroy()) destroying export ffff9181c39f7800/a9ce6693-0634-46c5-bf21-5ca7d026d44e for lustre-MDT0001-mdc-ffff9181c4d2b000 00000020:01000000:1.0:1787826191.609271:0:111835:0:(genops.c:441:class_free_dev()) finishing cleanup of obd lustre-MDT0001-mdc-ffff9181c4d2b000 (a9ce6693-0634-46c5-bf21-5ca7d026d44e) 00000020:01000004:3.0:1787826191.612281:0:159403:0:(obd_mount.c:897:lustre_common_put_super()) dropping sb ffff9181c4d2b000 00000100:00080000:3.0:1787826191.612298:0:159403:0:(pinger.c:398:ptlrpc_pinger_del_import()) removing pingable import 7528148b-9bee-4d31-b597-8aae3b0b4982->MGS 00010000:00010000:3.0:1787826191.612312:0:159403:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f116 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1008400000000 nid: local remote: 0x878a5eea00ced198 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1787826191.612323:0:159403:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f116 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1009400000000 nid: local remote: 0x878a5eea00ced198 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787826191.612327:0:159403:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 10000000:01000000:3.0:1787826191.612329:0:159403:0:(mgc_request.c:954:mgc_blocking_ast()) log lustre-s: disconnecting, won't requeue 10000000:01000000:3.0:1787826191.612331:0:159403:0:(mgc_request.c:130:config_log_put()) dropping config log lustre-sptlrpc 00010000:00010000:3.0:1787826191.612365:0:159403:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f116 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x5809400000020 nid: local remote: 0x878a5eea00ced198 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826191.612369:0:159403:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:3.0:1787826191.612381:0:159403:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5cb9600 going to sleep for 0 seconds 00000100:00100000:3.0:1787826191.612390:0:159403:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job umount:7528148b-9bee-4d31-b597-8aae3b0b4982:159403:1874663042982784:192.168.203.147@tcp:103:umount.0 00010000:00010000:3.0:1787826191.612396:0:159403:0:(ldlm_request.c:1625:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway 00010000:00010000:3.0:1787826191.612401:0:159403:0:(ldlm_request.c:2491:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 00010000:00010000:3.0:1787826191.612403:0:159403:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f116 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x5809400000020 nid: local remote: 0x878a5eea00ced198 expref: -99 pid: 15163 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787826191.612421:0:159403:0:(import.c:65:import_set_state_nolock()) ffff9181c5a6c800 MGS: changing import state from FULL to CONNECTING 00000100:00100000:3.0:1787826191.620127:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7528148b-9bee-4d31-b597-8aae3b0b4982:2388:1874663042982912:192.168.203.147@tcp:251:umount.0 00000100:00080000:3.0:1787826191.632216:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5a6c800 MGS: changing import state from CONNECTING to CLOSED 10000000:01000000:3.0:1787826191.632229:0:2388:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808001 00000100:00080000:3.0:1787826191.632233:0:2388:0:(pinger.c:193:ptlrpc_pinger_ir_down()) IR down 10000000:01000000:3.0:1787826191.632234:0:2388:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808002 00000100:00100000:3.0:1787826191.632255:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:7528148b-9bee-4d31-b597-8aae3b0b4982:2388:1874663042982912:192.168.203.147@tcp:251:umount.0 00000100:00080000:0.0:1787826191.634052:0:159403:0:(import.c:138:ptlrpc_deactivate_import_nolock()) setting import MGS INVALID 10000000:01000000:0.0:1787826191.634060:0:159403:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808002 00000100:00100000:0.0:1787826191.634065:0:159403:0:(import.c:323:ptlrpc_invalidate_import()) Sleeping 20s for inflight to error out 10000000:01000000:0.0:1787826191.634068:0:159403:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808003 00000020:00000080:0.0:1787826191.634079:0:159403:0:(genops.c:1468:class_disconnect()) disconnect: cookie 0xbef42c0221d108a6 00000020:00000080:0.0:1787826191.634089:0:159403:0:(genops.c:885:class_export_put()) final put ffff9181d146c000/7528148b-9bee-4d31-b597-8aae3b0b4982 00000020:01000000:0.0:1787826191.634116:0:159403:0:(obd_config.c:2477:class_manual_cleanup()) Manual cleanup of MGC192.168.203.147@tcp (flags='') 00000020:00000080:0.0:1787826191.634125:0:159403:0:(obd_config.c:1472:class_process_config()) processing cmd: cf004 00000020:00000080:0.0:1787826191.634132:0:159403:0:(obd_config.c:956:class_cleanup()) MGC192.168.203.147@tcp: forcing exports to disconnect: 0/3 00000020:00080000:0.0:1787826191.634136:0:159403:0:(genops.c:1558:class_disconnect_exports()) OBD device 0 (ffff9181c3d3cd28) has no exports 00000020:00000080:3.0:1787826191.634215:0:111835:0:(genops.c:844:class_export_destroy()) destroying export ffff9181d146c000/7528148b-9bee-4d31-b597-8aae3b0b4982 for MGC192.168.203.147@tcp 10000000:01000000:2.0:1787826191.635370:0:15163:0:(mgc_request.c:708:mgc_requeue_thread()) Ending requeue thread 10000000:01000000:0.0:1787826191.635814:0:159403:0:(obd_class.h:670:obd_cleanup_client_import()) MGC192.168.203.147@tcp: client import never connected 00000100:00080000:0.0:1787826191.635819:0:159403:0:(import.c:138:ptlrpc_deactivate_import_nolock()) setting import MGS INVALID 10000000:01000000:0.0:1787826191.635827:0:159403:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808002 00000100:00100000:0.0:1787826191.635830:0:159403:0:(import.c:323:ptlrpc_invalidate_import()) Sleeping 20s for inflight to error out 10000000:01000000:0.0:1787826191.635833:0:159403:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808003 00000020:00000080:0.0:1787826191.635922:0:159403:0:(obd_config.c:1472:class_process_config()) processing cmd: cf002 00000020:00000080:0.0:1787826191.635931:0:159403:0:(obd_config.c:883:class_detach()) detach on obd MGC192.168.203.147@tcp (uuid 7528148b-9bee-4d31-b597-8aae3b0b4982) 00000020:01000004:0.0:1787826191.635936:0:159403:0:(obd_mount.c:710:lustre_put_lsi()) put ffff9181c4d2b000 1 00000020:01000004:0.0:1787826191.635940:0:159403:0:(obd_mount.c:668:lustre_free_lsi()) Freeing lsi ffff9181c4d2a000 00000020:00000080:3.0:1787826191.637410:0:111835:0:(genops.c:1183:obd_zombie_import_free()) destroying import ffff9181c5a6c800 for MGC192.168.203.147@tcp 00000020:00000080:3.0:1787826191.637424:0:111835:0:(genops.c:885:class_export_put()) final put ffff9181c4d29800/7528148b-9bee-4d31-b597-8aae3b0b4982 00000020:00000080:3.0:1787826191.637427:0:111835:0:(genops.c:844:class_export_destroy()) destroying export ffff9181c4d29800/7528148b-9bee-4d31-b597-8aae3b0b4982 for MGC192.168.203.147@tcp 00000020:01000000:3.0:1787826191.637435:0:111835:0:(genops.c:441:class_free_dev()) finishing cleanup of obd MGC192.168.203.147@tcp (7528148b-9bee-4d31-b597-8aae3b0b4982) 00010000:00010000:3.0:1787826191.638256:0:15160:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 0 locks of 0 total in blp. (-1 blwis in pool) 00010000:00010000:3.0:1787826191.638441:0:33350:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 0 locks of 0 total in blp. (-2 blwis in pool) 00010000:00010000:3.0:1787826191.639090:0:15161:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 0 locks of 0 total in blp. (-3 blwis in pool) 00010000:00010000:3.0:1787826191.639231:0:33329:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 0 locks of 0 total in blp. (-4 blwis in pool) 00000100:00100000:1.0:1787826191.639860:0:33333:0:(service.c:3183:ptlrpc_main()) ldlm_cb00_002: service thread [ffff9181d1c32400:33333] 2 exiting: rc = 0 00000100:00100000:0.0:1787826191.639990:0:15150:0:(service.c:3183:ptlrpc_main()) ldlm_cb00_000: service thread [ffff9181d001b600:15150] 0 exiting: rc = 1 00000100:00100000:1.0:1787826191.640072:0:15151:0:(service.c:3183:ptlrpc_main()) ldlm_cb00_001: service thread [ffff9181d001ac00:15151] 1 exiting: rc = 0 00000100:00100000:3.0:1787826191.640649:0:33332:0:(service.c:3183:ptlrpc_main()) ldlm_cb01_002: service thread [ffff9181d1c86e00:33332] 2 exiting: rc = 0 00000100:00100000:3.0:1787826191.641005:0:15153:0:(service.c:3183:ptlrpc_main()) ldlm_cb01_001: service thread [ffff9181c2fc6a00:15153] 1 exiting: rc = 0 00000100:00100000:2.0:1787826191.641637:0:15152:0:(service.c:3183:ptlrpc_main()) ldlm_cb01_000: service thread [ffff9181c59f3800:15152] 0 exiting: rc = 1 00000100:00100000:3.0:1787826191.642033:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:3.0:1787826191.642043:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.649089:0:15154:0:(service.c:3183:ptlrpc_main()) ldlm_cn00_000: service thread [ffff9181c59f2600:15154] 0 exiting: rc = 1 00000100:00100000:1.0:1787826191.650679:0:15155:0:(service.c:3183:ptlrpc_main()) ldlm_cn00_001: service thread [ffff9181d14e6600:15155] 1 exiting: rc = 0 00000100:00100000:1.0:1787826191.651004:0:15156:0:(service.c:3183:ptlrpc_main()) ldlm_cn00_002: service thread [ffff9181d14e7600:15156] 2 exiting: rc = 0 00000100:00100000:3.0:1787826191.667688:0:15158:0:(service.c:3183:ptlrpc_main()) ldlm_cn01_001: service thread [ffff9181d14e6e00:15158] 1 exiting: rc = 0 00000100:00100000:2.0:1787826191.668576:0:15157:0:(service.c:3183:ptlrpc_main()) ldlm_cn01_000: service thread [ffff9181d14e6400:15157] 0 exiting: rc = 1 00000100:00100000:2.0:1787826191.668650:0:15159:0:(service.c:3183:ptlrpc_main()) ldlm_cn01_002: service thread [ffff9181d38b6400:15159] 2 exiting: rc = 0 00000100:00100000:1.0:1787826191.672711:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.672721:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.672728:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.672733:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.672738:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.672759:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.672765:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.672771:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.672776:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.672781:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.672786:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.672791:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.672796:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.672801:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.672806:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.672811:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.672816:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.672821:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.672827:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.672832:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.672836:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.672841:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.672855:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.672860:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.672865:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.672871:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.672876:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.672880:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.672886:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.674379:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.674401:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.674406:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.674412:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.674417:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.674422:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.674428:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.674433:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.674438:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.674444:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.674449:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.674453:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.674458:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.674463:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.675653:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.675660:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.675666:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.675672:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.675679:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.675684:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.675690:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.675696:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.675702:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.675708:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.675714:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.675720:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.675726:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.675732:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.675760:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.675768:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.675775:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.675780:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.675786:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.675792:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.675799:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.675807:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.675813:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.675819:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.675826:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.675832:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.675838:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.675854:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.675860:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.675865:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.675870:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.675875:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.675880:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.675885:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.675890:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.676021:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.676030:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.676035:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.676040:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.676046:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.676051:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.676056:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.676061:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.676068:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.676073:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.676079:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.676085:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.676091:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.676096:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.676103:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.676109:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.676114:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.676120:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.676126:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.676131:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.676137:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.676144:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.676151:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.676157:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.676164:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.676170:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.676176:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.676183:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.676189:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.676196:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.676203:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.676210:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.676664:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.676673:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.676679:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.676686:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.676693:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.676699:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.676704:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.676710:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.676717:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.676723:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.676728:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.676758:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.676766:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.676772:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.676779:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.676785:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.676792:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000100:00100000:1.0:1787826191.676798:0:111835:0:(events.c:340:request_in_callback()) peer: 0-0@<0:0> (source: 0-0@<0:0>) 00000020:01000004:0.0:1787826234.893604:0:159721:0:(obd_mount.c:923:lmd_print()) mount data: 00000020:01000004:0.0:1787826234.893610:0:159721:0:(obd_mount.c:925:lmd_print()) profile: lustre-client 00000020:01000004:0.0:1787826234.893611:0:159721:0:(obd_mount.c:926:lmd_print()) device: 192.168.203.147@tcp:/lustre 00000020:01000004:0.0:1787826234.893613:0:159721:0:(obd_mount.c:929:lmd_print()) options: user_xattr,flock 00000080:01200004:0.0:1787826234.893718:0:159721:0:(super25.c:131:lustre_fill_super()) VFS Op: sb ffff9181c5620000 00000080:01000004:0.0:1787826234.893727:0:159721:0:(super25.c:162:lustre_fill_super()) Mounting client lustre-client 00000020:01000004:0.0:1787826234.893741:0:159721:0:(obd_mount.c:411:lustre_start_mgc()) Start MGC 'MGC192.168.203.147@tcp' 00000020:01000004:0.0:1787826234.893809:0:159721:0:(obd_mount.c:230:lustre_start_simple()) Starting OBD MGC192.168.203.147@tcp (typ=mgc) 00000020:00000080:0.0:1787826234.893823:0:159721:0:(genops.c:413:class_newdev()) Allocate new device MGC192.168.203.147@tcp (ffff9181c87b19b8) 00000020:00000080:0.0:1787826234.893837:0:159721:0:(obd_config.c:720:class_attach_name()) OBD: dev 0 attached type mgc with refcount 2 00000100:00100000:0.0:1787826234.894005:0:159721:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/1 left), rc = 0 00000100:00100000:0.0:1787826234.894026:0:159721:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/1 left), rc = 0 00000100:00100000:0.0:1787826234.894153:0:159721:0:(service.c:3462:ptlrpc_start_thread()) ldlm_cbd[0] started 0 min 2 max 34 00000100:00100000:0.0:1787826234.894159:0:159721:0:(service.c:3521:ptlrpc_start_thread()) starting thread 'ldlm_cb00_000' 00000100:00100000:2.0:1787826234.894855:0:159721:0:(service.c:3462:ptlrpc_start_thread()) ldlm_cbd[0] started 1 min 2 max 34 00000100:00100000:2.0:1787826234.894863:0:159721:0:(service.c:3521:ptlrpc_start_thread()) starting thread 'ldlm_cb00_001' 00000100:00100000:2.0:1787826234.895204:0:159721:0:(service.c:3462:ptlrpc_start_thread()) ldlm_cbd[1] started 0 min 2 max 34 00000100:00100000:2.0:1787826234.895210:0:159721:0:(service.c:3521:ptlrpc_start_thread()) starting thread 'ldlm_cb01_000' 00000100:00100000:0.0:1787826234.895689:0:159721:0:(service.c:3462:ptlrpc_start_thread()) ldlm_cbd[1] started 1 min 2 max 34 00000100:00100000:0.0:1787826234.895696:0:159721:0:(service.c:3521:ptlrpc_start_thread()) starting thread 'ldlm_cb01_001' 00000100:00100000:0.0:1787826234.896418:0:159721:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_canceld: allocate 64 new 8192-byte reqbufs (0/64 left), rc = 0 00000100:00100000:0.0:1787826234.896727:0:159721:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_canceld: allocate 64 new 8192-byte reqbufs (0/64 left), rc = 0 00000100:00100000:0.0:1787826234.897105:0:159721:0:(service.c:3462:ptlrpc_start_thread()) ldlm_canceld[0] started 0 min 3 max 34 00000100:00100000:0.0:1787826234.897110:0:159721:0:(service.c:3521:ptlrpc_start_thread()) starting thread 'ldlm_cn00_000' 00000100:00100000:0.0:1787826234.897574:0:159721:0:(service.c:3462:ptlrpc_start_thread()) ldlm_canceld[0] started 1 min 3 max 34 00000100:00100000:0.0:1787826234.897581:0:159721:0:(service.c:3521:ptlrpc_start_thread()) starting thread 'ldlm_cn00_001' 00000100:00100000:2.0:1787826234.897956:0:159721:0:(service.c:3462:ptlrpc_start_thread()) ldlm_canceld[0] started 2 min 3 max 34 00000100:00100000:2.0:1787826234.897963:0:159721:0:(service.c:3521:ptlrpc_start_thread()) starting thread 'ldlm_cn00_002' 00000100:00100000:2.0:1787826234.898633:0:159721:0:(service.c:3462:ptlrpc_start_thread()) ldlm_canceld[1] started 0 min 3 max 34 00000100:00100000:2.0:1787826234.898640:0:159721:0:(service.c:3521:ptlrpc_start_thread()) starting thread 'ldlm_cn01_000' 00000100:00100000:0.0:1787826234.899194:0:159721:0:(service.c:3462:ptlrpc_start_thread()) ldlm_canceld[1] started 1 min 3 max 34 00000100:00100000:0.0:1787826234.899200:0:159721:0:(service.c:3521:ptlrpc_start_thread()) starting thread 'ldlm_cn01_001' 00000100:00100000:0.0:1787826234.899535:0:159721:0:(service.c:3462:ptlrpc_start_thread()) ldlm_canceld[1] started 2 min 3 max 34 00000100:00100000:0.0:1787826234.899550:0:159721:0:(service.c:3521:ptlrpc_start_thread()) starting thread 'ldlm_cn01_002' 00010000:00010000:3.0:1787826234.900257:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826234.900260:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826234.900262:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826234.900532:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826234.900535:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826234.900537:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00080000:0.0:1787826234.901039:0:159721:0:(ldlm_lib.c:102:import_set_conn()) imp ffff9181c4d28800@MGC192.168.203.147@tcp: add connection 192.168.203.147@tcp at head 00010000:00010000:0.0:1787826234.901202:0:159721:0:(ldlm_pool.c:973:ldlm_pool_init()) Lock pool ldlm-pool-MGC192.168.203.147@tcp-0 is initialized 00000040:01000000:0.0:1787826234.901209:0:159721:0:(llog_obd.c:197:llog_setup()) obd MGC192.168.203.147@tcp ctxt 1 is initialized 10000000:01000000:1.0:1787826234.902098:0:159735:0:(mgc_request.c:631:mgc_requeue_thread()) Starting requeue thread 00000020:00000080:2.0:1787826234.902161:0:159721:0:(obd_config.c:826:class_setup()) finished setup of obd MGC192.168.203.147@tcp (uuid f40147eb-9b53-47d4-b77c-99ceedd9664c) 00000020:00000080:2.0:1787826234.902182:0:159721:0:(genops.c:1385:class_connect()) connect: client f40147eb-9b53-47d4-b77c-99ceedd9664c, cookie 0xbef42c0221d6f140 00000100:00080000:2.0:1787826234.902195:0:159721:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 0 00000100:00080000:2.0:1787826234.902205:0:159721:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787826234.902234:0:159721:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c71f0e00 x1874663042983040/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826234.902281:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663042983040:192.168.203.147@tcp:250:mount.lustre.0 00000100:00080000:2.0:1787826234.902380:0:159721:0:(pinger.c:373:ptlrpc_pinger_add_import()) adding pingable import f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS 00000080:00200000:2.0:1787826234.902405:0:159721:0:(llite_lib.c:1506:ll_fill_super()) VFS Op: cfg_instance lustre-client-ffff9181c5620000 (sb ffff9181c5620000) 00000080:01000000:2.0:1787826234.902477:0:159721:0:(llite_lib.c:1232:ll_options()) Parsing opts user_xattr,flock 00000080:00000004:2.0:1787826234.902479:0:159721:0:(llite_lib.c:1242:ll_options()) next opt=user_xattr 00000080:00000004:2.0:1787826234.902484:0:159721:0:(llite_lib.c:1242:ll_options()) next opt=flock 00000080:01000000:2.0:1787826234.902489:0:159721:0:(llite_lib.c:1540:ll_fill_super()) llite sb uuid: c0d7758a-4501-4a89-8ec1-407fef4f0c7f 10000000:01000000:2.0:1787826234.902878:0:159721:0:(mgc_request.c:1944:mgc_process_config()) parse_log lustre-client from 0 10000000:01000000:2.0:1787826234.902881:0:159721:0:(mgc_request.c:329:config_log_add()) add config log lustre-client-ffff9181c5620000 10000000:01000000:2.0:1787826234.902885:0:159721:0:(mgc_request.c:194:do_config_log_add()) do adding config log lustre-sptlrpc-ffff9181c87b19b8 10000000:01000000:2.0:1787826234.902889:0:159721:0:(mgc_request.c:68:mgc_name2resid()) log lustre-sptlrpc to resid 0x65727473756c/0x0 (lustre) 10000000:01000000:2.0:1787826234.902896:0:159721:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff9181c87b19b8 from 1 10000000:01000000:2.0:1787826234.902899:0:159721:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00010000:00010000:2.0:1787826234.902928:0:159721:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e06f6000/0xbef42c0221d6f147 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826234.902938:0:159721:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e06f6000/0xbef42c0221d6f147 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826234.902950:0:159721:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e06f6000/0xbef42c0221d6f147 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787826234.902968:0:159721:0:(client.c:1845:ptlrpc_send_new_req()) @@@ req waiting for recovery: (FULL != CONNECTING) req@ffff9181c71f1f80 x1874663042983168/t0(0) o101->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 328/344 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'mount.lustre.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826234.902977:0:159721:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1c5bf00 going to sleep for 0 seconds 00010000:00010000:0.0:1787826234.907513:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826234.915185:0:2388:0:(client.c:3125:ptlrpc_free_committed()) MGC192.168.203.147@tcp: committing for last_committed 0 gen 1 00000100:00080000:3.0:1787826234.915215:0:2388:0:(import.c:1132:ptlrpc_connect_interpret()) MGC192.168.203.147@tcp: connect to target with instance 0 00000100:00080000:3.0:1787826234.915224:0:2388:0:(import.c:956:ptlrpc_connect_set_flags()) MGC192.168.203.147@tcp: Resetting ns_connect_flags to server flags: 0xa000011001002060 10000000:01000000:3.0:1787826234.915239:0:2388:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808005 00000100:00080000:3.0:1787826234.915244:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to FULL 10000000:01000000:3.0:1787826234.915248:0:2388:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808004 00000100:00080000:3.0:1787826234.915266:0:2388:0:(pinger.c:186:ptlrpc_pinger_ir_up()) IR up 00000100:00080000:3.0:1787826234.915277:0:2388:0:(recover.c:191:ptlrpc_wake_delayed()) @@@ waking (set 00000000c50a68a7): req@ffff9181c71f1f80 x1874663042983168/t0(0) o101->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 328/344 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'mount.lustre.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826234.915303:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663042983040:192.168.203.147@tcp:250:mount.lustre.0 00000100:00100000:2.0:1787826234.916421:0:159721:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1c5bf00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826234.923003:0:159721:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f1f80 pname:cluuid:pid:xid:nid:opc:job mount.lustre:f40147eb-9b53-47d4-b77c-99ceedd9664c:159721:1874663042983168:192.168.203.147@tcp:101:mount.lustre.0 00010000:00010000:2.0:1787826234.923034:0:159721:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.147@tcp lock: ffff9181e06f6000/0xbef42c0221d6f147 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00ced6d1 expref: -99 pid: 159721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826234.923047:0:159721:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.147@tcp lock: ffff9181e06f6000/0xbef42c0221d6f147 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x878a5eea00ced6d1 expref: -99 pid: 159721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826234.923127:0:159721:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f1f80 pname:cluuid:pid:xid:nid:opc:job mount.lustre:f40147eb-9b53-47d4-b77c-99ceedd9664c:159721:1874663042983296:192.168.203.147@tcp:501:mount.lustre.0 00000100:00100000:2.0:1787826234.923209:0:159721:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1c5bf00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826234.927875:0:159721:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f1f80 pname:cluuid:pid:xid:nid:opc:job mount.lustre:f40147eb-9b53-47d4-b77c-99ceedd9664c:159721:1874663042983296:192.168.203.147@tcp:501:mount.lustre.0 10000000:01000000:2.0:1787826234.927921:0:159721:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-sptlrpc' failed (-2). 00010000:00010000:2.0:1787826234.927953:0:159721:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e06f6000/0xbef42c0221d6f147 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00ced6d1 expref: -99 pid: 159721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826234.927965:0:159721:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.147@tcp lock: ffff9181e06f6000/0xbef42c0221d6f147 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00ced6d1 expref: -99 pid: 159721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787826234.927974:0:159721:0:(mgc_request.c:194:do_config_log_add()) do adding config log params-ffff9181c5620000 10000000:01000000:2.0:1787826234.927980:0:159721:0:(mgc_request.c:68:mgc_name2resid()) log params to resid 0x736d61726170/0x3 (params) 10000000:01000000:2.0:1787826234.927985:0:159721:0:(mgc_request.c:194:do_config_log_add()) do adding config log lustre-client-ffff9181c5620000 10000000:01000000:2.0:1787826234.927987:0:159721:0:(mgc_request.c:68:mgc_name2resid()) log lustre-client to resid 0x65727473756c/0x0 (lustre) 10000000:01000000:2.0:1787826234.928002:0:159721:0:(mgc_request.c:194:do_config_log_add()) do adding config log lustre-cliir-ffff9181c5620000 10000000:01000000:2.0:1787826234.928005:0:159721:0:(mgc_request.c:68:mgc_name2resid()) log lustre-cliir to resid 0x65727473756c/0x2 (lustre) 10000000:01000000:2.0:1787826234.928008:0:159721:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff9181c5620000 from 1 10000000:01000000:2.0:1787826234.928011:0:159721:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:2.0:1787826234.928040:0:159721:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e06f6a00/0xbef42c0221d6f14e lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826234.928049:0:159721:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e06f6a00/0xbef42c0221d6f14e lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826234.928060:0:159721:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e06f6a00/0xbef42c0221d6f14e lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826234.928089:0:159721:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f1f80 pname:cluuid:pid:xid:nid:opc:job mount.lustre:f40147eb-9b53-47d4-b77c-99ceedd9664c:159721:1874663042983424:192.168.203.147@tcp:101:mount.lustre.0 00000100:00100000:2.0:1787826234.928435:0:159721:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1c5bf00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826234.933603:0:159721:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f1f80 pname:cluuid:pid:xid:nid:opc:job mount.lustre:f40147eb-9b53-47d4-b77c-99ceedd9664c:159721:1874663042983424:192.168.203.147@tcp:101:mount.lustre.0 00010000:00010000:2.0:1787826234.933634:0:159721:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.147@tcp lock: ffff9181e06f6a00/0xbef42c0221d6f14e lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00ced6d8 expref: -99 pid: 159721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826234.933647:0:159721:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.147@tcp lock: ffff9181e06f6a00/0xbef42c0221d6f14e lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x878a5eea00ced6d8 expref: -99 pid: 159721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826234.933723:0:159721:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f1f80 pname:cluuid:pid:xid:nid:opc:job mount.lustre:f40147eb-9b53-47d4-b77c-99ceedd9664c:159721:1874663042983552:192.168.203.147@tcp:501:mount.lustre.0 00000100:00100000:2.0:1787826234.933813:0:159721:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1c5bf00 going to sleep for 11 seconds 00000100:00100000:3.0:1787826234.947259:0:159721:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f1f80 pname:cluuid:pid:xid:nid:opc:job mount.lustre:f40147eb-9b53-47d4-b77c-99ceedd9664c:159721:1874663042983552:192.168.203.147@tcp:501:mount.lustre.0 00000100:00100000:3.0:1787826234.947361:0:159721:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job mount.lustre:f40147eb-9b53-47d4-b77c-99ceedd9664c:159721:1874663042983680:192.168.203.147@tcp:503:mount.lustre.0 00000100:00100000:3.0:1787826234.947478:0:159721:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5cb99c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826234.958835:0:159721:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job mount.lustre:f40147eb-9b53-47d4-b77c-99ceedd9664c:159721:1874663042983680:192.168.203.147@tcp:503:mount.lustre.0 00000100:00100000:3.0:1787826234.964682:0:159739:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job llog_process_th:f40147eb-9b53-47d4-b77c-99ceedd9664c:159739:1874663042983808:192.168.203.147@tcp:502:llog_process_th.0 00000100:00100000:3.0:1787826234.964872:0:159739:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5cb9cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826234.970587:0:159739:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job llog_process_th:f40147eb-9b53-47d4-b77c-99ceedd9664c:159739:1874663042983808:192.168.203.147@tcp:502:llog_process_th.0 00000020:01000000:3.0:1787826234.970641:0:159739:0:(obd_config.c:1893:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x1 00000020:00000080:3.0:1787826234.970652:0:159739:0:(obd_config.c:1472:class_process_config()) processing cmd: cf010 00000020:00000080:3.0:1787826234.970654:0:159739:0:(obd_config.c:1563:class_process_config()) marker 4 (0x1) lustre-clilov lov setup 00000020:01000000:3.0:1787826234.970661:0:159739:0:(obd_config.c:2004:class_config_llog_handler()) cmd cf001, instance name: lustre-clilov-ffff9181c5620000 00000020:00000080:3.0:1787826234.970663:0:159739:0:(obd_config.c:1472:class_process_config()) processing cmd: cf001 00000020:00000080:3.0:1787826234.970677:0:159739:0:(genops.c:413:class_newdev()) Allocate new device lustre-clilov-ffff9181c5620000 (ffff9181d1a1b370) 00000020:00000080:3.0:1787826234.970699:0:159739:0:(obd_config.c:720:class_attach_name()) OBD: dev 1 attached type lov with refcount 2 00000020:01000000:3.0:1787826234.970705:0:159739:0:(obd_config.c:2004:class_config_llog_handler()) cmd cf003, instance name: lustre-clilov-ffff9181c5620000 00000020:00000080:3.0:1787826234.970707:0:159739:0:(obd_config.c:1472:class_process_config()) processing cmd: cf003 00000020:00000080:3.0:1787826234.971254:0:159739:0:(obd_config.c:826:class_setup()) finished setup of obd lustre-clilov-ffff9181c5620000 (uuid c0d7758a-4501-4a89-8ec1-407fef4f0c7f) 00000020:01000000:3.0:1787826234.971260:0:159739:0:(obd_config.c:1893:class_config_llog_handler()) Marker, inst_flg=0x2 mark_flg=0x2 00000020:00000080:3.0:1787826234.971264:0:159739:0:(obd_config.c:1472:class_process_config()) processing cmd: cf010 00000020:00000080:3.0:1787826234.971265:0:159739:0:(obd_config.c:1563:class_process_config()) marker 4 (0x2) lustre-clilov lov setup 00000020:01000000:3.0:1787826234.971268:0:159739:0:(obd_config.c:1893:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x1 00000020:00000080:3.0:1787826234.971272:0:159739:0:(obd_config.c:1472:class_process_config()) processing cmd: cf010 00000020:00000080:3.0:1787826234.971274:0:159739:0:(obd_config.c:1563:class_process_config()) marker 5 (0x1) lustre-clilmv lmv setup 00000020:01000000:3.0:1787826234.971277:0:159739:0:(obd_config.c:2004:class_config_llog_handler()) cmd cf001, instance name: lustre-clilmv-ffff9181c5620000 00000020:00000080:3.0:1787826234.971279:0:159739:0:(obd_config.c:1472:class_process_config()) processing cmd: cf001 00000020:00000080:3.0:1787826234.971285:0:159739:0:(genops.c:413:class_newdev()) Allocate new device lustre-clilmv-ffff9181c5620000 (ffff9181d1a199b8) 00000020:00000080:3.0:1787826234.971296:0:159739:0:(obd_config.c:720:class_attach_name()) OBD: dev 2 attached type lmv with refcount 2 00000020:01000000:3.0:1787826234.971300:0:159739:0:(obd_config.c:2004:class_config_llog_handler()) cmd cf003, instance name: lustre-clilmv-ffff9181c5620000 00000020:00000080:3.0:1787826234.971302:0:159739:0:(obd_config.c:1472:class_process_config()) processing cmd: cf003 00000020:00000080:2.0:1787826234.976809:0:159739:0:(obd_config.c:826:class_setup()) finished setup of obd lustre-clilmv-ffff9181c5620000 (uuid c0d7758a-4501-4a89-8ec1-407fef4f0c7f) 00000020:01000000:2.0:1787826234.976827:0:159739:0:(obd_config.c:1893:class_config_llog_handler()) Marker, inst_flg=0x2 mark_flg=0x2 00000020:00000080:2.0:1787826234.976835:0:159739:0:(obd_config.c:1472:class_process_config()) processing cmd: cf010 00000020:00000080:2.0:1787826234.976837:0:159739:0:(obd_config.c:1563:class_process_config()) marker 5 (0x2) lustre-clilmv lmv setup 00000020:01000000:2.0:1787826234.977794:0:159739:0:(obd_config.c:1893:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x1 00000020:00000080:2.0:1787826234.977819:0:159739:0:(obd_config.c:1472:class_process_config()) processing cmd: cf010 00000020:00000080:2.0:1787826234.977822:0:159739:0:(obd_config.c:1563:class_process_config()) marker 6 (0x1) lustre-client mount opts 00000020:00000080:2.0:1787826234.977830:0:159739:0:(obd_config.c:1472:class_process_config()) processing cmd: cf007 00000020:00000080:2.0:1787826234.977832:0:159739:0:(obd_config.c:1516:class_process_config()) mountopt: profile lustre-client osc lustre-clilov mdc lustre-clilmv 00000020:01000000:2.0:1787826234.977835:0:159739:0:(obd_config.c:1149:class_add_profile()) Add profile lustre-client 00000020:01000000:2.0:1787826234.977841:0:159739:0:(obd_config.c:1893:class_config_llog_handler()) Marker, inst_flg=0x2 mark_flg=0x2 00000020:00000080:2.0:1787826234.977844:0:159739:0:(obd_config.c:1472:class_process_config()) processing cmd: cf010 00000020:00000080:2.0:1787826234.977953:0:159739:0:(obd_config.c:1563:class_process_config()) marker 6 (0x2) lustre-client mount opts 00000020:01000000:2.0:1787826234.977958:0:159739:0:(obd_config.c:1893:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x1 00000020:00000080:2.0:1787826234.977966:0:159739:0:(obd_config.c:1472:class_process_config()) processing cmd: cf010 00000020:00000080:2.0:1787826234.977967:0:159739:0:(obd_config.c:1563:class_process_config()) marker 7 (0x1) lustre-MDT0000 add mdc 00000020:00000080:2.0:1787826234.977972:0:159739:0:(obd_config.c:1472:class_process_config()) processing cmd: cf005 00000020:00000080:2.0:1787826234.977977:0:159739:0:(obd_config.c:1487:class_process_config()) adding mapping from uuid 192.168.203.147@tcp to nid 0x20000c0a8cb93 (192.168.203.147@tcp) 00000020:01000000:2.0:1787826234.978010:0:159739:0:(obd_config.c:2004:class_config_llog_handler()) cmd cf001, instance name: lustre-MDT0000-mdc-ffff9181c5620000 00000020:00000080:2.0:1787826234.978014:0:159739:0:(obd_config.c:1472:class_process_config()) processing cmd: cf001 00000020:00000080:2.0:1787826234.978026:0:159739:0:(genops.c:413:class_newdev()) Allocate new device lustre-MDT0000-mdc-ffff9181c5620000 (ffff9181c354cd28) 00000020:00000080:2.0:1787826234.978048:0:159739:0:(obd_config.c:720:class_attach_name()) OBD: dev 3 attached type mdc with refcount 2 00000020:01000000:2.0:1787826234.978054:0:159739:0:(obd_config.c:2004:class_config_llog_handler()) cmd cf003, instance name: lustre-MDT0000-mdc-ffff9181c5620000 00000020:00000080:2.0:1787826234.978056:0:159739:0:(obd_config.c:1472:class_process_config()) processing cmd: cf003 00010000:00080000:2.0:1787826234.978139:0:159739:0:(ldlm_lib.c:102:import_set_conn()) imp ffff9181c5bc0000@lustre-MDT0000-mdc-ffff9181c5620000: add connection 192.168.203.147@tcp at head 00010000:00010000:2.0:1787826234.980295:0:159739:0:(ldlm_pool.c:973:ldlm_pool_init()) Lock pool ldlm-pool-lustre-MDT0000-mdc-ffff9181c5620000-1 is initialized 00000040:01000000:2.0:1787826234.980560:0:159739:0:(llog_obd.c:197:llog_setup()) obd lustre-MDT0000-mdc-ffff9181c5620000 ctxt 13 is initialized 00000020:00000080:2.0:1787826234.980718:0:159739:0:(obd_config.c:826:class_setup()) finished setup of obd lustre-MDT0000-mdc-ffff9181c5620000 (uuid c0d7758a-4501-4a89-8ec1-407fef4f0c7f) 00000020:01000000:2.0:1787826234.980731:0:159739:0:(obd_config.c:2004:class_config_llog_handler()) cmd cf014, instance name: lustre-clilmv-ffff9181c5620000 00000020:00000080:2.0:1787826234.980736:0:159739:0:(obd_config.c:1472:class_process_config()) processing cmd: cf014 00800000:01000000:2.0:1787826234.980811:0:159739:0:(lmv_obd.c:442:lmv_add_target()) Target uuid: lustre-MDT0000_UUID. index 0 00000020:01000000:2.0:1787826234.981645:0:159739:0:(obd_config.c:1893:class_config_llog_handler()) Marker, inst_flg=0x2 mark_flg=0x2 00000020:00000080:2.0:1787826234.981652:0:159739:0:(obd_config.c:1472:class_process_config()) processing cmd: cf010 00000020:00000080:2.0:1787826234.981654:0:159739:0:(obd_config.c:1563:class_process_config()) marker 7 (0x2) lustre-MDT0000 add mdc 00000020:01000000:2.0:1787826234.981658:0:159739:0:(obd_config.c:1893:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x5 00000020:01000000:2.0:1787826234.981661:0:159739:0:(obd_config.c:1902:class_config_llog_handler()) SKIP #8 00000020:01000000:2.0:1787826234.981663:0:159739:0:(obd_config.c:1928:class_config_llog_handler()) skipping 0x6 00000020:01000000:2.0:1787826234.981664:0:159739:0:(obd_config.c:1928:class_config_llog_handler()) skipping 0x6 00000020:01000000:2.0:1787826234.981665:0:159739:0:(obd_config.c:1893:class_config_llog_handler()) Marker, inst_flg=0x6 mark_flg=0x6 00000020:00000080:2.0:1787826234.981668:0:159739:0:(obd_config.c:1472:class_process_config()) processing cmd: cf010 00000020:00000080:2.0:1787826234.981669:0:159739:0:(obd_config.c:1563:class_process_config()) marker 8 (0x6) lustre sys.timeout 00000020:01000000:2.0:1787826234.981672:0:159739:0:(obd_config.c:1893:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x1 00000020:00000080:2.0:1787826234.981676:0:159739:0:(obd_config.c:1472:class_process_config()) processing cmd: cf010 00000020:00000080:2.0:1787826234.981677:0:159739:0:(obd_config.c:1563:class_process_config()) marker 16 (0x1) lustre-MDT0001 add mdc 00000020:00000080:2.0:1787826234.981680:0:159739:0:(obd_config.c:1472:class_process_config()) processing cmd: cf005 00000020:00000080:2.0:1787826234.981685:0:159739:0:(obd_config.c:1487:class_process_config()) adding mapping from uuid 192.168.203.147@tcp to nid 0x20000c0a8cb93 (192.168.203.147@tcp) 00000020:01000000:2.0:1787826234.981709:0:159739:0:(obd_config.c:2004:class_config_llog_handler()) cmd cf001, instance name: lustre-MDT0001-mdc-ffff9181c5620000 00000020:00000080:2.0:1787826234.981711:0:159739:0:(obd_config.c:1472:class_process_config()) processing cmd: cf001 00000020:00000080:2.0:1787826234.981721:0:159739:0:(genops.c:413:class_newdev()) Allocate new device lustre-MDT0001-mdc-ffff9181c5620000 (ffff9181c3548000) 00000020:00000080:2.0:1787826234.981734:0:159739:0:(obd_config.c:720:class_attach_name()) OBD: dev 4 attached type mdc with refcount 2 00000020:01000000:2.0:1787826234.981739:0:159739:0:(obd_config.c:2004:class_config_llog_handler()) cmd cf003, instance name: lustre-MDT0001-mdc-ffff9181c5620000 00000020:00000080:2.0:1787826234.981742:0:159739:0:(obd_config.c:1472:class_process_config()) processing cmd: cf003 00010000:00080000:2.0:1787826234.981833:0:159739:0:(ldlm_lib.c:102:import_set_conn()) imp ffff9181c5bc2800@lustre-MDT0001-mdc-ffff9181c5620000: add connection 192.168.203.147@tcp at head 00010000:00010000:2.0:1787826234.988881:0:159739:0:(ldlm_pool.c:973:ldlm_pool_init()) Lock pool ldlm-pool-lustre-MDT0001-mdc-ffff9181c5620000-2 is initialized 00000040:01000000:2.0:1787826234.989366:0:159739:0:(llog_obd.c:197:llog_setup()) obd lustre-MDT0001-mdc-ffff9181c5620000 ctxt 13 is initialized 00000020:00000080:2.0:1787826234.989697:0:159739:0:(obd_config.c:826:class_setup()) finished setup of obd lustre-MDT0001-mdc-ffff9181c5620000 (uuid c0d7758a-4501-4a89-8ec1-407fef4f0c7f) 00000020:01000000:2.0:1787826234.989713:0:159739:0:(obd_config.c:2004:class_config_llog_handler()) cmd cf014, instance name: lustre-clilmv-ffff9181c5620000 00000020:00000080:2.0:1787826234.989718:0:159739:0:(obd_config.c:1472:class_process_config()) processing cmd: cf014 00800000:01000000:2.0:1787826234.989765:0:159739:0:(lmv_obd.c:442:lmv_add_target()) Target uuid: lustre-MDT0001_UUID. index 1 00000020:01000000:2.0:1787826234.989804:0:159739:0:(obd_config.c:1893:class_config_llog_handler()) Marker, inst_flg=0x2 mark_flg=0x2 00000020:00000080:2.0:1787826234.989808:0:159739:0:(obd_config.c:1472:class_process_config()) processing cmd: cf010 00000020:00000080:2.0:1787826234.989811:0:159739:0:(obd_config.c:1563:class_process_config()) marker 16 (0x2) lustre-MDT0001 add mdc 00000020:01000000:2.0:1787826234.989815:0:159739:0:(obd_config.c:1893:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x5 00000020:01000000:2.0:1787826234.989818:0:159739:0:(obd_config.c:1902:class_config_llog_handler()) SKIP #18 00000020:01000000:2.0:1787826234.989820:0:159739:0:(obd_config.c:1928:class_config_llog_handler()) skipping 0x6 00000020:01000000:2.0:1787826234.989822:0:159739:0:(obd_config.c:1928:class_config_llog_handler()) skipping 0x6 00000020:01000000:2.0:1787826234.989823:0:159739:0:(obd_config.c:1893:class_config_llog_handler()) Marker, inst_flg=0x6 mark_flg=0x6 00000020:00000080:2.0:1787826234.989826:0:159739:0:(obd_config.c:1472:class_process_config()) processing cmd: cf010 00000020:00000080:2.0:1787826234.989828:0:159739:0:(obd_config.c:1563:class_process_config()) marker 18 (0x6) lustre sys.timeout 00000020:01000000:2.0:1787826234.989832:0:159739:0:(obd_config.c:1893:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x1 00000020:01000004:2.0:1787826234.989837:0:159739:0:(obd_mount.c:961:lustre_check_exclusion()) Check exclusion lustre-OST0000 (0) in 0 of 192.168.203.147@tcp:/lustre 00000020:00000080:2.0:1787826234.989841:0:159739:0:(obd_config.c:1472:class_process_config()) processing cmd: cf010 00000020:00000080:2.0:1787826234.989843:0:159739:0:(obd_config.c:1563:class_process_config()) marker 24 (0x1) lustre-OST0000 add osc 00000020:00000080:2.0:1787826234.989918:0:159739:0:(obd_config.c:1472:class_process_config()) processing cmd: cf005 00000020:00000080:2.0:1787826234.989924:0:159739:0:(obd_config.c:1487:class_process_config()) adding mapping from uuid 192.168.203.147@tcp to nid 0x20000c0a8cb93 (192.168.203.147@tcp) 00000020:01000000:2.0:1787826234.990034:0:159739:0:(obd_config.c:2004:class_config_llog_handler()) cmd cf001, instance name: lustre-OST0000-osc-ffff9181c5620000 00000020:00000080:2.0:1787826234.990037:0:159739:0:(obd_config.c:1472:class_process_config()) processing cmd: cf001 00000020:00000080:2.0:1787826234.990048:0:159739:0:(genops.c:413:class_newdev()) Allocate new device lustre-OST0000-osc-ffff9181c5620000 (ffff9181c35499b8) 00000020:00000080:2.0:1787826234.990063:0:159739:0:(obd_config.c:720:class_attach_name()) OBD: dev 5 attached type osc with refcount 2 00000020:01000000:2.0:1787826234.990069:0:159739:0:(obd_config.c:2004:class_config_llog_handler()) cmd cf003, instance name: lustre-OST0000-osc-ffff9181c5620000 00000020:00000080:2.0:1787826234.990071:0:159739:0:(obd_config.c:1472:class_process_config()) processing cmd: cf003 00010000:00080000:2.0:1787826234.990149:0:159739:0:(ldlm_lib.c:102:import_set_conn()) imp ffff9181c5bc3800@lustre-OST0000-osc-ffff9181c5620000: add connection 192.168.203.147@tcp at head 00010000:00010000:2.0:1787826234.990365:0:159739:0:(ldlm_pool.c:973:ldlm_pool_init()) Lock pool ldlm-pool-lustre-OST0000-osc-ffff9181c5620000-3 is initialized 00000020:00000080:2.0:1787826234.990677:0:159739:0:(obd_config.c:826:class_setup()) finished setup of obd lustre-OST0000-osc-ffff9181c5620000 (uuid c0d7758a-4501-4a89-8ec1-407fef4f0c7f) 00000020:01000000:2.0:1787826234.990686:0:159739:0:(obd_config.c:2004:class_config_llog_handler()) cmd cf00d, instance name: lustre-clilov-ffff9181c5620000 00000020:00000080:2.0:1787826234.990689:0:159739:0:(obd_config.c:1472:class_process_config()) processing cmd: cf00d 00020000:01000000:2.0:1787826234.990702:0:159739:0:(lov_obd.c:481:lov_add_target()) uuid:lustre-OST0000_UUID idx:0 gen:1 active:1 00020000:01000000:2.0:1787826234.990708:0:159739:0:(lov_obd.c:510:lov_add_target()) lustre-clilov-ffff9181c5620000: Adding target lustre-OST0000_UUID 00020000:01000000:2.0:1787826234.990712:0:159739:0:(lov_obd.c:532:lov_add_target()) idx=0 ltd_gen=1 ld_tgt_count=1 00000020:01000000:2.0:1787826234.990723:0:159739:0:(obd_config.c:1893:class_config_llog_handler()) Marker, inst_flg=0x2 mark_flg=0x2 00000020:00000080:2.0:1787826234.990726:0:159739:0:(obd_config.c:1472:class_process_config()) processing cmd: cf010 00000020:00000080:2.0:1787826234.990728:0:159739:0:(obd_config.c:1563:class_process_config()) marker 24 (0x2) lustre-OST0000 add osc 00000020:01000000:2.0:1787826234.990730:0:159739:0:(obd_config.c:1893:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x5 00000020:01000000:2.0:1787826234.990733:0:159739:0:(obd_config.c:1902:class_config_llog_handler()) SKIP #25 00000020:01000000:2.0:1787826234.990734:0:159739:0:(obd_config.c:1928:class_config_llog_handler()) skipping 0x6 00000020:01000000:2.0:1787826234.990736:0:159739:0:(obd_config.c:1928:class_config_llog_handler()) skipping 0x6 00000020:01000000:2.0:1787826234.990737:0:159739:0:(obd_config.c:1893:class_config_llog_handler()) Marker, inst_flg=0x6 mark_flg=0x6 00000020:00000080:2.0:1787826234.990739:0:159739:0:(obd_config.c:1472:class_process_config()) processing cmd: cf010 00000020:00000080:2.0:1787826234.990741:0:159739:0:(obd_config.c:1563:class_process_config()) marker 25 (0x6) lustre sys.timeout 00000020:01000000:2.0:1787826234.990761:0:159739:0:(obd_config.c:1893:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x1 00000020:01000004:2.0:1787826234.990764:0:159739:0:(obd_mount.c:961:lustre_check_exclusion()) Check exclusion lustre-OST0001 (1) in 0 of 192.168.203.147@tcp:/lustre 00000020:00000080:2.0:1787826234.990768:0:159739:0:(obd_config.c:1472:class_process_config()) processing cmd: cf010 00000020:00000080:2.0:1787826234.990769:0:159739:0:(obd_config.c:1563:class_process_config()) marker 32 (0x1) lustre-OST0001 add osc 00000020:00000080:2.0:1787826234.990773:0:159739:0:(obd_config.c:1472:class_process_config()) processing cmd: cf005 00000020:00000080:2.0:1787826234.990776:0:159739:0:(obd_config.c:1487:class_process_config()) adding mapping from uuid 192.168.203.147@tcp to nid 0x20000c0a8cb93 (192.168.203.147@tcp) 00000020:01000000:2.0:1787826234.990791:0:159739:0:(obd_config.c:2004:class_config_llog_handler()) cmd cf001, instance name: lustre-OST0001-osc-ffff9181c5620000 00000020:00000080:2.0:1787826234.990794:0:159739:0:(obd_config.c:1472:class_process_config()) processing cmd: cf001 00000020:00000080:2.0:1787826234.990800:0:159739:0:(genops.c:413:class_newdev()) Allocate new device lustre-OST0001-osc-ffff9181c5620000 (ffff9181c354b370) 00000020:00000080:2.0:1787826234.990810:0:159739:0:(obd_config.c:720:class_attach_name()) OBD: dev 6 attached type osc with refcount 2 00000020:01000000:2.0:1787826234.990815:0:159739:0:(obd_config.c:2004:class_config_llog_handler()) cmd cf003, instance name: lustre-OST0001-osc-ffff9181c5620000 00000020:00000080:2.0:1787826234.990817:0:159739:0:(obd_config.c:1472:class_process_config()) processing cmd: cf003 00010000:00080000:2.0:1787826234.990911:0:159739:0:(ldlm_lib.c:102:import_set_conn()) imp ffff9181c5bc6000@lustre-OST0001-osc-ffff9181c5620000: add connection 192.168.203.147@tcp at head 00010000:00010000:2.0:1787826234.991118:0:159739:0:(ldlm_pool.c:973:ldlm_pool_init()) Lock pool ldlm-pool-lustre-OST0001-osc-ffff9181c5620000-4 is initialized 00000020:00000080:2.0:1787826234.991235:0:159739:0:(obd_config.c:826:class_setup()) finished setup of obd lustre-OST0001-osc-ffff9181c5620000 (uuid c0d7758a-4501-4a89-8ec1-407fef4f0c7f) 00000020:01000000:2.0:1787826234.991241:0:159739:0:(obd_config.c:2004:class_config_llog_handler()) cmd cf00d, instance name: lustre-clilov-ffff9181c5620000 00000020:00000080:2.0:1787826234.991243:0:159739:0:(obd_config.c:1472:class_process_config()) processing cmd: cf00d 00020000:01000000:2.0:1787826234.991248:0:159739:0:(lov_obd.c:481:lov_add_target()) uuid:lustre-OST0001_UUID idx:1 gen:1 active:1 00020000:01000000:2.0:1787826234.991251:0:159739:0:(lov_obd.c:510:lov_add_target()) lustre-clilov-ffff9181c5620000: Adding target lustre-OST0001_UUID 00020000:01000000:2.0:1787826234.991253:0:159739:0:(lov_obd.c:532:lov_add_target()) idx=1 ltd_gen=1 ld_tgt_count=2 00000020:01000000:2.0:1787826234.991257:0:159739:0:(obd_config.c:1893:class_config_llog_handler()) Marker, inst_flg=0x2 mark_flg=0x2 00000020:00000080:2.0:1787826234.991259:0:159739:0:(obd_config.c:1472:class_process_config()) processing cmd: cf010 00000020:00000080:2.0:1787826234.991260:0:159739:0:(obd_config.c:1563:class_process_config()) marker 32 (0x2) lustre-OST0001 add osc 00000020:01000000:2.0:1787826234.991262:0:159739:0:(obd_config.c:1893:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x1 00000020:00000080:2.0:1787826234.991264:0:159739:0:(obd_config.c:1472:class_process_config()) processing cmd: cf010 00000020:00000080:2.0:1787826234.991265:0:159739:0:(obd_config.c:1563:class_process_config()) marker 33 (0x1) lustre sys.timeout 00000020:00000080:2.0:1787826234.991267:0:159739:0:(obd_config.c:1472:class_process_config()) processing cmd: cf009 00000020:00000080:2.0:1787826234.991269:0:159739:0:(obd_config.c:1537:class_process_config()) changing lustre timeout from 20 to 20 00000020:01000000:2.0:1787826234.991270:0:159739:0:(obd_config.c:1893:class_config_llog_handler()) Marker, inst_flg=0x2 mark_flg=0x2 00000020:00000080:2.0:1787826234.991272:0:159739:0:(obd_config.c:1472:class_process_config()) processing cmd: cf010 00000020:00000080:2.0:1787826234.991272:0:159739:0:(obd_config.c:1563:class_process_config()) marker 33 (0x2) lustre sys.timeout 00000100:00100000:2.0:1787826234.991307:0:159739:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f3b80 pname:cluuid:pid:xid:nid:opc:job llog_process_th:f40147eb-9b53-47d4-b77c-99ceedd9664c:159739:1874663042983936:192.168.203.147@tcp:502:llog_process_th.0 00000100:00100000:2.0:1787826234.991438:0:159739:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c65bb600 going to sleep for 11 seconds 00000100:00100000:2.0:1787826235.004388:0:159739:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f3b80 pname:cluuid:pid:xid:nid:opc:job llog_process_th:f40147eb-9b53-47d4-b77c-99ceedd9664c:159739:1874663042983936:192.168.203.147@tcp:502:llog_process_th.0 00000020:01000000:2.0:1787826235.004432:0:159739:0:(obd_config.c:1893:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x1 00000020:00000080:2.0:1787826235.004443:0:159739:0:(obd_config.c:1472:class_process_config()) processing cmd: cf010 00000020:00000080:2.0:1787826235.004445:0:159739:0:(obd_config.c:1563:class_process_config()) marker 38 (0x1) lustre sys.jobid_var 00000020:00000080:2.0:1787826235.004450:0:159739:0:(obd_config.c:1472:class_process_config()) processing cmd: cf00f 00000100:00100000:2.0:1787826235.004486:0:159739:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f3b80 pname:cluuid:pid:xid:nid:opc:job llog_process_th:f40147eb-9b53-47d4-b77c-99ceedd9664c:159739:1874663042984064:192.168.203.147@tcp:502:llog_process_th.0 00000100:00100000:2.0:1787826235.004578:0:159739:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c65bb600 going to sleep for 11 seconds 00000100:00100000:3.0:1787826235.011081:0:159739:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f3b80 pname:cluuid:pid:xid:nid:opc:job llog_process_th:f40147eb-9b53-47d4-b77c-99ceedd9664c:159739:1874663042984064:192.168.203.147@tcp:502:llog_process_th.0 00000020:01000000:3.0:1787826235.011133:0:159739:0:(obd_config.c:1893:class_config_llog_handler()) Marker, inst_flg=0x2 mark_flg=0x2 00000020:00000080:3.0:1787826235.011142:0:159739:0:(obd_config.c:1472:class_process_config()) processing cmd: cf010 00000020:00000080:3.0:1787826235.011144:0:159739:0:(obd_config.c:1563:class_process_config()) marker 38 (0x2) lustre sys.jobid_var 00000040:00080000:3.0:1787826235.011149:0:159739:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x3:0x0] index 52 count 51 00000020:01000000:2.0:1787826235.011382:0:159721:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-client gen 1-51 (rc=0) 10000000:01000000:2.0:1787826235.011399:0:159721:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-client' succeeded (0). 00010000:00010000:2.0:1787826235.011422:0:159721:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e06f6a00/0xbef42c0221d6f14e lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00ced6d8 expref: -99 pid: 159721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826235.011433:0:159721:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.147@tcp lock: ffff9181e06f6a00/0xbef42c0221d6f14e lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00ced6d8 expref: -99 pid: 159721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787826235.011441:0:159721:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff9181c5620000 from 1 10000000:01000000:2.0:1787826235.011444:0:159721:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:2.0:1787826235.011477:0:159721:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e06f6c00/0xbef42c0221d6f17f lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826235.011485:0:159721:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e06f6c00/0xbef42c0221d6f17f lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826235.011498:0:159721:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e06f6c00/0xbef42c0221d6f17f lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826235.011519:0:159721:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f2d80 pname:cluuid:pid:xid:nid:opc:job mount.lustre:f40147eb-9b53-47d4-b77c-99ceedd9664c:159721:1874663042984192:192.168.203.147@tcp:101:mount.lustre.0 00000100:00100000:2.0:1787826235.012030:0:159721:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c65bb600 going to sleep for 11 seconds 00000100:00100000:2.0:1787826235.030576:0:159721:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f2d80 pname:cluuid:pid:xid:nid:opc:job mount.lustre:f40147eb-9b53-47d4-b77c-99ceedd9664c:159721:1874663042984192:192.168.203.147@tcp:101:mount.lustre.0 00010000:00010000:2.0:1787826235.030622:0:159721:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.147@tcp lock: ffff9181e06f6c00/0xbef42c0221d6f17f lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00ced6df expref: -99 pid: 159721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826235.030635:0:159721:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.147@tcp lock: ffff9181e06f6c00/0xbef42c0221d6f17f lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x878a5eea00ced6df expref: -99 pid: 159721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826235.031210:0:159721:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f2d80 pname:cluuid:pid:xid:nid:opc:job mount.lustre:f40147eb-9b53-47d4-b77c-99ceedd9664c:159721:1874663042984320:192.168.203.147@tcp:256:mount.lustre.0 00000100:00100000:2.0:1787826235.031331:0:159721:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c65bb600 going to sleep for 11 seconds 00000100:00100000:2.0:1787826235.039573:0:159721:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f2d80 pname:cluuid:pid:xid:nid:opc:job mount.lustre:f40147eb-9b53-47d4-b77c-99ceedd9664c:159721:1874663042984320:192.168.203.147@tcp:256:mount.lustre.0 10000000:01000000:2.0:1787826235.040471:0:159721:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-cliir' succeeded (0). 00010000:00010000:2.0:1787826235.040551:0:159721:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e06f6c00/0xbef42c0221d6f17f lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00ced6df expref: -99 pid: 159721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826235.040700:0:159721:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.147@tcp lock: ffff9181e06f6c00/0xbef42c0221d6f17f lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00ced6df expref: -99 pid: 159721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787826235.040712:0:159721:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff9181c5620000 from 1 10000000:01000000:2.0:1787826235.040715:0:159721:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00010000:00010000:2.0:1787826235.040810:0:159721:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e06f6e00/0xbef42c0221d6f186 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826235.040819:0:159721:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e06f6e00/0xbef42c0221d6f186 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826235.040830:0:159721:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e06f6e00/0xbef42c0221d6f186 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826235.040852:0:159721:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f2d80 pname:cluuid:pid:xid:nid:opc:job mount.lustre:f40147eb-9b53-47d4-b77c-99ceedd9664c:159721:1874663042984448:192.168.203.147@tcp:101:mount.lustre.0 00000100:00100000:2.0:1787826235.040932:0:159721:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c65bb600 going to sleep for 11 seconds 00000100:00100000:3.0:1787826235.049132:0:159721:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f2d80 pname:cluuid:pid:xid:nid:opc:job mount.lustre:f40147eb-9b53-47d4-b77c-99ceedd9664c:159721:1874663042984448:192.168.203.147@tcp:101:mount.lustre.0 00010000:00010000:3.0:1787826235.049162:0:159721:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.147@tcp lock: ffff9181e06f6e00/0xbef42c0221d6f186 lrc: 4/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00ced6e6 expref: -99 pid: 159721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826235.049186:0:159721:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.147@tcp lock: ffff9181e06f6e00/0xbef42c0221d6f186 lrc: 4/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x878a5eea00ced6e6 expref: -99 pid: 159721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826235.049271:0:159721:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job mount.lustre:f40147eb-9b53-47d4-b77c-99ceedd9664c:159721:1874663042984576:192.168.203.147@tcp:501:mount.lustre.0 00000100:00100000:3.0:1787826235.049347:0:159721:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5cb99c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826235.056021:0:159721:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job mount.lustre:f40147eb-9b53-47d4-b77c-99ceedd9664c:159721:1874663042984576:192.168.203.147@tcp:501:mount.lustre.0 00000100:00100000:3.0:1787826235.056079:0:159721:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job mount.lustre:f40147eb-9b53-47d4-b77c-99ceedd9664c:159721:1874663042984704:192.168.203.147@tcp:503:mount.lustre.0 00000100:00100000:3.0:1787826235.056177:0:159721:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5cb99c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826235.060465:0:159721:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job mount.lustre:f40147eb-9b53-47d4-b77c-99ceedd9664c:159721:1874663042984704:192.168.203.147@tcp:503:mount.lustre.0 00000100:00100000:1.0:1787826235.061159:0:159745:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job llog_process_th:f40147eb-9b53-47d4-b77c-99ceedd9664c:159745:1874663042984832:192.168.203.147@tcp:502:llog_process_th.0 00000100:00100000:1.0:1787826235.061274:0:159745:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d225f780 going to sleep for 11 seconds 00000100:00100000:2.0:1787826235.065501:0:159745:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job llog_process_th:f40147eb-9b53-47d4-b77c-99ceedd9664c:159745:1874663042984832:192.168.203.147@tcp:502:llog_process_th.0 00000020:01000000:2.0:1787826235.065561:0:159745:0:(obd_config.c:1893:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x1 00000020:00000080:2.0:1787826235.065572:0:159745:0:(obd_config.c:1472:class_process_config()) processing cmd: cf010 00000020:00000080:2.0:1787826235.065574:0:159745:0:(obd_config.c:1563:class_process_config()) marker 2 (0x1) general debug_raw_pointers 00000020:01000000:2.0:1787826235.065580:0:159745:0:(obd_config.c:2004:class_config_llog_handler()) cmd ce032, instance name: general-ffff9181c5620000 00000020:00000080:2.0:1787826235.065583:0:159745:0:(obd_config.c:1472:class_process_config()) processing cmd: ce032 00000020:01000000:2.0:1787826235.065591:0:159745:0:(obd_config.c:1893:class_config_llog_handler()) Marker, inst_flg=0x2 mark_flg=0x2 00000020:00000080:2.0:1787826235.065594:0:159745:0:(obd_config.c:1472:class_process_config()) processing cmd: cf010 00000020:00000080:2.0:1787826235.065595:0:159745:0:(obd_config.c:1563:class_process_config()) marker 2 (0x2) general debug_raw_pointers 00000020:01000000:2.0:1787826235.065699:0:159745:0:(obd_config.c:1893:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x1 00000020:00000080:2.0:1787826235.065705:0:159745:0:(obd_config.c:1472:class_process_config()) processing cmd: cf010 00000020:00000080:2.0:1787826235.065707:0:159745:0:(obd_config.c:1563:class_process_config()) marker 3 (0x1) general lod.*.mdt_hash 00000020:01000000:2.0:1787826235.065712:0:159745:0:(obd_config.c:2004:class_config_llog_handler()) cmd ce032, instance name: general-ffff9181c5620000 00000020:00000080:2.0:1787826235.065714:0:159745:0:(obd_config.c:1472:class_process_config()) processing cmd: ce032 00000100:00100000:2.0:1787826235.065880:0:159745:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f3100 pname:cluuid:pid:xid:nid:opc:job llog_process_th:f40147eb-9b53-47d4-b77c-99ceedd9664c:159745:1874663042984960:192.168.203.147@tcp:502:llog_process_th.0 00000100:00100000:2.0:1787826235.065983:0:159745:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c65bb6c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826235.072395:0:159745:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f3100 pname:cluuid:pid:xid:nid:opc:job llog_process_th:f40147eb-9b53-47d4-b77c-99ceedd9664c:159745:1874663042984960:192.168.203.147@tcp:502:llog_process_th.0 00000020:01000000:2.0:1787826235.072447:0:159745:0:(obd_config.c:1893:class_config_llog_handler()) Marker, inst_flg=0x2 mark_flg=0x2 00000020:00000080:2.0:1787826235.072455:0:159745:0:(obd_config.c:1472:class_process_config()) processing cmd: cf010 00000020:00000080:2.0:1787826235.072457:0:159745:0:(obd_config.c:1563:class_process_config()) marker 3 (0x2) general lod.*.mdt_hash 00000040:00080000:2.0:1787826235.072461:0:159745:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x2:0x0] index 7 count 7 00000020:01000000:3.0:1787826235.073699:0:159721:0:(obd_config.c:2145:class_config_parse_llog()) Processed log params gen 1-6 (rc=0) 10000000:01000000:3.0:1787826235.073724:0:159721:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'params' succeeded (0). 00010000:00010000:3.0:1787826235.073774:0:159721:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e06f6e00/0xbef42c0221d6f186 lrc: 3/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00ced6e6 expref: -99 pid: 159721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826235.073787:0:159721:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.147@tcp lock: ffff9181e06f6e00/0xbef42c0221d6f186 lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00ced6e6 expref: -99 pid: 159721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000080:01000000:3.0:1787826235.073808:0:159721:0:(llite_lib.c:1614:ll_fill_super()) Found profile lustre-client: mdc=lustre-clilmv osc=lustre-clilov 00000020:00000080:3.0:1787826235.073835:0:159721:0:(genops.c:1385:class_connect()) connect: client c0d7758a-4501-4a89-8ec1-407fef4f0c7f, cookie 0xbef42c0221d6f18d 00800000:01000000:3.0:1787826235.073861:0:159721:0:(lmv_obd.c:515:lmv_check_connect()) Time to connect c0d7758a-4501-4a89-8ec1-407fef4f0c7f to lustre-clilmv-ffff9181c5620000 00800000:01000000:3.0:1787826235.073868:0:159721:0:(lmv_obd.c:338:lmv_connect_mdc()) connect to lustre-MDT0000-mdc-ffff9181c5620000(c0d7758a-4501-4a89-8ec1-407fef4f0c7f) - lustre-MDT0000_UUID, c0d7758a-4501-4a89-8ec1-407fef4f0c7f 00000020:00000080:3.0:1787826235.073878:0:159721:0:(genops.c:1385:class_connect()) connect: client c0d7758a-4501-4a89-8ec1-407fef4f0c7f, cookie 0xbef42c0221d6f194 00000100:00080000:3.0:1787826235.073888:0:159721:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 0 00000100:00080000:3.0:1787826235.073898:0:159721:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826235.073910:0:159721:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 134217728, lost: 0. 00000100:00100000:3.0:1787826235.073936:0:159721:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663042985088/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826235.073956:0:159721:0:(pinger.c:373:ptlrpc_pinger_add_import()) adding pingable import c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID 00800000:01000000:3.0:1787826235.074045:0:159721:0:(lmv_obd.c:392:lmv_connect_mdc()) Connected to lustre-MDT0000-mdc-ffff9181c5620000(c0d7758a-4501-4a89-8ec1-407fef4f0c7f) successfully (5) 00800000:00000004:3.0:1787826235.074049:0:159721:0:(obd_class.h:1056:obd_statfs_async()) lustre-MDT0000-mdc-ffff9181c5620000: age 7306, max_age 0 00800000:01000000:3.0:1787826235.074080:0:159721:0:(lmv_obd.c:338:lmv_connect_mdc()) connect to lustre-MDT0001-mdc-ffff9181c5620000(c0d7758a-4501-4a89-8ec1-407fef4f0c7f) - lustre-MDT0001_UUID, c0d7758a-4501-4a89-8ec1-407fef4f0c7f 00000020:00000080:3.0:1787826235.074088:0:159721:0:(genops.c:1385:class_connect()) connect: client c0d7758a-4501-4a89-8ec1-407fef4f0c7f, cookie 0xbef42c0221d6f19b 00000100:00080000:3.0:1787826235.074093:0:159721:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 0 00000100:00080000:3.0:1787826235.074098:0:159721:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826235.074104:0:159721:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 134217728, lost: 0. 00000100:00100000:3.0:1787826235.074114:0:159721:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663042985344/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826235.074124:0:159721:0:(pinger.c:373:ptlrpc_pinger_add_import()) adding pingable import c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID 00800000:01000000:3.0:1787826235.074165:0:159721:0:(lmv_obd.c:392:lmv_connect_mdc()) Connected to lustre-MDT0001-mdc-ffff9181c5620000(c0d7758a-4501-4a89-8ec1-407fef4f0c7f) successfully (5) 00800000:00000004:3.0:1787826235.074168:0:159721:0:(obd_class.h:1056:obd_statfs_async()) lustre-MDT0001-mdc-ffff9181c5620000: age 7306, max_age 0 00000080:00000004:3.0:1787826235.074190:0:159721:0:(obd_class.h:1090:obd_statfs()) lustre-clilmv-ffff9181c5620000: age 7306, max_age 8305 00800000:00000004:3.0:1787826235.074196:0:159721:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9181c5620000: age 7306, max_age 8305 00000100:00080000:3.0:1787826235.074214:0:159721:0:(client.c:1845:ptlrpc_send_new_req()) @@@ req waiting for recovery: (FULL != CONNECTING) req@ffff9181c5634e00 x1874663042985600/t0(0) o41->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 224/368 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'mount.lustre.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826235.074222:0:159721:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5cb9c00 going to sleep for 0 seconds 00000100:00080000:3.0:1787826235.074271:0:2391:0:(client.c:1845:ptlrpc_send_new_req()) @@@ req waiting for recovery: (FULL != CONNECTING) req@ffff9181c5634380 x1874663042985216/t0(0) o41->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 224/368 e 0 to 0 dl 0 ref 1 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'mount.lustre.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826235.074283:0:2391:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00080000:3.0:1787826235.074288:0:2391:0:(client.c:1845:ptlrpc_send_new_req()) @@@ req waiting for recovery: (FULL != CONNECTING) req@ffff9181c5636680 x1874663042985472/t0(0) o41->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 224/368 e 0 to 0 dl 0 ref 1 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'mount.lustre.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826235.078015:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663042985088:192.168.203.147@tcp:38:mount.lustre.0 00000100:00100000:2.0:1787826235.078124:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663042985344:192.168.203.147@tcp:38:mount.lustre.0 00000100:00100000:2.0:1787826235.086368:0:2388:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff9181c5620000: committing for last_committed 0 gen 1 00000100:00080000:2.0:1787826235.086385:0:2388:0:(import.c:1132:ptlrpc_connect_interpret()) lustre-MDT0001-mdc-ffff9181c5620000: connect to target with instance 4 00000100:00080000:2.0:1787826235.086394:0:2388:0:(import.c:956:ptlrpc_connect_set_flags()) lustre-MDT0001-mdc-ffff9181c5620000: Resetting ns_connect_flags to server flags: 0xae7a7e7be344d3b8 00000100:00080000:2.0:1787826235.086406:0:2388:0:(import.c:1213:ptlrpc_connect_interpret()) connected to replayable target: lustre-MDT0001_UUID 00000100:00080000:2.0:1787826235.086410:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to FULL 00000100:00080000:2.0:1787826235.092426:0:2388:0:(recover.c:191:ptlrpc_wake_delayed()) @@@ waking (set 000000003e6b408c): req@ffff9181c5636680 x1874663042985472/t0(0) o41->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 224/368 e 0 to 0 dl 0 ref 1 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'mount.lustre.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826235.092455:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663042985344:192.168.203.147@tcp:38:mount.lustre.0 00010000:00080000:2.0:1787826235.109719:0:2388:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@ffff9181c5634000 x1874663042985088/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/416 e 0 to 0 dl 1787826245 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'mount.lustre.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826235.109733:0:2388:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9181c5620000: committing for last_committed 0 gen 1 00000100:00080000:2.0:1787826235.109757:0:2388:0:(import.c:1132:ptlrpc_connect_interpret()) lustre-MDT0000-mdc-ffff9181c5620000: connect to target with instance 81 00000100:00080000:2.0:1787826235.109767:0:2388:0:(import.c:956:ptlrpc_connect_set_flags()) lustre-MDT0000-mdc-ffff9181c5620000: Resetting ns_connect_flags to server flags: 0xae7a7e7be344d3b8 00000100:00080000:2.0:1787826235.109779:0:2388:0:(import.c:1213:ptlrpc_connect_interpret()) connected to replayable target: lustre-MDT0000_UUID 00000100:00080000:2.0:1787826235.109783:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to FULL 00000100:00080000:2.0:1787826235.111644:0:2388:0:(recover.c:191:ptlrpc_wake_delayed()) @@@ waking (set 0000000029b09a7a): req@ffff9181c5634e00 x1874663042985600/t0(0) o41->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 224/368 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'mount.lustre.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826235.111674:0:2388:0:(recover.c:191:ptlrpc_wake_delayed()) @@@ waking (set 000000003e6b408c): req@ffff9181c5634380 x1874663042985216/t0(0) o41->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 224/368 e 0 to 0 dl 0 ref 1 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'mount.lustre.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826235.111698:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663042985088:192.168.203.147@tcp:38:mount.lustre.0 00000100:00100000:1.0:1787826235.114515:0:159721:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5cb9c00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826235.117944:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:0:1874663042985472:192.168.203.147@tcp:41:mount.lustre.0 00010000:00080000:2.0:1787826235.125037:0:2391:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@ffff9181c5634380 x1874663042985216/t0(0) o41->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 224/368 e 0 to 0 dl 1787826251 ref 1 fl Rpc:RQU/200/0 rc 0/0 job:'mount.lustre.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826235.125062:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:0:1874663042985216:192.168.203.147@tcp:41:mount.lustre.0 00010000:00080000:3.0:1787826235.125364:0:159721:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@ffff9181c5634e00 x1874663042985600/t0(0) o41->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 224/368 e 0 to 0 dl 1787826251 ref 2 fl Rpc:RQU/200/0 rc 0/0 job:'mount.lustre.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826235.125376:0:159721:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job mount.lustre:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:159721:1874663042985600:192.168.203.147@tcp:41:mount.lustre.0 00800000:00000004:3.0:1787826235.125391:0:159721:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0000-mdc-ffff9181c5620000: update ffff9181c354d178 cache blocks 317231/353529 objects 1023646/1024000 00000080:00000004:3.0:1787826235.125398:0:159721:0:(obd_class.h:1116:obd_statfs()) lustre-clilmv-ffff9181c5620000: update ffff9181d1a19e08 cache blocks 317231/353529 objects 1023646/1024000 00000100:00100000:3.0:1787826235.125438:0:159721:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job mount.lustre:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:159721:1874663042985728:192.168.203.147@tcp:40:mount.lustre.0 00000100:00100000:3.0:1787826235.125514:0:159721:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5cb96c0 going to sleep for 11 seconds 00010000:00080000:3.0:1787826235.130103:0:159721:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@ffff9181c5634e00 x1874663042985728/t0(0) o40->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 440/448 e 0 to 0 dl 1787826251 ref 2 fl Rpc:RQU/209/0 rc 0/0 job:'mount.lustre.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826235.130124:0:159721:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job mount.lustre:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:159721:1874663042985728:192.168.203.147@tcp:40:mount.lustre.0 00000080:00000004:3.0:1787826235.130148:0:159721:0:(llite_lib.c:614:client_common_fill_super()) rootfid [0x200000007:0x1:0x0] 00000080:00100000:3.0:1787826235.130162:0:159721:0:(llite_lib.c:775:client_common_fill_super()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683216 ocd_grant: 2146304 00020000:01000000:3.0:1787826235.130167:0:159721:0:(lov_obd.c:186:lov_connect()) connect #0 00000020:00000080:3.0:1787826235.130180:0:159721:0:(genops.c:1385:class_connect()) connect: client c0d7758a-4501-4a89-8ec1-407fef4f0c7f, cookie 0xbef42c0221d6f1a2 00000020:00000080:3.0:1787826235.130197:0:159721:0:(genops.c:1385:class_connect()) connect: client LOV_OSC_UUID, cookie 0xbef42c0221d6f1a9 00000100:00080000:3.0:1787826235.130203:0:159721:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 0 00000100:00080000:3.0:1787826235.130211:0:159721:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9181c5620000: import ffff9181c5bc3800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826235.130222:0:159721:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683216 ocd_grant: 134217728, lost: 0. 00000100:00100000:3.0:1787826235.130319:0:159721:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634e00 x1874663042985856/t0(0) o8->lustre-OST0000-osc-ffff9181c5620000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826235.130336:0:159721:0:(pinger.c:373:ptlrpc_pinger_add_import()) adding pingable import c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-OST0000_UUID 00000100:00100000:2.0:1787826235.130434:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663042985856:192.168.203.147@tcp:8:mount.lustre.0 00020000:01000000:3.0:1787826235.130498:0:159721:0:(lov_obd.c:159:lov_connect_osc()) Connected tgt idx 0 lustre-OST0000_UUID (lustre-OST0000-osc-ffff9181c5620000) active 00000020:00000080:3.0:1787826235.130525:0:159721:0:(genops.c:1385:class_connect()) connect: client LOV_OSC_UUID, cookie 0xbef42c0221d6f1b0 00000100:00080000:3.0:1787826235.130530:0:159721:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 0 00000100:00080000:3.0:1787826235.130535:0:159721:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9181c5620000: import ffff9181c5bc6000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826235.130549:0:159721:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683216 ocd_grant: 134217728, lost: 0. 00000100:00100000:3.0:1787826235.130562:0:159721:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663042985984/t0(0) o8->lustre-OST0001-osc-ffff9181c5620000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826235.130595:0:159721:0:(pinger.c:373:ptlrpc_pinger_add_import()) adding pingable import c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-OST0001_UUID 00020000:01000000:3.0:1787826235.131062:0:159721:0:(lov_obd.c:159:lov_connect_osc()) Connected tgt idx 1 lustre-OST0001_UUID (lustre-OST0001-osc-ffff9181c5620000) active 00000100:00100000:2.0:1787826235.131064:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663042985984:192.168.203.147@tcp:8:mount.lustre.0 00000100:00100000:3.0:1787826235.131101:0:159721:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job mount.lustre:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:159721:1874663042986112:192.168.203.147@tcp:900:mount.lustre.0 00000100:00100000:3.0:1787826235.131128:0:159721:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5cb96c0 going to sleep for 11 seconds 00010000:00080000:3.0:1787826235.136026:0:159721:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@ffff9181c5637100 x1874663042986112/t0(0) o900->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:29/10 lens 264/248 e 0 to 0 dl 1787826251 ref 2 fl Rpc:RQU/200/0 rc 0/0 job:'mount.lustre.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826235.136045:0:159721:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job mount.lustre:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:159721:1874663042986112:192.168.203.147@tcp:900:mount.lustre.0 00000100:00100000:3.0:1787826235.136105:0:159721:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job mount.lustre:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:159721:1874663042986240:192.168.203.147@tcp:33:mount.lustre.0 00000100:00100000:3.0:1787826235.136214:0:159721:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5cb96c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826235.136268:0:2388:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-ffff9181c5620000: committing for last_committed 0 gen 1 00000100:00080000:2.0:1787826235.136276:0:2388:0:(import.c:1132:ptlrpc_connect_interpret()) lustre-OST0001-osc-ffff9181c5620000: connect to target with instance 1 00000100:00080000:2.0:1787826235.136286:0:2388:0:(import.c:956:ptlrpc_connect_set_flags()) lustre-OST0001-osc-ffff9181c5620000: Resetting ns_connect_flags to server flags: 0xa0427af2e3440038 00000100:00080000:2.0:1787826235.136297:0:2388:0:(import.c:1213:ptlrpc_connect_interpret()) connected to replayable target: lustre-OST0001_UUID 00000100:00080000:2.0:1787826235.136301:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc6000 lustre-OST0001_UUID: changing import state from CONNECTING to FULL 00020000:01000000:2.0:1787826235.136309:0:2388:0:(lov_obd.c:398:lov_set_osc_active()) Marking OSC lustre-OST0001_UUID active 00000080:00000004:2.0:1787826235.136314:0:2388:0:(lcommon_misc.c:94:cl_ocd_update()) Changing connect_flags: 0xa0467af2e3640038 -> 0xa0427af2e3440038 00000080:00080000:2.0:1787826235.136319:0:2388:0:(lcommon_misc.c:56:cl_init_ea_size()) updating def/max_easize: 72/65536 00000100:00100000:2.0:1787826235.136329:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663042985984:192.168.203.147@tcp:8:mount.lustre.0 00000100:00080000:2.0:1787826235.136342:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc3800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826235.136346:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-19) 00000100:00100000:2.0:1787826235.136349:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663042985856:192.168.203.147@tcp:8:mount.lustre.0 00010000:00080000:3.0:1787826235.158526:0:159721:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@ffff9181c5637100 x1874663042986240/t0(0) o33->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 448/464 e 0 to 0 dl 1787826251 ref 2 fl Rpc:RQU/600/0 rc 0/0 job:'mount.lustre.0' uid:0 gid:0 projid:0 00000100:00100000:3.0:1787826235.158556:0:159721:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job mount.lustre:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:159721:1874663042986240:192.168.203.147@tcp:33:mount.lustre.0 00000080:00200000:3.0:1787826235.158729:0:159721:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00200000:3.0:1787826235.158738:0:159721:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 69632 00000080:02000400:3.0:1787826235.158838:0:159721:0:(llite_lib.c:1653:ll_fill_super()) Mounted lustre-client - version 2.17.57_80_gdb9f866 00000080:00000004:3.0:1787826235.167261:0:159721:0:(super25.c:182:lustre_fill_super()) 192.168.203.147@tcp:/lustre: Mount complete 00010000:00010000:3.0:1787826235.937604:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826236.961134:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826237.985039:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826239.008944:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826240.076360:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787826240.545066:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 0 suppress 1 00000100:00080000:3.0:1787826240.545075:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1787826240.545080:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc3800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826240.545094:0:99984:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8306 00000100:00080000:3.0:1787826240.545108:0:99984:0:(import.c:589:import_select_connection()) lustre-OST0000-osc-ffff9181c5620000: tried all connections, increasing latency to 5s 00000100:00080000:3.0:1787826240.545116:0:99984:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9181c5620000: import ffff9181c5bc3800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826240.545139:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683216 ocd_grant: 134217728, lost: 0. 00000100:00100000:3.0:1787826240.545161:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634700 x1874663042986752/t0(0) o8->lustre-OST0000-osc-ffff9181c5620000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826240.545251:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663042986496:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826240.545365:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663042986880:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826240.545405:0:2392:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1787826240.545412:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:f40147eb-9b53-47d4-b77c-99ceedd9664c:2392:1874663042986368:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826240.545448:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663042986624:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826240.547535:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663042986752:192.168.203.147@tcp:8:kworker.0 00000100:00100000:3.0:1787826240.561868:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:f40147eb-9b53-47d4-b77c-99ceedd9664c:2392:1874663042986368:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826240.561896:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663042986624:192.168.203.147@tcp:400:kworker.0 00010000:00080000:3.0:1787826240.561911:0:2392:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@ffff9181c5635f80 x1874663042986496/t0(0) o400->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 224/224 e 0 to 0 dl 1787826256 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826240.561922:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663042986496:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826240.561934:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663042986880:192.168.203.147@tcp:400:kworker.0 00000100:00080000:2.0:1787826240.567779:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc3800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826240.567797:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-19) 00000100:00100000:2.0:1787826240.567803:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663042986752:192.168.203.147@tcp:8:kworker.0 00010000:00010000:1.0:1787826241.128076:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826242.148266:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826242.640592:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826242.640661:0:159722:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049341056 00000100:00100000:0.0:1787826242.640676:0:159722:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049341056 00000100:00100000:0.0:1787826242.640739:0:159722:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 0 00000100:00100000:0.0:1787826242.640810:0:159722:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a4000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:f40147eb-9b53-47d4-b77c-99ceedd9664c+3:193935:x1874663049341056:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826242.640834:0:159722:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.147@tcp lock: ffff9181e06f6c00/0xbef42c0221d6f17f lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1400000000000 nid: local remote: 0x878a5eea00ced6df expref: -99 pid: 159721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826242.641044:0:159722:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826242.641053:0:159722:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a4000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:f40147eb-9b53-47d4-b77c-99ceedd9664c+3:193935:x1874663049341056:12345-192.168.203.147@tcp:104: Request processed in 248us (474us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826242.641061:0:159722:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 0 00010000:00010000:3.0:1787826242.641563:0:159732:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826242.641573:0:159732:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e06f6c00/0xbef42c0221d6f17f lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1400000000000 nid: local remote: 0x878a5eea00ced6df expref: -99 pid: 159721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826242.641582:0:159732:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e06f6c00 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:3.0:1787826242.641588:0:159732:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e06f6c00/0xbef42c0221d6f17f lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1400400000000 nid: local remote: 0x878a5eea00ced6df expref: -99 pid: 159721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826242.641597:0:159732:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e06f6c00/0xbef42c0221d6f17f lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1408400000000 nid: local remote: 0x878a5eea00ced6df expref: -99 pid: 159721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1787826242.641610:0:159732:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e06f6c00/0xbef42c0221d6f17f lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1409400000000 nid: local remote: 0x878a5eea00ced6df expref: -99 pid: 159721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787826242.641615:0:159732:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:3.0:1787826242.641673:0:159732:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.147@tcp lock: ffff9181e06f6c00/0xbef42c0221d6f17f lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0x878a5eea00ced6df expref: -99 pid: 159721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826242.641680:0:159732:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826242.641714:0:159732:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e06f6c00/0xbef42c0221d6f17f lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0x878a5eea00ced6df expref: -99 pid: 159721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826242.641721:0:159732:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e06f6c00/0xbef42c0221d6f17f lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0x878a5eea00ced6df expref: -99 pid: 159721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826242.641785:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:f40147eb-9b53-47d4-b77c-99ceedd9664c:2391:1874663042987008:192.168.203.147@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826242.641800:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826242.641804:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826242.641806:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826242.648070:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:f40147eb-9b53-47d4-b77c-99ceedd9664c:2391:1874663042987008:192.168.203.147@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826243.171803:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787826244.193070:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:1.0:1787826244.193079:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff9181c5620000 from 335 10000000:01000000:1.0:1787826244.193086:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:1.0:1787826244.193139:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4600/0xbef42c0221d6f1b7 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826244.193152:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b4600/0xbef42c0221d6f1b7 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826244.193163:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b4600/0xbef42c0221d6f1b7 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826244.193194:0:159735:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663042987136:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:3.0:1787826244.193286:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826244.193293:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d225f180 going to sleep for 11 seconds 00000100:00100000:1.0:1787826244.196597:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663042987136:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787826244.196629:0:159735:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.147@tcp lock: ffff9181e01b4600/0xbef42c0221d6f1b7 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00ced7a3 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826244.196642:0:159735:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.147@tcp lock: ffff9181e01b4600/0xbef42c0221d6f1b7 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x878a5eea00ced7a3 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826244.196724:0:159735:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663042987264:192.168.203.147@tcp:256:ll_cfg_requeue.0 00000100:00100000:1.0:1787826244.196846:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d225f180 going to sleep for 11 seconds 00000100:00100000:1.0:1787826244.204969:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663042987264:192.168.203.147@tcp:256:ll_cfg_requeue.0 00000100:00080000:1.0:1787826244.205040:0:159735:0:(import.c:214:ptlrpc_set_import_discon()) osc: import ffff9181c5bc3800 already not connected (conn 2, was 0): DISCONN 00010000:00080000:1.0:1787826244.205057:0:159735:0:(ldlm_lib.c:85:import_set_conn()) imp ffff9181c5bc3800@lustre-OST0000-osc-ffff9181c5620000: found existing conn 192.168.203.147@tcp, moved to head 00000100:00080000:1.0:1787826244.205063:0:159735:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc3800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826244.205068:0:159735:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 0 00000100:00080000:1.0:1787826244.205077:0:159735:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9181c5620000: import ffff9181c5bc3800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787826244.205090:0:159735:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683216 ocd_grant: 134217728, lost: 0. 00000100:00100000:1.0:1787826244.205142:0:159735:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0a80 x1874663042987392/t0(0) o8->lustre-OST0000-osc-ffff9181c5620000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 10000000:01000000:1.0:1787826244.205211:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-cliir' succeeded (0). 00010000:00010000:1.0:1787826244.205225:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4600/0xbef42c0221d6f1b7 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00ced7a3 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826244.205236:0:159735:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.147@tcp lock: ffff9181e01b4600/0xbef42c0221d6f1b7 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00ced7a3 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826244.207052:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663042987392:192.168.203.147@tcp:8:ll_cfg_requeue.0 00000100:02020000:2.0:1787826244.228901:0:2388:0:(client.c:1505:ptlrpc_check_status()) lustre-OST0000-osc-ffff9181c5620000: operation ost_connect to node 192.168.203.147@tcp failed: rc = -16 00000100:00100000:2.0:1787826244.250347:0:2388:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff9181c5620000: committing for last_committed 0 gen 1 00000100:00080000:2.0:1787826244.251988:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc3800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826244.252005:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-16) 00000100:00100000:2.0:1787826244.252011:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663042987392:192.168.203.147@tcp:8:ll_cfg_requeue.0 00010000:00010000:1.0:1787826245.219602:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787826245.672108:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 0 suppress 1 00000100:00080000:0.0:1787826245.672116:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787826245.672120:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc3800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826245.672138:0:99984:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9181c5620000: import ffff9181c5bc3800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787826245.672151:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683216 ocd_grant: 134217728, lost: 0. 00000100:00100000:0.0:1787826245.672172:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4e00 x1874663042987904/t0(0) o8->lustre-OST0000-osc-ffff9181c5620000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826245.672259:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663042987648:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826245.672388:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663042988032:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826245.672423:0:2390:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1787826245.672429:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:f40147eb-9b53-47d4-b77c-99ceedd9664c:2390:1874663042987520:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826245.672459:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663042987904:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1787826245.672461:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663042987776:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826245.676862:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:f40147eb-9b53-47d4-b77c-99ceedd9664c:2390:1874663042987520:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826245.685616:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663042987776:192.168.203.147@tcp:400:kworker.0 00010000:00080000:0.0:1787826245.685640:0:2390:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffff9181c55a5180 x1874663042987648/t0(0) o400->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 224/224 e 0 to 0 dl 1787826261 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826245.685653:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663042987648:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826245.686127:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663042988032:192.168.203.147@tcp:400:kworker.0 00000100:02020000:3.0:1787826245.713846:0:2388:0:(client.c:1505:ptlrpc_check_status()) lustre-OST0000-osc-ffff9181c5620000: operation ost_connect to node 192.168.203.147@tcp failed: rc = -16 00000100:00080000:3.0:1787826245.713867:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc3800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826245.713874:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.147@tcp failed (-16) 00000100:00100000:3.0:1787826245.713879:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663042987904:192.168.203.147@tcp:8:kworker.0 00010000:00010000:3.0:1787826246.241334:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826247.265222:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826248.295288:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826249.313056:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826250.338552:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826250.785245:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:f40147eb-9b53-47d4-b77c-99ceedd9664c:2389:1874663042988160:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826250.787597:0:2389:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1787826250.787623:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663042988288:192.168.203.147@tcp:400:kworker.0 00000100:00080000:0.0:1787826250.788582:0:152015:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 0 suppress 1 00000100:00080000:0.0:1787826250.788588:0:152015:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1787826250.788591:0:152015:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc3800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826250.788604:0:152015:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9181c5620000: import ffff9181c5bc3800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787826250.788616:0:152015:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683216 ocd_grant: 134217728, lost: 0. 00000100:00100000:0.0:1787826250.788632:0:152015:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7800 x1874663042988544/t0(0) o8->lustre-OST0000-osc-ffff9181c5620000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826250.788686:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663042988672:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826250.788740:0:2390:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1787826250.788767:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663042988416:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826250.790982:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663042988544:192.168.203.147@tcp:8:kworker.0 00000100:00100000:0.0:1787826250.802259:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663042988416:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826250.802289:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663042988672:192.168.203.147@tcp:400:kworker.0 00000100:00080000:3.0:1787826250.802311:0:2388:0:(import.c:1132:ptlrpc_connect_interpret()) lustre-OST0000-osc-ffff9181c5620000: connect to target with instance 9 00000100:00080000:3.0:1787826250.802329:0:2388:0:(import.c:956:ptlrpc_connect_set_flags()) lustre-OST0000-osc-ffff9181c5620000: Resetting ns_connect_flags to server flags: 0xa0427af2e3440038 00000100:00080000:3.0:1787826250.802347:0:2388:0:(import.c:1213:ptlrpc_connect_interpret()) connected to replayable target: lustre-OST0000_UUID 00000100:00080000:3.0:1787826250.802353:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc3800 lustre-OST0000_UUID: changing import state from CONNECTING to FULL 00020000:01000000:3.0:1787826250.802361:0:2388:0:(lov_obd.c:398:lov_set_osc_active()) Marking OSC lustre-OST0000_UUID active 00000080:00000004:3.0:1787826250.802366:0:2388:0:(lcommon_misc.c:94:cl_ocd_update()) Changing connect_flags: 0xa0427af2e3440038 -> 0xa0427af2e3440038 00000080:00080000:3.0:1787826250.802374:0:2388:0:(lcommon_misc.c:56:cl_init_ea_size()) updating def/max_easize: 72/65536 00000100:00100000:3.0:1787826250.802390:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663042988544:192.168.203.147@tcp:8:kworker.0 00010000:00080000:1.0:1787826250.802493:0:2389:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffff9181c55a5180 x1874663042988288/t0(0) o400->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 224/224 e 0 to 0 dl 1787826266 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787826250.802507:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663042988288:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826250.802519:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:f40147eb-9b53-47d4-b77c-99ceedd9664c:2389:1874663042988160:192.168.203.147@tcp:400:kworker.0 00000080:00200000:1.0:1787826251.329627:0:159931:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:1.0:1787826251.329661:0:159931:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826251.329717:0:159931:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5e00/0xbef42c0221d6f1be lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 159931 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826251.329729:0:159931:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5e00/0xbef42c0221d6f1be lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 159931 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826251.329740:0:159931:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5e00/0xbef42c0221d6f1be lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 159931 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826251.329997:0:159931:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job touch:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:159931:1874663042988800:192.168.203.147@tcp:101:touch.0 00000100:00100000:1.0:1787826251.330113:0:159931:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d225f180 going to sleep for 11 seconds 00000100:00100000:1.0:1787826251.333009:0:159931:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job touch:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:159931:1874663042988800:192.168.203.147@tcp:101:touch.0 00010000:00010000:1.0:1787826251.333038:0:159931:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5e00/0xbef42c0221d6f1be lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced898 expref: -99 pid: 159931 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826251.333051:0:159931:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5e00/0xbef42c0221d6f1be lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced898 expref: -99 pid: 159931 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826251.333064:0:159931:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5e00/0xbef42c0221d6f1be lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced898 expref: -99 pid: 159931 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826251.333077:0:159931:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5e00/0xbef42c0221d6f1be lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced898 expref: -99 pid: 159931 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826251.333096:0:159931:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5e00/0xbef42c0221d6f1be lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced898 expref: -99 pid: 159931 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826251.333105:0:159931:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5e00/0xbef42c0221d6f1be lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced898 expref: -99 pid: 159931 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826251.333112:0:159931:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5e00/0xbef42c0221d6f1be lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced898 expref: -99 pid: 159931 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826251.333122:0:159931:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c35b3480 x1874663042988800/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826267 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'touch.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826251.333135:0:159931:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5e00/0xbef42c0221d6f1be lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced898 expref: -99 pid: 159931 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826251.333146:0:159931:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d1a77c00 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:1.0:1787826251.333152:0:159931:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787826251.333178:0:159931:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 69632 00000080:00010000:1.0:1787826251.333182:0:159931:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f1be 00000080:00010000:1.0:1787826251.333187:0:159931:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:1.0:1787826251.333190:0:159931:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f1be from it ffffb9750303bbf8 00010000:00010000:1.0:1787826251.333193:0:159931:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5e00/0xbef42c0221d6f1be lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced898 expref: -99 pid: 159931 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826251.335741:0:159931:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5e00/0xbef42c0221d6f1be lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced898 expref: -99 pid: 159931 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826251.335931:0:159931:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826251.336112:0:159931:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:1.0:1787826251.336135:0:159931:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826251.336151:0:159931:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5e00/0xbef42c0221d6f1be lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced898 expref: -99 pid: 159931 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826251.336165:0:159931:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5e00/0xbef42c0221d6f1be lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced898 expref: -99 pid: 159931 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826251.336172:0:159931:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826251.336179:0:159931:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:1.0:1787826251.336182:0:159931:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f1be 00000080:00010000:1.0:1787826251.336188:0:159931:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f1be from it ffffb9750303bb38 00010000:00010000:1.0:1787826251.336192:0:159931:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5e00/0xbef42c0221d6f1be lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced898 expref: -99 pid: 159931 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826251.336200:0:159931:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5e00/0xbef42c0221d6f1be lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced898 expref: -99 pid: 159931 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 1 00000080:00200000:1.0:1787826251.336209:0:159931:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 3 00000080:00200000:1.0:1787826251.336214:0:159931:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f74.replay-single, dir=[0x200000007:0x1:0x0](ffff9181d080cb08), file ffff9181c39dcf00, open_flags 8941, mode 81b6 opened 0 00000080:00200000:1.0:1787826251.336223:0:159931:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f74.replay-single, dir=[0x200000007:0x1:0x0](ffff9181d080cb08), intent=open|creat 00000100:00100000:1.0:1787826251.336283:0:159931:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job touch:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:159931:1874663042988928:192.168.203.147@tcp:700:touch.0 00000100:00100000:1.0:1787826251.336417:0:159931:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d225f180 going to sleep for 11 seconds 00000100:00100000:1.0:1787826251.345200:0:159931:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job touch:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:159931:1874663042988928:192.168.203.147@tcp:700:touch.0 00000002:00010000:1.0:1787826251.345241:0:159931:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f74.replay-single,[0x200027cd1:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0104502 00010000:00010000:1.0:1787826251.345261:0:159931:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5e00/0xbef42c0221d6f1be lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ced898 expref: -99 pid: 159931 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 1 00010000:00010000:1.0:1787826251.345320:0:159931:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5e00/0xbef42c0221d6f1be lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ced898 expref: -99 pid: 159931 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 1 00010000:00010000:1.0:1787826251.345327:0:159931:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826251.345330:0:159931:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5e00/0xbef42c0221d6f1be lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ced898 expref: -99 pid: 159931 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 1 00010000:00010000:1.0:1787826251.345366:0:159931:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4400/0xbef42c0221d6f1c5 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 159931 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826251.345374:0:159931:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4400/0xbef42c0221d6f1c5 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 159931 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:1.0:1787826251.345385:0:159931:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:1.0:1787826251.345392:0:159931:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c35b3480 x1874663042989184/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:1.0:1787826251.345401:0:159931:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4400/0xbef42c0221d6f1c5 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 159931 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826251.345420:0:159931:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job touch:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:159931:1874663042989184:192.168.203.147@tcp:101:touch.0 00000100:00100000:1.0:1787826251.345551:0:159931:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d225f180 going to sleep for 11 seconds 00000100:00100000:1.0:1787826251.352083:0:159931:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job touch:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:159931:1874663042989184:192.168.203.147@tcp:101:touch.0 00010000:00010000:1.0:1787826251.352113:0:159931:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4400/0xbef42c0221d6f1c5 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced8b4 expref: -99 pid: 159931 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826251.352154:0:159931:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4400/0xbef42c0221d6f1c5 lrc: 4/0,1 mode: --/PR res: [0x200027cd1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced8b4 expref: -99 pid: 159931 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826251.352167:0:159931:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4400/0xbef42c0221d6f1c5 lrc: 4/0,1 mode: --/PR res: [0x200027cd1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced8b4 expref: -99 pid: 159931 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826251.352176:0:159931:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4400/0xbef42c0221d6f1c5 lrc: 4/0,1 mode: PR/PR res: [0x200027cd1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced8b4 expref: -99 pid: 159931 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826251.352186:0:159931:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4400/0xbef42c0221d6f1c5 lrc: 4/0,1 mode: PR/PR res: [0x200027cd1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced8b4 expref: -99 pid: 159931 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826251.352199:0:159931:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4400/0xbef42c0221d6f1c5 lrc: 5/1,1 mode: PR/PR res: [0x200027cd1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced8b4 expref: -99 pid: 159931 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826251.352206:0:159931:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4400/0xbef42c0221d6f1c5 lrc: 5/1,1 mode: PR/PR res: [0x200027cd1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced8b4 expref: -99 pid: 159931 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826251.352213:0:159931:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4400/0xbef42c0221d6f1c5 lrc: 4/1,0 mode: PR/PR res: [0x200027cd1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced8b4 expref: -99 pid: 159931 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826251.352224:0:159931:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9181c35b3480 x1874663042989184/t347892350977(347892350977) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826267 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'touch.0' uid:0 gid:0 projid:0 00000002:00100000:1.0:1787826251.352235:0:159931:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c35b3480 x1874663042989184/t347892350977(347892350977) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 664/664 e 0 to 0 dl 1787826267 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'touch.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826251.352248:0:159931:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4400/0xbef42c0221d6f1c5 lrc: 3/1,0 mode: PR/PR res: [0x200027cd1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced8b4 expref: -99 pid: 159931 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:1.0:1787826251.352261:0:159931:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4400/0xbef42c0221d6f1c5 lrc: 3/1,0 mode: PR/PR res: [0x200027cd1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced8b4 expref: -99 pid: 159931 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826251.352270:0:159931:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d1a77c00 type 13 mode 4 res 8590097617/1 (0 0) 00000080:00200000:1.0:1787826251.352275:0:159931:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1787826251.352289:0:159931:0:(namei.c:1628:ll_create_it()) VFS Op:name=f74.replay-single, dir=[0x200000007:0x1:0x0](ffff9181d080cb08), intent=open|creat 00000080:00200000:1.0:1787826251.352315:0:159931:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200027cd1:0x1:0x0](ffff9181c83e7448) 00000080:00200000:1.0:1787826251.352418:0:159931:0:(file.c:6843:ll_layout_conf()) [0x200027cd1:0x1:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:1.0:1787826251.352422:0:159931:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x200027cd1:0x1:0x0](ffff9181c83e7448) 00000080:00010000:1.0:1787826251.352426:0:159931:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200027cd1:0x1:0x0](ffff9181c83e7448) for lock 0xbef42c0221d6f1c5 00000080:00010000:1.0:1787826251.352446:0:159931:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200027cd1:0x1:0x0](ffff9181c83e7448) 00000080:00200008:1.0:1787826251.352461:0:159931:0:(file.c:1090:ll_file_open()) START file f74.replay-single:[0x200027cd1:0x1:0x0](ffff9181c83e7448), flags 104501 00000100:00100000:1.0:1787826251.352508:0:159931:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job touch:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:159931:1874663042989312:192.168.203.147@tcp:900:touch.0 00000100:00100000:1.0:1787826251.352595:0:159931:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d225fe40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826251.356521:0:159931:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job touch:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:159931:1874663042989312:192.168.203.147@tcp:900:touch.0 00000002:00100000:1.0:1787826251.356571:0:159931:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c35b3480 x1874663042989184/t347892350977(347892350977) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826267 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'touch.0' uid:0 gid:0 projid:0 00000080:00010000:1.0:1787826251.356613:0:159931:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f1c5 from it ffff9181d1d8ac60 00010000:00010000:1.0:1787826251.356622:0:159931:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4400/0xbef42c0221d6f1c5 lrc: 3/1,0 mode: PR/PR res: [0x200027cd1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00ced8b4 expref: -99 pid: 159931 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826251.356634:0:159931:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4400/0xbef42c0221d6f1c5 lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00ced8b4 expref: -99 pid: 159931 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826251.356665:0:159931:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x1:0x0](ffff9181c83e7448), inode mode 81a4 mask 40 00000080:00200008:1.0:1787826251.356705:0:159931:0:(llite_lib.c:2427:ll_setattr_raw()) START file f74.replay-single:[0x200027cd1:0x1:0x0](ffff9181c83e7448) current size 0, valid attrs 20070, mode 9181, uid 50576976, gid -18059, new size 18446744071751268887, atime 1787826251.000000000, mtime 1787826251.000000000, ctime 1787826251.000000000 00010000:00010000:1.0:1787826251.356729:0:159931:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4400/0xbef42c0221d6f1c5 lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x878a5eea00ced8b4 expref: -99 pid: 159931 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826251.356793:0:159931:0:(vvp_object.c:117:vvp_conf_set()) [0x200027cd1:0x1:0x0]: losing layout lock 00010000:00010000:1.0:1787826251.356833:0:159931:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4400/0xbef42c0221d6f1c5 lrc: 1/0,0 mode: --/PR res: [0x200027cd1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00ced8b4 expref: -99 pid: 159931 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826251.356840:0:159931:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826251.356843:0:159931:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4400/0xbef42c0221d6f1c5 lrc: 0/0,0 mode: --/PR res: [0x200027cd1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00ced8b4 expref: -99 pid: 159931 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000020:00100000:1.0:1787826251.356866:0:159931:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787826251.356872:0:159931:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c35b2a00 x1874663042989568/t0(0) o36->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 488/512 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826251.356894:0:159931:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job touch:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:159931:1874663042989568:192.168.203.147@tcp:36:touch.0 00000100:00100000:1.0:1787826251.356967:0:159931:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d225fe40 going to sleep for 11 seconds 00010000:00010000:3.0:1787826251.361027:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826251.366301:0:159931:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job touch:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:159931:1874663042989568:192.168.203.147@tcp:36:touch.0 00000080:00200000:1.0:1787826251.366363:0:159931:0:(vvp_io.c:1826:vvp_io_init()) [0x200027cd1:0x1:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00010000:1.0:1787826251.366390:0:159931:0:(file.c:7065:ll_layout_intent()) ### lustre: requeue layout lock for file [0x200027cd1:0x1:0x0](ffff9181c83e7448) 00000002:00010000:1.0:1787826251.366401:0:159931:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200027cd1:0x1:0x0]) in obj [0x200027cd1:0x1:0x0], intent: layout flags 0 00010000:00010000:1.0:1787826251.366463:0:159931:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5e00/0xbef42c0221d6f1cc lrc: 3/1,0 mode: --/CR res: [0x200027cd1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 159931 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826251.366478:0:159931:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5e00/0xbef42c0221d6f1cc lrc: 3/1,0 mode: --/CR res: [0x200027cd1:0x1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x0 pid: 159931 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826251.366488:0:159931:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5e00/0xbef42c0221d6f1cc lrc: 3/1,0 mode: --/CR res: [0x200027cd1:0x1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x0 pid: 159931 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826251.366506:0:159931:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1880 pname:cluuid:pid:xid:nid:opc:job touch:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:159931:1874663042989696:192.168.203.147@tcp:101:touch.0 00000100:00100000:1.0:1787826251.366579:0:159931:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d225fe40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826251.373183:0:159931:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1880 pname:cluuid:pid:xid:nid:opc:job touch:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:159931:1874663042989696:192.168.203.147@tcp:101:touch.0 00010000:00010000:1.0:1787826251.373216:0:159931:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5e00/0xbef42c0221d6f1cc lrc: 4/1,0 mode: CR/CR res: [0x200027cd1:0x1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced8c9 expref: -99 pid: 159931 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826251.373232:0:159931:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5e00/0xbef42c0221d6f1cc lrc: 4/1,0 mode: CR/CR res: [0x200027cd1:0x1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced8c9 expref: -99 pid: 159931 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826251.373247:0:159931:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x400 disposition=0x1, status=0 req@ffff9181c35b1880 x1874663042989696/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 376/400 e 0 to 0 dl 1787826267 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'touch.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826251.373258:0:159931:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: layout, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5e00/0xbef42c0221d6f1cc lrc: 3/1,0 mode: CR/CR res: [0x200027cd1:0x1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced8c9 expref: -99 pid: 159931 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00000002:00010000:1.0:1787826251.373274:0:159931:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5e00/0xbef42c0221d6f1cc lrc: 3/1,0 mode: CR/CR res: [0x200027cd1:0x1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced8c9 expref: -99 pid: 159931 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826251.373288:0:159931:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d1a77c00 type 13 mode 16 res 8590097617/1 (0 0) 00000080:00200000:1.0:1787826251.373295:0:159931:0:(namei.c:952:ll_intent_lock()) intent lock 1024 on i1 [0x200027cd1:0x1:0x0] suppgids 0 0: rc 0 00000080:00010000:1.0:1787826251.373313:0:159931:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200027cd1:0x1:0x0](ffff9181c83e7448) for lock 0xbef42c0221d6f1cc 00000080:00010000:1.0:1787826251.373322:0:159931:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f1cc from it ffffb9750303ba78 00010000:00010000:1.0:1787826251.373325:0:159931:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5e00/0xbef42c0221d6f1cc lrc: 3/1,0 mode: CR/CR res: [0x200027cd1:0x1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced8c9 expref: -99 pid: 159931 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826251.373337:0:159931:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5e00/0xbef42c0221d6f1cc lrc: 2/0,0 mode: CR/CR res: [0x200027cd1:0x1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced8c9 expref: -99 pid: 159931 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826251.373361:0:159931:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5e00/0xbef42c0221d6f1cc lrc: 2/1,0 mode: CR/CR res: [0x200027cd1:0x1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced8c9 expref: -99 pid: 159931 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826251.373372:0:159931:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5e00/0xbef42c0221d6f1cc lrc: 2/1,0 mode: CR/CR res: [0x200027cd1:0x1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced8c9 expref: -99 pid: 159931 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00010000:1.0:1787826251.373381:0:159931:0:(file.c:6948:ll_layout_lock_set()) ### file [0x200027cd1:0x1:0x0](00000000da86e177) being reconfigured ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5e00/0xbef42c0221d6f1cc lrc: 3/1,0 mode: CR/CR res: [0x200027cd1:0x1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced8c9 expref: -99 pid: 159931 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826251.373458:0:159931:0:(file.c:6843:ll_layout_conf()) [0x200027cd1:0x1:0x0]: layout version change: 4294967294 -> 0 00010000:00010000:1.0:1787826251.373464:0:159931:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5e00/0xbef42c0221d6f1cc lrc: 3/1,0 mode: CR/CR res: [0x200027cd1:0x1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00ced8c9 expref: -99 pid: 159931 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826251.373472:0:159931:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5e00/0xbef42c0221d6f1cc lrc: 2/0,0 mode: CR/CR res: [0x200027cd1:0x1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00ced8c9 expref: -99 pid: 159931 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00020000:00200000:1.0:1787826251.373492:0:159931:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000100:00100000:0.0:1787826251.374121:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663042989824:192.168.203.147@tcp:2:touch.0 00000100:00100000:1.0:1787826251.396059:0:2389:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff9181c5620000: committing for last_committed 38654705666 gen 1 00000100:00100000:1.0:1787826251.396104:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663042989824:192.168.203.147@tcp:2:touch.0 00000080:00200000:2.0:1787826251.396500:0:159931:0:(vvp_io.c:304:vvp_io_fini()) [0x200027cd1:0x1:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200008:2.0:1787826251.396521:0:159931:0:(llite_lib.c:2686:ll_setattr_raw()) COMPLETED file f74.replay-single:[0x200027cd1:0x1:0x0](ffff9181c83e7448) current size 0, valid attrs 100201f0, mode 9181, uid 50576976, gid -18059, new size 18446744071751268887, atime 1787826251.000000000, mtime 1787826251.000000000, ctime 1787826251.000000000, rc 0 00000080:00200008:2.0:1787826251.396558:0:159931:0:(file.c:479:ll_file_release()) START file f74.replay-single:[0x200027cd1:0x1:0x0](ffff9181c83e7448), flags 104001 00000002:00100000:2.0:1787826251.396626:0:159931:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181c35b3480 x1874663042989184/t347892350977(347892350977) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826267 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'touch.0' uid:0 gid:0 projid:0 00000020:00100000:2.0:1787826251.396639:0:159931:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1787826251.396645:0:159931:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c71f3100 x1874663042990080/t0(0) o35->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:2.0:1787826251.396670:0:159931:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f3100 pname:cluuid:pid:xid:nid:opc:job touch:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:159931:1874663042990080:192.168.203.147@tcp:35:touch.0 00000100:00100000:2.0:1787826251.396798:0:159931:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1dc33c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826251.407907:0:159931:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f3100 pname:cluuid:pid:xid:nid:opc:job touch:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:159931:1874663042990080:192.168.203.147@tcp:35:touch.0 00000002:00100000:2.0:1787826251.407953:0:159931:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181c35b3480 x1874663042989184/t347892350977(347892350977) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826267 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'touch.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1787826251.425052:0:159932:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:2.0:1787826251.425092:0:159932:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826251.425180:0:159932:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7e00/0xbef42c0221d6f1d3 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 159932 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.425192:0:159932:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7e00/0xbef42c0221d6f1d3 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 159932 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.425203:0:159932:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7e00/0xbef42c0221d6f1d3 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 159932 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826251.425237:0:159932:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f2d80 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:159932:1874663042990208:192.168.203.147@tcp:101:rm.0 00000100:00100000:2.0:1787826251.426474:0:159932:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1dc33c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826251.432196:0:159932:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f2d80 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:159932:1874663042990208:192.168.203.147@tcp:101:rm.0 00010000:00010000:2.0:1787826251.432233:0:159932:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7e00/0xbef42c0221d6f1d3 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced8d7 expref: -99 pid: 159932 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.432247:0:159932:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7e00/0xbef42c0221d6f1d3 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced8d7 expref: -99 pid: 159932 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.432548:0:159932:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7e00/0xbef42c0221d6f1d3 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced8d7 expref: -99 pid: 159932 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.432567:0:159932:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7e00/0xbef42c0221d6f1d3 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced8d7 expref: -99 pid: 159932 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.432591:0:159932:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7e00/0xbef42c0221d6f1d3 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced8d7 expref: -99 pid: 159932 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.432601:0:159932:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7e00/0xbef42c0221d6f1d3 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced8d7 expref: -99 pid: 159932 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.432609:0:159932:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7e00/0xbef42c0221d6f1d3 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced8d7 expref: -99 pid: 159932 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787826251.432624:0:159932:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c71f2d80 x1874663042990208/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826267 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787826251.432640:0:159932:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7e00/0xbef42c0221d6f1d3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced8d7 expref: -99 pid: 159932 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.432655:0:159932:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d1a77c00 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:2.0:1787826251.432664:0:159932:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:2.0:1787826251.432719:0:159932:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 69632 00000080:00010000:2.0:1787826251.432737:0:159932:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f1d3 00000080:00010000:2.0:1787826251.432784:0:159932:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:2.0:1787826251.432789:0:159932:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f1d3 from it ffffb97503013b58 00010000:00010000:2.0:1787826251.432794:0:159932:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7e00/0xbef42c0221d6f1d3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced8d7 expref: -99 pid: 159932 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.432803:0:159932:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7e00/0xbef42c0221d6f1d3 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced8d7 expref: -99 pid: 159932 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826251.432842:0:159932:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:2.0:1787826251.432870:0:159932:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f74.replay-single, dir=[0x200000007:0x1:0x0](ffff9181d080cb08), flags=8 00000080:00200000:2.0:1787826251.432876:0:159932:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f74.replay-single, dir=[0x200000007:0x1:0x0](ffff9181d080cb08), intent=getattr 00000002:00010000:2.0:1787826251.432906:0:159932:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f74.replay-single,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 01 00010000:00010000:2.0:1787826251.432958:0:159932:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7400/0xbef42c0221d6f1da lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 159932 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.432966:0:159932:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7400/0xbef42c0221d6f1da lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 159932 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.432977:0:159932:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7400/0xbef42c0221d6f1da lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 159932 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826251.433004:0:159932:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f2d80 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:159932:1874663042990336:192.168.203.147@tcp:101:rm.0 00000100:00100000:2.0:1787826251.433102:0:159932:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1dc33c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826251.440736:0:159932:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f2d80 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:159932:1874663042990336:192.168.203.147@tcp:101:rm.0 00010000:00010000:2.0:1787826251.440834:0:159932:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7400/0xbef42c0221d6f1da lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced8f3 expref: -99 pid: 159932 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.440862:0:159932:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7400/0xbef42c0221d6f1da lrc: 4/1,0 mode: --/PR res: [0x200027cd1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced8f3 expref: -99 pid: 159932 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.440873:0:159932:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7400/0xbef42c0221d6f1da lrc: 4/1,0 mode: --/PR res: [0x200027cd1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced8f3 expref: -99 pid: 159932 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.440890:0:159932:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7400/0xbef42c0221d6f1da lrc: 4/1,0 mode: PR/PR res: [0x200027cd1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced8f3 expref: -99 pid: 159932 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.440906:0:159932:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7400/0xbef42c0221d6f1da lrc: 4/1,0 mode: PR/PR res: [0x200027cd1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced8f3 expref: -99 pid: 159932 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.440922:0:159932:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7400/0xbef42c0221d6f1da lrc: 5/2,0 mode: PR/PR res: [0x200027cd1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced8f3 expref: -99 pid: 159932 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.440931:0:159932:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7400/0xbef42c0221d6f1da lrc: 5/2,0 mode: PR/PR res: [0x200027cd1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced8f3 expref: -99 pid: 159932 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.440939:0:159932:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7400/0xbef42c0221d6f1da lrc: 4/1,0 mode: PR/PR res: [0x200027cd1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced8f3 expref: -99 pid: 159932 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787826251.440953:0:159932:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c71f2d80 x1874663042990336/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 592/640 e 0 to 0 dl 1787826267 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787826251.440965:0:159932:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: getattr, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7400/0xbef42c0221d6f1da lrc: 3/1,0 mode: PR/PR res: [0x200027cd1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced8f3 expref: -99 pid: 159932 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:2.0:1787826251.440984:0:159932:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7400/0xbef42c0221d6f1da lrc: 3/1,0 mode: PR/PR res: [0x200027cd1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced8f3 expref: -99 pid: 159932 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.440994:0:159932:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d1a77c00 type 13 mode 4 res 8590097617/1 (0 0) 00000080:00200000:2.0:1787826251.441003:0:159932:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1787826251.441065:0:159932:0:(namei.c:147:ll_iget()) got inode: [0x200027cd1:0x1:0x0](ffff9181c83e7448): rc = 0 00000080:00200000:2.0:1787826251.441083:0:159932:0:(file.c:6843:ll_layout_conf()) [0x200027cd1:0x1:0x0]: layout version change: 0 -> 0 00000080:00010000:2.0:1787826251.441090:0:159932:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200027cd1:0x1:0x0](ffff9181c83e7448) for lock 0xbef42c0221d6f1da 00000080:00010000:2.0:1787826251.441105:0:159932:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200027cd1:0x1:0x0](ffff9181c83e7448) 00000080:00010000:2.0:1787826251.441111:0:159932:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f1da from it ffffb97503013b20 00010000:00010000:2.0:1787826251.441114:0:159932:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7400/0xbef42c0221d6f1da lrc: 3/1,0 mode: PR/PR res: [0x200027cd1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00ced8f3 expref: -99 pid: 159932 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826251.441125:0:159932:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7400/0xbef42c0221d6f1da lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00ced8f3 expref: -99 pid: 159932 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200008:0.0:1787826251.441220:0:159932:0:(file.c:6296:ll_getattr_dentry()) START file f74.replay-single:[0x200027cd1:0x1:0x0](ffff9181c83e7448), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1787826251.441230:0:159932:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200027cd1:0x1:0x0](ffff9181c83e7448),name=f74.replay-single 00000002:00010000:0.0:1787826251.441445:0:159932:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200027cd1:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826251.441460:0:159932:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7400/0xbef42c0221d6f1da lrc: 2/1,0 mode: PR/PR res: [0x200027cd1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00ced8f3 expref: -99 pid: 159932 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826251.441473:0:159932:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7400/0xbef42c0221d6f1da lrc: 2/1,0 mode: PR/PR res: [0x200027cd1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00ced8f3 expref: -99 pid: 159932 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826251.441553:0:159932:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826251.441562:0:159932:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200027cd1:0x1:0x0](ffff9181c83e7448) 00000080:00010000:0.0:1787826251.441565:0:159932:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200027cd1:0x1:0x0](ffff9181c83e7448) for lock 0xbef42c0221d6f1da 00000080:00010000:0.0:1787826251.441647:0:159932:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f1da from it ffffb97503013cb8 00010000:00010000:0.0:1787826251.441650:0:159932:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7400/0xbef42c0221d6f1da lrc: 3/1,0 mode: PR/PR res: [0x200027cd1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00ced8f3 expref: -99 pid: 159932 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826251.441658:0:159932:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7400/0xbef42c0221d6f1da lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00ced8f3 expref: -99 pid: 159932 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787826251.441717:0:159932:0:(vvp_io.c:1826:vvp_io_init()) [0x200027cd1:0x1:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:0.0:1787826251.441729:0:159932:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x200027cd1:0x1:0x0] 00000008:00010000:0.0:1787826251.441804:0:159932:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: a 00000008:00010000:0.0:1787826251.441809:0:159932:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff9181d2891cb0, osc lock ffff9181c716ef70, flags 0x400001002 00010000:00010000:0.0:1787826251.441873:0:159932:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff9181c5620000 lock: 0000000029ce9f28/0xbef42c0221d6f1e1 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x18a2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159932 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826251.441883:0:159932:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x400001000 ns: lustre-OST0000-osc-ffff9181c5620000 lock: 0000000029ce9f28/0xbef42c0221d6f1e1 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x18a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159932 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826251.441990:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663042990464:192.168.203.147@tcp:101:rm.0 00000020:00200000:0.0:1787826251.442130:0:159932:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00010000:00010000:1.0:1787826251.444472:0:2390:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff9181c5620000 lock: 0000000029ce9f28/0xbef42c0221d6f1e1 lrc: 6/2,0 mode: --/PR res: [0x280000401:0x18a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159932 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826251.444504:0:2390:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0000-osc-ffff9181c5620000 lock: 0000000029ce9f28/0xbef42c0221d6f1e1 lrc: 6/2,0 mode: --/PR res: [0x280000401:0x18a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced8fa expref: -99 pid: 159932 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826251.444523:0:2390:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff9181c5620000 lock: 0000000029ce9f28/0xbef42c0221d6f1e1 lrc: 6/2,0 mode: PR/PR res: [0x280000401:0x18a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced8fa expref: -99 pid: 159932 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826251.444543:0:2390:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0000-osc-ffff9181c5620000 lock: 0000000029ce9f28/0xbef42c0221d6f1e1 lrc: 6/2,0 mode: PR/PR res: [0x280000401:0x18a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced8fa expref: -99 pid: 159932 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:1.0:1787826251.444562:0:2390:0:(osc_lock.c:187:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0000-osc-ffff9181c5620000 lock: 0000000029ce9f28/0xbef42c0221d6f1e1 lrc: 5/2,0 mode: PR/PR res: [0x280000401:0x18a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00ced8fa expref: -99 pid: 159932 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:1.0:1787826251.444577:0:2390:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00010000:00010000:1.0:1787826251.444595:0:2390:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9181c5620000 lock: 0000000029ce9f28/0xbef42c0221d6f1e1 lrc: 6/2,0 mode: PR/PR res: [0x280000401:0x18a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x878a5eea00ced8fa expref: -99 pid: 159932 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826251.444605:0:2390:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0000-osc-ffff9181c5620000 lock: 0000000029ce9f28/0xbef42c0221d6f1e1 lrc: 5/1,0 mode: PR/PR res: [0x280000401:0x18a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x878a5eea00ced8fa expref: -99 pid: 159932 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826251.444614:0:2390:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9181c5620000 lock: 0000000029ce9f28/0xbef42c0221d6f1e1 lrc: 5/1,0 mode: PR/PR res: [0x280000401:0x18a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x878a5eea00ced8fa expref: -99 pid: 159932 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826251.444629:0:2390:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff9181c5620000 lock: 0000000029ce9f28/0xbef42c0221d6f1e1 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x18a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x878a5eea00ced8fa expref: -99 pid: 159932 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000100:00100000:1.0:1787826251.444654:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663042990464:192.168.203.147@tcp:101:rm.0 00000080:00200000:0.0:1787826251.444655:0:159932:0:(file.c:1775:ll_merge_attr_nolock()) [0x200027cd1:0x1:0x0] updating i_size 0 i_blocks 0 00000020:00010000:0.0:1787826251.444663:0:159932:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9181c50b2bf0 (ffff9181c5a67878/0) at cl_lock_release():245 00000020:00010000:0.0:1787826251.444666:0:159932:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9181c50b2bf0 (ffff9181c5a67878/0) at cl_lock_cancel():142 00000020:00010000:0.0:1787826251.444672:0:159932:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9181d2891cb0 (ffff9181c5a67518/1) at cl_lock_cancel():142 00000020:00010000:0.0:1787826251.444677:0:159932:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9181c50b2bf0 (ffff9181c5a67878/0) at cl_lock_fini():72 00000020:00010000:0.0:1787826251.444680:0:159932:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9181d2891cb0 (ffff9181c5a67878/1) at cl_lock_fini():72 00000080:00200000:0.0:1787826251.444698:0:159932:0:(vvp_io.c:304:vvp_io_fini()) [0x200027cd1:0x1:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1787826251.444812:0:159932:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:0.0:1787826251.444841:0:159932:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826251.444860:0:159932:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7e00/0xbef42c0221d6f1d3 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced8d7 expref: -99 pid: 159932 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826251.444874:0:159932:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7e00/0xbef42c0221d6f1d3 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced8d7 expref: -99 pid: 159932 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826251.444882:0:159932:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826251.444890:0:159932:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:0.0:1787826251.444894:0:159932:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f1d3 00000080:00010000:0.0:1787826251.444901:0:159932:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f1d3 from it ffffb97503013c10 00010000:00010000:0.0:1787826251.444905:0:159932:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7e00/0xbef42c0221d6f1d3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced8d7 expref: -99 pid: 159932 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826251.444913:0:159932:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7e00/0xbef42c0221d6f1d3 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced8d7 expref: -99 pid: 159932 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787826251.444923:0:159932:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826251.444936:0:159932:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f74.replay-single, flags=0 00000080:00200000:0.0:1787826251.444944:0:159932:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:0.0:1787826251.444956:0:159932:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826251.444963:0:159932:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7e00/0xbef42c0221d6f1d3 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced8d7 expref: -99 pid: 159932 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787826251.444971:0:159932:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7e00/0xbef42c0221d6f1d3 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced8d7 expref: -99 pid: 159932 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787826251.444978:0:159932:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826251.444983:0:159932:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:0.0:1787826251.444986:0:159932:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f1d3 00000080:00010000:0.0:1787826251.444989:0:159932:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f1d3 from it ffffb97503013d68 00010000:00010000:0.0:1787826251.444993:0:159932:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7e00/0xbef42c0221d6f1d3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced8d7 expref: -99 pid: 159932 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787826251.444999:0:159932:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7e00/0xbef42c0221d6f1d3 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced8d7 expref: -99 pid: 159932 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00000080:00200000:0.0:1787826251.445008:0:159932:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 3 00000080:00200000:0.0:1787826251.445014:0:159932:0:(namei.c:2418:ll_unlink()) VFS Op:name=f74.replay-single, dir=[0x200000007:0x1:0x0](ffff9181d080cb08) 00010000:00010000:0.0:1787826251.445047:0:159932:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7e00/0xbef42c0221d6f1d3 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ced8d7 expref: -99 pid: 159932 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00010000:00010000:0.0:1787826251.445137:0:159932:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7400/0xbef42c0221d6f1da lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x878a5eea00ced8f3 expref: -99 pid: 159932 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787826251.445149:0:159932:0:(vvp_object.c:117:vvp_conf_set()) [0x200027cd1:0x1:0x0]: losing layout lock 00010000:00010000:0.0:1787826251.445165:0:159932:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5e00/0xbef42c0221d6f1cc lrc: 2/0,0 mode: CR/CR res: [0x200027cd1:0x1:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x878a5eea00ced8c9 expref: -99 pid: 159931 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787826251.445172:0:159932:0:(vvp_object.c:117:vvp_conf_set()) [0x200027cd1:0x1:0x0]: losing layout lock 00010000:00010000:0.0:1787826251.445207:0:159932:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7400/0xbef42c0221d6f1da lrc: 1/0,0 mode: --/PR res: [0x200027cd1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00ced8f3 expref: -99 pid: 159932 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787826251.445214:0:159932:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5e00/0xbef42c0221d6f1cc lrc: 1/0,0 mode: --/CR res: [0x200027cd1:0x1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00ced8c9 expref: -99 pid: 159931 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787826251.445221:0:159932:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7e00/0xbef42c0221d6f1d3 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ced8d7 expref: -99 pid: 159932 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00010000:00010000:0.0:1787826251.445236:0:159932:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 3 locks packed 00010000:00010000:0.0:1787826251.445239:0:159932:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7400/0xbef42c0221d6f1da lrc: 0/0,0 mode: --/PR res: [0x200027cd1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00ced8f3 expref: -99 pid: 159932 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787826251.445410:0:159932:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5e00/0xbef42c0221d6f1cc lrc: 0/0,0 mode: --/CR res: [0x200027cd1:0x1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00ced8c9 expref: -99 pid: 159931 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787826251.445441:0:159932:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7e00/0xbef42c0221d6f1d3 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ced8d7 expref: -99 pid: 159932 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00000020:00100000:0.0:1787826251.445468:0:159932:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787826251.445480:0:159932:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c55a6300 x1874663042990720/t0(0) o36->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 512/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826251.445511:0:159932:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6300 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:159932:1874663042990720:192.168.203.147@tcp:36:rm.0 00000100:00100000:0.0:1787826251.445618:0:159932:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8300 going to sleep for 11 seconds 00000100:00100000:1.0:1787826251.452506:0:159932:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6300 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:159932:1874663042990720:192.168.203.147@tcp:36:rm.0 00000080:00200000:1.0:1787826251.452635:0:159932:0:(vvp_io.c:1826:vvp_io_init()) [0x200027cd1:0x1:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1787826251.452650:0:159932:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1787826251.453618:0:159932:0:(vvp_io.c:304:vvp_io_fini()) [0x200027cd1:0x1:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1787826251.453635:0:159932:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200027cd1:0x1:0x0](ffff9181c83e7448) 00000008:00010000:1.0:1787826251.453775:0:159932:0:(osc_object.c:202:osc_object_ast_clear()) ### update lock size 0 blocks 0 [cma]time: 1787826251 1787826251 1787826251 by oinfo size 0 blocks 0 [cma]time 1787826251 1787826251 1787826251 ns: lustre-OST0000-osc-ffff9181c5620000 lock: 0000000029ce9f28/0xbef42c0221d6f1e1 lrc: 1/0,0 mode: PR/PR res: [0x280000401:0x18a2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x878a5eea00ced8fa expref: -99 pid: 159932 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826252.384950:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826252.940310:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826252.944287:0:159723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049347968 00000100:00100000:0.0:1787826252.944300:0:159723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049347968 00000100:00100000:0.0:1787826252.944371:0:159723:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 1 00000100:00100000:0.0:1787826252.944381:0:159723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b1880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:LOV_OSC_UUID+4:179226:x1874663049347968:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826252.944399:0:159723:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-OST0000-osc-ffff9181c5620000 lock: 0000000029ce9f28/0xbef42c0221d6f1e1 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x18a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x420000010000 nid: local remote: 0x878a5eea00ced8fa expref: -99 pid: 159932 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826252.944489:0:159723:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826252.944496:0:159723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b1880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:LOV_OSC_UUID+4:179226:x1874663049347968:12345-192.168.203.147@tcp:104: Request processed in 117us (4199us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826252.944504:0:159723:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 1 00010000:00010000:3.0:1787826252.944563:0:159733:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826252.944576:0:159733:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0000-osc-ffff9181c5620000 lock: 0000000029ce9f28/0xbef42c0221d6f1e1 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x18a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x420000010000 nid: local remote: 0x878a5eea00ced8fa expref: -99 pid: 159932 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826252.944591:0:159733:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0512200 already unused, calling callback (ffffffffc10ec300) 00010000:00010000:3.0:1787826252.944599:0:159733:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0000-osc-ffff9181c5620000 lock: 0000000029ce9f28/0xbef42c0221d6f1e1 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x18a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x428400010000 nid: local remote: 0x878a5eea00ced8fa expref: -99 pid: 159932 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826252.944680:0:159733:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0000-osc-ffff9181c5620000 lock: 0000000029ce9f28/0xbef42c0221d6f1e1 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x18a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4c29400010020 nid: local remote: 0x878a5eea00ced8fa expref: -99 pid: 159932 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826252.944688:0:159733:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826252.944721:0:159733:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0000-osc-ffff9181c5620000 lock: 0000000029ce9f28/0xbef42c0221d6f1e1 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x18a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4c29400010020 nid: local remote: 0x878a5eea00ced8fa expref: -99 pid: 159932 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826252.944730:0:159733:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0000-osc-ffff9181c5620000 lock: 0000000029ce9f28/0xbef42c0221d6f1e1 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x18a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4c29400010020 nid: local remote: 0x878a5eea00ced8fa expref: -99 pid: 159932 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826252.944820:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826252.944823:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826252.944825:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826252.944850:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663042990848:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1787826252.959499:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663042990848:192.168.203.147@tcp:103:ldlm_bl.0 00000080:00200008:2.0:1787826253.385430:0:159959:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181d080cb08), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1787826253.385447:0:159959:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:2.0:1787826253.385477:0:159959:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826253.385534:0:159959:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f1e8 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 159959 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826253.385543:0:159959:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f1e8 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 159959 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826253.385554:0:159959:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f1e8 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 159959 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826253.385577:0:159959:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f2d80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:159959:1874663042990976:192.168.203.147@tcp:101:lfs.0 00000100:00100000:2.0:1787826253.386404:0:159959:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c3c96780 going to sleep for 11 seconds 00000100:00100000:2.0:1787826253.391299:0:159959:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9181c5620000: committing for last_committed 347892350980 gen 1 00000100:00100000:2.0:1787826253.391349:0:159959:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f2d80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:159959:1874663042990976:192.168.203.147@tcp:101:lfs.0 00010000:00010000:2.0:1787826253.391373:0:159959:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f1e8 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced92b expref: -99 pid: 159959 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826253.391385:0:159959:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f1e8 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced92b expref: -99 pid: 159959 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826253.391404:0:159959:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f1e8 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced92b expref: -99 pid: 159959 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826253.391425:0:159959:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f1e8 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced92b expref: -99 pid: 159959 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826253.391438:0:159959:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f1e8 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced92b expref: -99 pid: 159959 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826253.391445:0:159959:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f1e8 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced92b expref: -99 pid: 159959 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826253.391451:0:159959:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f1e8 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced92b expref: -99 pid: 159959 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787826253.391461:0:159959:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c71f2d80 x1874663042990976/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826269 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787826253.391474:0:159959:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f1e8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced92b expref: -99 pid: 159959 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826253.391486:0:159959:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d1a77c00 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:2.0:1787826253.391492:0:159959:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:2.0:1787826253.391525:0:159959:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 69632 00000080:00010000:2.0:1787826253.391533:0:159959:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f1e8 00000080:00010000:2.0:1787826253.391538:0:159959:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:2.0:1787826253.391541:0:159959:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f1e8 from it ffffb97500da3cb8 00010000:00010000:2.0:1787826253.391545:0:159959:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f1e8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced92b expref: -99 pid: 159959 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826253.391552:0:159959:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f1e8 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced92b expref: -99 pid: 159959 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:2.0:1787826253.393054:0:159959:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181d080cb08), request_mask 0, flags 0, foreign 0 00000080:00200000:2.0:1787826253.393063:0:159959:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:2.0:1787826253.393081:0:159959:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826253.393093:0:159959:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f1e8 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced92b expref: -99 pid: 159959 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826253.393107:0:159959:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f1e8 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced92b expref: -99 pid: 159959 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826253.393119:0:159959:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826253.393126:0:159959:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:2.0:1787826253.393129:0:159959:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f1e8 00000080:00010000:2.0:1787826253.393135:0:159959:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f1e8 from it ffffb97500da3cb8 00010000:00010000:2.0:1787826253.393139:0:159959:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f1e8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced92b expref: -99 pid: 159959 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826253.393148:0:159959:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f1e8 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced92b expref: -99 pid: 159959 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1787826253.393251:0:159959:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:2.0:1787826253.393259:0:159959:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826253.393266:0:159959:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f1e8 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced92b expref: -99 pid: 159959 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787826253.393275:0:159959:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f1e8 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced92b expref: -99 pid: 159959 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1787826253.393281:0:159959:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826253.393285:0:159959:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:2.0:1787826253.393287:0:159959:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f1e8 00000080:00010000:2.0:1787826253.393291:0:159959:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f1e8 from it ffffb97500da3bb0 00010000:00010000:2.0:1787826253.393293:0:159959:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f1e8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced92b expref: -99 pid: 159959 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787826253.393300:0:159959:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f1e8 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced92b expref: -99 pid: 159959 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:2.0:1787826253.393308:0:159959:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 44 00000080:00200008:2.0:1787826253.393315:0:159959:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9181d080cb08), flags 100000 00000080:00200080:2.0:1787826253.393426:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.393440:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00800000:00000004:2.0:1787826253.393448:0:159959:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9181c5620000: age 8306, max_age 8324 00000100:00100000:2.0:1787826253.393485:0:159959:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f2d80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:159959:1874663042991104:192.168.203.147@tcp:41:lfs.0 00000100:00100000:2.0:1787826253.393573:0:159959:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c3c96780 going to sleep for 11 seconds 00000100:00100000:2.0:1787826253.406127:0:159959:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f2d80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:159959:1874663042991104:192.168.203.147@tcp:41:lfs.0 00800000:00000004:2.0:1787826253.406156:0:159959:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0000-mdc-ffff9181c5620000: update ffff9181c354d178 cache blocks 317234/353529 objects 1023647/1024000 00000080:00200080:2.0:1787826253.406251:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.406268:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00800000:00000004:2.0:1787826253.406275:0:159959:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9181c5620000: age 7306, max_age 8324 00000100:00100000:2.0:1787826253.406310:0:159959:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f2d80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:159959:1874663042991232:192.168.203.147@tcp:41:lfs.0 00000100:00100000:2.0:1787826253.406404:0:159959:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c3c96780 going to sleep for 11 seconds 00010000:00010000:0.0:1787826253.413019:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826253.413920:0:159959:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f2d80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:159959:1874663042991232:192.168.203.147@tcp:41:lfs.0 00800000:00000004:2.0:1787826253.413958:0:159959:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0001-mdc-ffff9181c5620000: update ffff9181c3548450 cache blocks 317336/353529 objects 1023704/1024000 00000080:00200080:2.0:1787826253.414864:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.414894:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.414905:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.414910:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.414914:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.414918:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.414924:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.414929:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.414935:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.414940:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.414945:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.414948:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.414953:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.414956:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.414960:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.414964:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.414968:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.414971:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.414975:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.414979:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.414986:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.414990:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.414994:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.414997:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.415001:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.415005:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.415010:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.415014:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.415017:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.415021:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.415024:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.415028:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.415032:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.416618:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.416660:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.416683:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.416687:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.416691:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.416695:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.416698:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.416702:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.416705:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.416709:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.416712:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.416716:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.416721:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.416726:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.416729:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.416734:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.416737:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.416741:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.416767:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.416772:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.416776:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.416780:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.416784:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.416787:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.416790:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.416794:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.416798:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.416802:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.416805:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.416809:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.416812:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.416816:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.416819:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.416823:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.416827:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.416831:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.416834:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.416838:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.416841:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.416845:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.416848:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.416852:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.416855:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.416859:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.416862:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.416867:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.416870:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.416874:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.416878:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.416882:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.416885:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.416890:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.416893:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.416897:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.416901:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.416905:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.416909:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.416912:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.416916:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.416920:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.416923:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.416926:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.416930:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.416934:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.416938:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.416942:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.416945:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.416949:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.416953:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.416956:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.416960:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.416964:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.416968:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.416972:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.416976:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.416979:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.416982:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.416987:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.416990:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.416994:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.416998:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.417002:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.417005:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.417009:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.417012:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.417016:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.417020:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.417025:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.417028:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.417032:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.417036:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.417052:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00800000:00000080:2.0:1787826253.417058:0:159959:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.417064:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.417071:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00020000:00000004:2.0:1787826253.417081:0:159959:0:(obd_class.h:1090:obd_statfs()) lustre-OST0000-osc-ffff9181c5620000: age 7306, max_age 8324 00000100:00100000:2.0:1787826253.417144:0:159959:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f2d80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:159959:1874663042991360:192.168.203.147@tcp:13:lfs.0 00000100:00100000:2.0:1787826253.417279:0:159959:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c3c96780 going to sleep for 11 seconds 00000100:00100000:2.0:1787826253.428202:0:159959:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f2d80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:159959:1874663042991360:192.168.203.147@tcp:13:lfs.0 00020000:00000004:2.0:1787826253.428246:0:159959:0:(obd_class.h:1116:obd_statfs()) lustre-OST0000-osc-ffff9181c5620000: update ffff9181c3549e08 cache blocks 901356/958279 objects 261710/262144 00000080:00200080:2.0:1787826253.428346:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.428367:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00020000:00000004:2.0:1787826253.428377:0:159959:0:(obd_class.h:1090:obd_statfs()) lustre-OST0001-osc-ffff9181c5620000: age 7306, max_age 8324 00000100:00100000:2.0:1787826253.428432:0:159959:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f2d80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:159959:1874663042991488:192.168.203.147@tcp:13:lfs.0 00000100:00100000:2.0:1787826253.429159:0:159959:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c3c96780 going to sleep for 11 seconds 00000100:00100000:2.0:1787826253.435022:0:159959:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f2d80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:159959:1874663042991488:192.168.203.147@tcp:13:lfs.0 00020000:00000004:2.0:1787826253.435077:0:159959:0:(obd_class.h:1116:obd_statfs()) lustre-OST0001-osc-ffff9181c5620000: update ffff9181c354b7c0 cache blocks 901360/958279 objects 261727/262144 00000080:00200080:2.0:1787826253.435316:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435344:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435354:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435360:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435380:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435386:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435391:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435395:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435516:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435523:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435528:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435531:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435536:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435539:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435543:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435547:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435552:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435556:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435561:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435565:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435576:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435581:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435585:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435589:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435594:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435598:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435602:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435605:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435609:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435614:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435618:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435622:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435625:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435630:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435634:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435638:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435642:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435645:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435649:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435653:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435657:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435660:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435664:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435668:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435672:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435676:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435680:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435684:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435688:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435692:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435696:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435700:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435703:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435707:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435712:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435716:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435720:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435723:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435727:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435731:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435735:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435739:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435781:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435788:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435799:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435804:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435809:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435812:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435817:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435821:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435825:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435829:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435834:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435837:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435841:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435845:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435850:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435854:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435858:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435863:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435868:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435871:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435875:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435880:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435884:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435887:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435892:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435895:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435900:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435904:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435908:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435911:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435915:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435920:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435924:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435927:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435931:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435934:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435938:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435942:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435946:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435949:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435953:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435957:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435962:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435966:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435971:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435975:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435980:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435984:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435988:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.435992:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.435996:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.436001:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.436005:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.436009:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.436013:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.436017:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.436022:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.436025:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.436029:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.436033:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.436037:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.436041:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200080:2.0:1787826253.436045:0:159959:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd028a9460 00020000:00000080:2.0:1787826253.436255:0:159959:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df1c00 uarg=0000000000000000 00000080:00200008:2.0:1787826253.436280:0:159959:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9181d080cb08), flags 100000 00010000:00010000:3.0:1787826254.434089:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826255.457508:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826255.907291:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:f40147eb-9b53-47d4-b77c-99ceedd9664c:2391:1874663042991616:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826255.907385:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663042991872:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826255.907428:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663042992128:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826255.907459:0:2391:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1787826255.907465:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663042991744:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826255.907489:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663042992000:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826255.913088:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663042991744:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826255.913106:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663042992000:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826255.913116:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:f40147eb-9b53-47d4-b77c-99ceedd9664c:2391:1874663042991616:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826255.913127:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663042991872:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826255.913139:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663042992128:192.168.203.147@tcp:400:kworker.0 00010000:00010000:3.0:1787826256.482288:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826257.504925:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826258.528901:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826259.552914:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:1.0:1787826259.904688:0:160550:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == replay-single test 80a: DNE: create remote dir, drop update rep from MDT0, fail MDT0 ========================================================== 06:24:18 (1787826258) 00000080:00200000:0.0:1787826260.088331:0:160559:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:0.0:1787826260.088383:0:160559:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826260.088412:0:160559:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f1e8 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced92b expref: -99 pid: 159959 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787826260.088434:0:160559:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f1e8 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced92b expref: -99 pid: 159959 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787826260.088442:0:160559:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826260.088453:0:160559:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:0.0:1787826260.088458:0:160559:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f1e8 00000080:00010000:0.0:1787826260.088467:0:160559:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f1e8 from it ffffb97501ddbb58 00010000:00010000:0.0:1787826260.088472:0:160559:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f1e8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced92b expref: -99 pid: 159959 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787826260.088483:0:160559:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f1e8 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced92b expref: -99 pid: 159959 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00000080:00200000:0.0:1787826260.088501:0:160559:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826260.088518:0:160559:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=d80a.replay-single, dir=[0x200000007:0x1:0x0](ffff9181d080cb08), flags=13 00000080:00200000:0.0:1787826260.088523:0:160559:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=d80a.replay-single, dir=[0x200000007:0x1:0x0](ffff9181d080cb08), intent=getattr 00000002:00010000:0.0:1787826260.088540:0:160559:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: d80a.replay-single,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787826260.088597:0:160559:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513600/0xbef42c0221d6f1ef lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 160559 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826260.088605:0:160559:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513600/0xbef42c0221d6f1ef lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 160559 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826260.088616:0:160559:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513600/0xbef42c0221d6f1ef lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 160559 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826260.088651:0:160559:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:160559:1874663042992256:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787826260.088955:0:160559:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181e00d4f00 going to sleep for 11 seconds 00000100:00100000:0.0:1787826260.112832:0:160559:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:160559:1874663042992256:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787826260.112866:0:160559:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513600/0xbef42c0221d6f1ef lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 160559 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826260.112884:0:160559:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513600/0xbef42c0221d6f1ef lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 160559 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826260.112893:0:160559:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513600/0xbef42c0221d6f1ef lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 160559 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826260.112901:0:160559:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513600/0xbef42c0221d6f1ef lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 160559 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826260.112912:0:160559:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513600/0xbef42c0221d6f1ef lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 160559 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826260.112919:0:160559:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0513600 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787826260.112924:0:160559:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513600/0xbef42c0221d6f1ef lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 160559 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826260.112938:0:160559:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513600/0xbef42c0221d6f1ef lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 160559 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826260.112947:0:160559:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513600/0xbef42c0221d6f1ef lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x20e400000000 pid: 160559 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826260.112956:0:160559:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787826260.112965:0:160559:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513600/0xbef42c0221d6f1ef lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 160559 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826260.112972:0:160559:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513600/0xbef42c0221d6f1ef lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 160559 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826260.113118:0:160559:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c55a5f80 x1874663042992256/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 592/584 e 0 to 0 dl 1787826276 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787826260.113140:0:160559:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1787826260.113178:0:160559:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f1e8 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced92b expref: -99 pid: 159959 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00010000:00010000:0.0:1787826260.113194:0:160559:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f1e8 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced92b expref: -99 pid: 159959 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00000080:00010000:0.0:1787826260.113204:0:160559:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f1e8 from it ffffb97501ddb930 00010000:00010000:0.0:1787826260.113209:0:160559:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f1e8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced92b expref: -99 pid: 159959 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00010000:00010000:0.0:1787826260.113218:0:160559:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f1e8 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced92b expref: -99 pid: 159959 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00000080:00200000:0.0:1787826260.113328:0:160559:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:0.0:1787826260.113347:0:160559:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826260.113354:0:160559:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f1e8 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced92b expref: -99 pid: 159959 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00010000:00010000:0.0:1787826260.113363:0:160559:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f1e8 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced92b expref: -99 pid: 159959 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00000080:00200000:0.0:1787826260.113381:0:160559:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826260.113387:0:160559:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:0.0:1787826260.113391:0:160559:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f1e8 00000080:00010000:0.0:1787826260.113396:0:160559:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f1e8 from it ffffb97501ddbbe8 00010000:00010000:0.0:1787826260.113400:0:160559:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f1e8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced92b expref: -99 pid: 159959 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00010000:00010000:0.0:1787826260.113407:0:160559:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f1e8 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced92b expref: -99 pid: 159959 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00000080:00200000:0.0:1787826260.113416:0:160559:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826260.113531:0:160559:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80a.replay-single, flags=16392 00000080:00200008:0.0:1787826260.113564:0:160559:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181d080cb08), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1787826260.113570:0:160559:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:0.0:1787826260.113579:0:160559:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826260.113586:0:160559:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f1e8 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced92b expref: -99 pid: 159959 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00010000:00010000:0.0:1787826260.113595:0:160559:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f1e8 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced92b expref: -99 pid: 159959 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00000080:00200000:0.0:1787826260.113603:0:160559:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826260.113607:0:160559:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:0.0:1787826260.113610:0:160559:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f1e8 00000080:00010000:0.0:1787826260.113614:0:160559:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f1e8 from it ffffb97501ddbcb8 00010000:00010000:0.0:1787826260.113618:0:160559:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f1e8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced92b expref: -99 pid: 159959 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00010000:00010000:0.0:1787826260.113626:0:160559:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f1e8 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced92b expref: -99 pid: 159959 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00000080:00200008:0.0:1787826260.131026:0:160559:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181d080cb08), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1787826260.131040:0:160559:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:0.0:1787826260.131073:0:160559:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826260.131097:0:160559:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f1e8 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced92b expref: -99 pid: 159959 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00010000:00010000:0.0:1787826260.131116:0:160559:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f1e8 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced92b expref: -99 pid: 159959 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00000080:00200000:0.0:1787826260.131125:0:160559:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826260.131135:0:160559:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:0.0:1787826260.131139:0:160559:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f1e8 00000080:00010000:0.0:1787826260.131149:0:160559:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f1e8 from it ffffb97501ddbcb8 00010000:00010000:0.0:1787826260.131154:0:160559:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f1e8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced92b expref: -99 pid: 159959 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00010000:00010000:0.0:1787826260.131171:0:160559:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f1e8 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced92b expref: -99 pid: 159959 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00000080:00200000:0.0:1787826260.131335:0:160559:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:0.0:1787826260.131345:0:160559:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826260.131353:0:160559:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f1e8 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced92b expref: -99 pid: 159959 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00010000:00010000:0.0:1787826260.131362:0:160559:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f1e8 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced92b expref: -99 pid: 159959 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00000080:00200000:0.0:1787826260.131380:0:160559:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826260.131385:0:160559:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:0.0:1787826260.131387:0:160559:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f1e8 00000080:00010000:0.0:1787826260.131392:0:160559:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f1e8 from it ffffb97501ddbbb0 00010000:00010000:0.0:1787826260.131395:0:160559:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f1e8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced92b expref: -99 pid: 159959 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00010000:00010000:0.0:1787826260.131403:0:160559:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f1e8 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced92b expref: -99 pid: 159959 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00000080:00200000:0.0:1787826260.131412:0:160559:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 44 00000080:00200008:0.0:1787826260.131422:0:160559:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9181d080cb08), flags 300000 00000080:00200080:0.0:1787826260.131444:0:160559:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c03066f0 arg=7fff70f4ad00 00000080:00200000:0.0:1787826260.131464:0:160559:0:(dir.c:451:ll_dir_setdirstripe()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) name=d80a.replay-single stripe_offset=0 stripe_count=1, hash_type=0 00010000:00010000:0.0:1787826260.131501:0:160559:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f1e8 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ced92b expref: -99 pid: 159959 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00010000:00010000:0.0:1787826260.131578:0:160559:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f1e8 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ced92b expref: -99 pid: 159959 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00010000:00010000:0.0:1787826260.131586:0:160559:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826260.131588:0:160559:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f1e8 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ced92b expref: -99 pid: 159959 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00000020:00100000:0.0:1787826260.131618:0:160559:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787826260.131634:0:160559:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c55a5f80 x1874663042992512/t0(0) o36->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 568/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826260.131664:0:160559:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:160559:1874663042992512:192.168.203.147@tcp:36:lfs.0 00000100:00100000:0.0:1787826260.131795:0:160559:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181e00d4f00 going to sleep for 11 seconds 00000100:00100000:0.0:1787826260.137684:0:160559:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:160559:1874663042992512:192.168.203.147@tcp:36:lfs.0 00000080:00200000:0.0:1787826260.137792:0:160559:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200027cd1:0x2:0x0](ffff9181c83f3248) 00000080:00200000:0.0:1787826260.137802:0:160559:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200027cd1:0x2:0x0], updating i_size 4096 00000080:00200008:0.0:1787826260.137841:0:160559:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9181d080cb08), flags 300000 00010000:00010000:2.0:1787826260.589932:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200000:0.0:1787826260.941606:0:160572:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:0.0:1787826260.941639:0:160572:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826260.941696:0:160572:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 160572 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826260.941719:0:160572:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 160572 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826260.941731:0:160572:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 160572 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826260.942155:0:160572:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:160572:1874663042992640:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787826260.942338:0:160572:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181e00d49c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826260.961524:0:160572:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:160572:1874663042992640:192.168.203.147@tcp:101:lfs.0 00010000:00010000:2.0:1787826260.961554:0:160572:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced963 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826260.961566:0:160572:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced963 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826260.961577:0:160572:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced963 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826260.961588:0:160572:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced963 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826260.961600:0:160572:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced963 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826260.961608:0:160572:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced963 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826260.961616:0:160572:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced963 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787826260.961628:0:160572:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c55a7b80 x1874663042992640/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826276 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787826260.961642:0:160572:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced963 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826260.961656:0:160572:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d1a77c00 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:2.0:1787826260.961663:0:160572:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:2.0:1787826260.961687:0:160572:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 69632 00000080:00010000:2.0:1787826260.961693:0:160572:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f1f6 00000080:00010000:2.0:1787826260.961698:0:160572:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:2.0:1787826260.961701:0:160572:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f1f6 from it ffffb975014fbb58 00010000:00010000:2.0:1787826260.961704:0:160572:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced963 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826260.961714:0:160572:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced963 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826260.961739:0:160572:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:2.0:1787826260.961766:0:160572:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=d80a.replay-single, dir=[0x200000007:0x1:0x0](ffff9181d080cb08), flags=29 00000080:00200000:2.0:1787826260.961771:0:160572:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=d80a.replay-single, dir=[0x200000007:0x1:0x0](ffff9181d080cb08), intent=0 00000002:00010000:2.0:1787826260.961789:0:160572:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: d80a.replay-single,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: lookup flags 0 00010000:00010000:2.0:1787826260.961826:0:160572:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7000/0xbef42c0221d6f1fd lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 160572 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826260.961834:0:160572:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7000/0xbef42c0221d6f1fd lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 160572 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826260.961841:0:160572:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7000/0xbef42c0221d6f1fd lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 160572 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826260.961860:0:160572:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f2d80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:160572:1874663042992768:192.168.203.147@tcp:101:lfs.0 00000100:00100000:2.0:1787826260.961938:0:160572:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c52569c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826260.978639:0:160572:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f2d80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:160572:1874663042992768:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787826260.978666:0:160572:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7000/0xbef42c0221d6f1fd lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced97f expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826260.978687:0:160572:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7000/0xbef42c0221d6f1fd lrc: 4/1,0 mode: --/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced97f expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826260.978696:0:160572:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7000/0xbef42c0221d6f1fd lrc: 4/1,0 mode: --/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced97f expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826260.978707:0:160572:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7000/0xbef42c0221d6f1fd lrc: 4/1,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced97f expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826260.978718:0:160572:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7000/0xbef42c0221d6f1fd lrc: 4/1,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced97f expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826260.978730:0:160572:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7000/0xbef42c0221d6f1fd lrc: 5/2,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced97f expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826260.978739:0:160572:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7000/0xbef42c0221d6f1fd lrc: 5/2,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced97f expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826260.978765:0:160572:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7000/0xbef42c0221d6f1fd lrc: 4/1,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced97f expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826260.978782:0:160572:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x10 disposition=0xb, status=0 req@ffff9181c71f2d80 x1874663042992768/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 592/688 e 0 to 0 dl 1787826276 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787826260.978796:0:160572:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7000/0xbef42c0221d6f1fd lrc: 3/1,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced97f expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826260.978807:0:160572:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d1a77c00 type 13 mode 4 res 8590097617/2 (0 0) 00000080:00200000:0.0:1787826260.978813:0:160572:0:(namei.c:952:ll_intent_lock()) intent lock 16 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:0.0:1787826260.978851:0:160572:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200027cd1:0x2:0x0], updating i_size 4096 00000080:00200000:0.0:1787826260.978855:0:160572:0:(namei.c:147:ll_iget()) got inode: [0x200027cd1:0x2:0x0](ffff9181c83f3248): rc = 0 00000080:00010000:0.0:1787826260.978860:0:160572:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200027cd1:0x2:0x0](ffff9181c83f3248) for lock 0xbef42c0221d6f1fd 00000080:00010000:0.0:1787826260.978871:0:160572:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200027cd1:0x2:0x0](ffff9181c83f3248) 00000080:00010000:0.0:1787826260.978874:0:160572:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f1fd from it ffffb975014fba28 00010000:00010000:0.0:1787826260.978877:0:160572:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7000/0xbef42c0221d6f1fd lrc: 3/1,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced97f expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826260.978886:0:160572:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7000/0xbef42c0221d6f1fd lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced97f expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826260.978921:0:160572:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x2:0x0](ffff9181c83f3248), inode mode 41ed mask 1 00000080:00200000:0.0:1787826260.978936:0:160572:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=remote_dir, dir=[0x200027cd1:0x2:0x0](ffff9181c83f3248), flags=13 00000080:00200000:0.0:1787826260.978941:0:160572:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=remote_dir, dir=[0x200027cd1:0x2:0x0](ffff9181c83f3248), intent=getattr 00000002:00010000:0.0:1787826260.978961:0:160572:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: remote_dir,[0x0:0x0:0x0]) in obj [0x200027cd1:0x2:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787826260.979012:0:160572:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f204 lrc: 3/1,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 160572 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826260.979018:0:160572:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f204 lrc: 3/1,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 160572 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826260.979028:0:160572:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f204 lrc: 3/1,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 160572 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826260.979046:0:160572:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:160572:1874663042992896:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787826260.979493:0:160572:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181e00d4780 going to sleep for 11 seconds 00000100:00100000:2.0:1787826260.996159:0:160572:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:160572:1874663042992896:192.168.203.147@tcp:101:lfs.0 00010000:00010000:2.0:1787826260.996185:0:160572:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f204 lrc: 4/1,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 160572 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826260.996200:0:160572:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f204 lrc: 4/1,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 160572 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826260.996208:0:160572:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f204 lrc: 4/1,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 160572 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826260.996214:0:160572:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f204 lrc: 3/0,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 160572 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826260.996223:0:160572:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f204 lrc: 4/0,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 160572 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826260.996229:0:160572:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0513200 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:2.0:1787826260.996234:0:160572:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f204 lrc: 4/0,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 160572 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826260.996241:0:160572:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f204 lrc: 4/0,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 160572 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826260.996249:0:160572:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f204 lrc: 5/0,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x20e400000000 pid: 160572 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826260.996255:0:160572:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787826260.996261:0:160572:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f204 lrc: 3/0,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 160572 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826260.996268:0:160572:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f204 lrc: 0/0,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 160572 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787826260.996283:0:160572:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c55a6680 x1874663042992896/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 584/584 e 0 to 0 dl 1787826276 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1787826260.996303:0:160572:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200027cd1:0x2:0x0] suppgids 0 -1: rc 0 00010000:00010000:2.0:1787826260.996422:0:160572:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d1a77c00 type 13 mode 30 res 8590097617/2 (0 0) 00000080:00200000:2.0:1787826260.996481:0:160572:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:2.0:1787826260.996495:0:160572:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826260.996505:0:160572:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced963 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826260.996515:0:160572:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced963 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826260.996522:0:160572:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826260.996530:0:160572:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:2.0:1787826260.996534:0:160572:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f1f6 00000080:00010000:2.0:1787826260.996538:0:160572:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f1f6 from it ffffb975014fbbe8 00010000:00010000:2.0:1787826260.996542:0:160572:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced963 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826260.996550:0:160572:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced963 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1787826260.996564:0:160572:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:2.0:1787826260.996571:0:160572:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80a.replay-single, flags=16408 00000080:00200000:2.0:1787826260.996676:0:160572:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x2:0x0](ffff9181c83f3248), inode mode 41ed mask 1 00000080:00200000:2.0:1787826260.996688:0:160572:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=remote_dir, dir=[0x200027cd1:0x2:0x0](ffff9181c83f3248), flags=16392 00000080:00200000:2.0:1787826260.996692:0:160572:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=remote_dir, dir=[0x200027cd1:0x2:0x0](ffff9181c83f3248), intent=getattr 00000002:00010000:2.0:1787826260.996706:0:160572:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: remote_dir,[0x0:0x0:0x0]) in obj [0x200027cd1:0x2:0x0], intent: getattr flags 01 00010000:00010000:2.0:1787826260.996767:0:160572:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6200/0xbef42c0221d6f20b lrc: 3/1,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 160572 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826260.996776:0:160572:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6200/0xbef42c0221d6f20b lrc: 3/1,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 160572 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826260.996889:0:160572:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6200/0xbef42c0221d6f20b lrc: 3/1,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 160572 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826260.996908:0:160572:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f3100 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:160572:1874663042993024:192.168.203.147@tcp:101:lfs.0 00000100:00100000:2.0:1787826260.996979:0:160572:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5256c00 going to sleep for 11 seconds 00000100:00100000:3.0:1787826261.004169:0:160572:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f3100 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:160572:1874663042993024:192.168.203.147@tcp:101:lfs.0 00010000:00010000:3.0:1787826261.004202:0:160572:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6200/0xbef42c0221d6f20b lrc: 4/1,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 160572 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826261.004218:0:160572:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6200/0xbef42c0221d6f20b lrc: 4/1,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 160572 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826261.004227:0:160572:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6200/0xbef42c0221d6f20b lrc: 4/1,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 160572 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826261.004236:0:160572:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6200/0xbef42c0221d6f20b lrc: 3/0,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 160572 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826261.004246:0:160572:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6200/0xbef42c0221d6f20b lrc: 4/0,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 160572 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826261.004254:0:160572:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e06f6200 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:3.0:1787826261.004259:0:160572:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6200/0xbef42c0221d6f20b lrc: 4/0,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 160572 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826261.004268:0:160572:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6200/0xbef42c0221d6f20b lrc: 4/0,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 160572 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826261.004276:0:160572:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6200/0xbef42c0221d6f20b lrc: 5/0,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x20e400000000 pid: 160572 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826261.004285:0:160572:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826261.004291:0:160572:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6200/0xbef42c0221d6f20b lrc: 3/0,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 160572 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826261.004299:0:160572:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6200/0xbef42c0221d6f20b lrc: 0/0,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 160572 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826261.004325:0:160572:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c71f3100 x1874663042993024/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 584/584 e 0 to 0 dl 1787826276 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:3.0:1787826261.004340:0:160572:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200027cd1:0x2:0x0] suppgids 0 -1: rc 0 00010000:00010000:3.0:1787826261.004376:0:160572:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d1a77c00 type 13 mode 30 res 8590097617/2 (0 0) 00000080:00200000:3.0:1787826261.004449:0:160572:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:3.0:1787826261.004464:0:160572:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826261.004473:0:160572:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced963 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826261.004483:0:160572:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced963 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:3.0:1787826261.004489:0:160572:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826261.004494:0:160572:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:3.0:1787826261.004497:0:160572:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f1f6 00000080:00010000:3.0:1787826261.004502:0:160572:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f1f6 from it ffffb975014fbb58 00010000:00010000:3.0:1787826261.004505:0:160572:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced963 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826261.004512:0:160572:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced963 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:3.0:1787826261.004527:0:160572:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826261.004536:0:160572:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80a.replay-single, flags=13 00000080:00200008:3.0:1787826261.004546:0:160572:0:(file.c:6296:ll_getattr_dentry()) START file d80a.replay-single:[0x200027cd1:0x2:0x0](ffff9181c83f3248), request_mask 0, flags 0, foreign 0 00000080:00200000:3.0:1787826261.004551:0:160572:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200027cd1:0x2:0x0](ffff9181c83f3248),name=d80a.replay-single 00000002:00010000:3.0:1787826261.004564:0:160572:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200027cd1:0x2:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826261.004568:0:160572:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d1a77c00 type 13 mode 30 res 8590097617/2 (0 0) 00010000:00010000:3.0:1787826261.004600:0:160572:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f212 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 160572 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826261.004607:0:160572:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f212 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 160572 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826261.004615:0:160572:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f212 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 160572 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826261.004633:0:160572:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:160572:1874663042993152:192.168.203.147@tcp:101:lfs.0 00000100:00100000:3.0:1787826261.004705:0:160572:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1801840 going to sleep for 11 seconds 00000100:00100000:2.0:1787826261.015800:0:160572:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:160572:1874663042993152:192.168.203.147@tcp:101:lfs.0 00010000:00010000:2.0:1787826261.015830:0:160572:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f212 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced9b7 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826261.015847:0:160572:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f212 lrc: 4/1,0 mode: --/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced9b7 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826261.015855:0:160572:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f212 lrc: 4/1,0 mode: --/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced9b7 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826261.015865:0:160572:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f212 lrc: 4/1,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced9b7 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826261.015876:0:160572:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f212 lrc: 4/1,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced9b7 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826261.015889:0:160572:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f212 lrc: 5/2,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced9b7 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826261.015903:0:160572:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f212 lrc: 5/2,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced9b7 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826261.015909:0:160572:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f212 lrc: 4/1,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced9b7 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787826261.015920:0:160572:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c5634a80 x1874663042993152/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826277 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787826261.015932:0:160572:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f212 lrc: 3/1,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced9b7 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826261.015941:0:160572:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d1a77c00 type 13 mode 4 res 8590097617/2 (0 0) 00000080:00200000:2.0:1787826261.015947:0:160572:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:2.0:1787826261.015973:0:160572:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200027cd1:0x2:0x0], updating i_size 4096 00000080:00010000:2.0:1787826261.015979:0:160572:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200027cd1:0x2:0x0](ffff9181c83f3248) for lock 0xbef42c0221d6f212 00000080:00010000:2.0:1787826261.015986:0:160572:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200027cd1:0x2:0x0](ffff9181c83f3248) 00000080:00010000:2.0:1787826261.015989:0:160572:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f212 from it ffffb975014fbcb8 00010000:00010000:2.0:1787826261.015992:0:160572:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f212 lrc: 3/1,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced9b7 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826261.016000:0:160572:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f212 lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced9b7 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:2.0:1787826261.016551:0:160572:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181d080cb08), request_mask 0, flags 0, foreign 0 00000080:00200000:2.0:1787826261.016557:0:160572:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:2.0:1787826261.016573:0:160572:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826261.016583:0:160572:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced963 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1787826261.016592:0:160572:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced963 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:2.0:1787826261.016599:0:160572:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826261.016604:0:160572:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:2.0:1787826261.016607:0:160572:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f1f6 00000080:00010000:2.0:1787826261.016611:0:160572:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f1f6 from it ffffb975014fbcb8 00010000:00010000:2.0:1787826261.016614:0:160572:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced963 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1787826261.017181:0:160572:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced963 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00000080:00200000:2.0:1787826261.017313:0:160572:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:2.0:1787826261.017325:0:160572:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826261.017334:0:160572:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced963 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:2.0:1787826261.017343:0:160572:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced963 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00000080:00200000:2.0:1787826261.017350:0:160572:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826261.017354:0:160572:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:2.0:1787826261.017357:0:160572:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f1f6 00000080:00010000:2.0:1787826261.017370:0:160572:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f1f6 from it ffffb975014fbbf8 00010000:00010000:2.0:1787826261.017374:0:160572:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced963 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:2.0:1787826261.017381:0:160572:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced963 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00000080:00200000:2.0:1787826261.017391:0:160572:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:2.0:1787826261.017397:0:160572:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80a.replay-single, flags=267 00000080:00200000:2.0:1787826261.017402:0:160572:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x2:0x0](ffff9181c83f3248), inode mode 41ed mask 44 00000080:00200008:2.0:1787826261.017408:0:160572:0:(file.c:1090:ll_file_open()) START file d80a.replay-single:[0x200027cd1:0x2:0x0](ffff9181c83f3248), flags 300000 00000002:00010000:2.0:1787826261.017422:0:160572:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200027cd1:0x2:0x0]) in obj [0x200000007:0x1:0x0], intent: open flags 0240300001 00010000:00010000:2.0:1787826261.017469:0:160572:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f219 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 160572 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826261.017477:0:160572:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f219 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 160572 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1787826261.017488:0:160572:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1787826261.017497:0:160572:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c71f3b80 x1874663042993408/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/131856 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1787826261.017508:0:160572:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f219 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 160572 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826261.017529:0:160572:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f3b80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:160572:1874663042993408:192.168.203.147@tcp:101:lfs.0 00000100:00100000:2.0:1787826261.017702:0:160572:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5256540 going to sleep for 11 seconds 00000100:00100000:3.0:1787826261.024995:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:f40147eb-9b53-47d4-b77c-99ceedd9664c:2392:1874663042993536:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826261.025058:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663042993792:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826261.025079:0:2392:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1787826261.025082:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663042993664:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826261.025094:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663042993920:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826261.025183:0:160572:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f3b80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:160572:1874663042993408:192.168.203.147@tcp:101:lfs.0 00010000:00010000:2.0:1787826261.025211:0:160572:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f219 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 160572 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826261.025230:0:160572:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f219 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 160572 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826261.025237:0:160572:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f219 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 160572 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826261.025249:0:160572:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f219 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 160572 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826261.025259:0:160572:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f219 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 160572 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826261.025265:0:160572:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e06f6400 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:2.0:1787826261.025269:0:160572:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f219 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x206400000000 pid: 160572 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826261.025276:0:160572:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f219 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x206400000000 pid: 160572 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826261.025283:0:160572:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f219 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x20e400000000 pid: 160572 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826261.025290:0:160572:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787826261.025297:0:160572:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f219 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 160572 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826261.025306:0:160572:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f219 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 160572 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787826261.025324:0:160572:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x1 disposition=0x2b, status=0 req@ffff9181c71f3b80 x1874663042993408/t347892350982(347892350982) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826277 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00100000:2.0:1787826261.025343:0:160572:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c71f3b80 x1874663042993408/t347892350982(347892350982) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826277 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1787826261.025357:0:160572:0:(namei.c:952:ll_intent_lock()) intent lock 1 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787826261.025914:0:160572:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200027cd1:0x2:0x0], updating i_size 4096 00000002:00100000:1.0:1787826261.025940:0:160572:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c71f3b80 x1874663042993408/t347892350982(347892350982) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 0 dl 1787826277 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200080:1.0:1787826261.026012:0:160572:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200027cd1:0x2:0x0](ffff9181c83f3248) cmd=c03066f0 arg=7ffc71d577b0 00000080:00200000:1.0:1787826261.026027:0:160572:0:(dir.c:451:ll_dir_setdirstripe()) VFS Op:inode=[0x200027cd1:0x2:0x0](ffff9181c83f3248) name=remote_dir stripe_offset=1 stripe_count=0, hash_type=0 00000100:00100000:1.0:1787826261.026073:0:160572:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:160572:1874663042994048:192.168.203.147@tcp:700:lfs.0 00000100:00100000:1.0:1787826261.026285:0:160572:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8180 going to sleep for 11 seconds 00000100:00100000:3.0:1787826261.035951:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663042993664:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826261.035967:0:160572:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:160572:1874663042994048:192.168.203.147@tcp:700:lfs.0 00000100:00100000:3.0:1787826261.035974:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663042993920:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826261.035986:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:f40147eb-9b53-47d4-b77c-99ceedd9664c:2392:1874663042993536:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826261.035998:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663042993792:192.168.203.147@tcp:400:kworker.0 00000020:00100000:2.0:1787826261.036038:0:160572:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787826261.036048:0:160572:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c71f1f80 x1874663042994304/t0(0) o36->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 560/520 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826261.036070:0:160572:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f1f80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:160572:1874663042994304:192.168.203.147@tcp:36:lfs.0 00000100:00100000:2.0:1787826261.036499:0:160572:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5256540 going to sleep for 11 seconds 00000100:00100000:1.0:1787826261.052799:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826261.052905:0:159722:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049352704 00000100:00100000:0.0:1787826261.052917:0:159722:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049352704 00000100:00100000:0.0:1787826261.052985:0:159722:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 2 00000100:00100000:0.0:1787826261.052996:0:159722:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:c0d7758a-4501-4a89-8ec1-407fef4f0c7f+5:8415:x1874663049352704:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826261.053014:0:159722:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f212 lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ced9b7 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826261.053091:0:159722:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826261.053097:0:159722:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:c0d7758a-4501-4a89-8ec1-407fef4f0c7f+5:8415:x1874663049352704:12345-192.168.203.147@tcp:104: Request processed in 103us (310us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826261.053104:0:159722:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 2 00010000:00010000:3.0:1787826261.053387:0:159732:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826261.053395:0:159732:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f212 lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ced9b7 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826261.053403:0:159732:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3480800 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:3.0:1787826261.053408:0:159732:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f212 lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x13 rrc: 3 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00ced9b7 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826261.053417:0:159732:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f212 lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x13 rrc: 3 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00ced9b7 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826261.053425:0:159732:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f212 lrc: 3/0,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x13 rrc: 3 type: IBT gid 0 flags: 0x408400000000 nid: local remote: 0x878a5eea00ced9b7 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826261.053451:0:159732:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7000/0xbef42c0221d6f1fd lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced97f expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826261.053493:0:159732:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f212 lrc: 2/0,0 mode: --/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x13 rrc: 3 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ced9b7 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826261.053499:0:159732:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826261.053529:0:159732:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f212 lrc: 1/0,0 mode: --/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x13 rrc: 3 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ced9b7 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826261.053537:0:159732:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f212 lrc: 0/0,0 mode: --/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x13 rrc: 3 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ced9b7 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826261.053551:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826261.053553:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826261.053555:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826261.053563:0:2391:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787826261.053578:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663042994432:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1787826261.064840:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663042994432:192.168.203.147@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826261.601261:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200008:1.0:1787826262.042636:0:160582:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181d080cb08), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1787826262.042654:0:160582:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:1.0:1787826262.042677:0:160582:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826262.042695:0:160582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced963 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00010000:00010000:1.0:1787826262.042710:0:160582:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced963 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00000080:00200000:1.0:1787826262.042719:0:160582:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826262.042726:0:160582:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:1.0:1787826262.042729:0:160582:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f1f6 00000080:00010000:1.0:1787826262.043069:0:160582:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f1f6 from it ffffb975015ebcb8 00010000:00010000:1.0:1787826262.043075:0:160582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced963 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00010000:00010000:1.0:1787826262.043086:0:160582:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced963 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00000080:00200008:1.0:1787826262.043674:0:160582:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181d080cb08), request_mask 0, flags 0, foreign 0 00000080:00200000:1.0:1787826262.043680:0:160582:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:1.0:1787826262.043691:0:160582:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826262.043700:0:160582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced963 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00010000:00010000:1.0:1787826262.043710:0:160582:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced963 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00000080:00200000:1.0:1787826262.043722:0:160582:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826262.043727:0:160582:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:1.0:1787826262.043730:0:160582:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f1f6 00000080:00010000:1.0:1787826262.044019:0:160582:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f1f6 from it ffffb975015ebcb8 00010000:00010000:1.0:1787826262.044024:0:160582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced963 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00010000:00010000:1.0:1787826262.044033:0:160582:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced963 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00000080:00200000:1.0:1787826262.044128:0:160582:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:1.0:1787826262.044137:0:160582:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826262.044145:0:160582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced963 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00010000:00010000:1.0:1787826262.044154:0:160582:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced963 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00000080:00200000:1.0:1787826262.044161:0:160582:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826262.044165:0:160582:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:1.0:1787826262.044168:0:160582:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f1f6 00000080:00010000:1.0:1787826262.044172:0:160582:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f1f6 from it ffffb975015ebbb0 00010000:00010000:1.0:1787826262.044176:0:160582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced963 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00010000:00010000:1.0:1787826262.044184:0:160582:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced963 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00000080:00200000:1.0:1787826262.044194:0:160582:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 44 00000080:00200008:1.0:1787826262.044201:0:160582:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9181d080cb08), flags 100000 00000080:00200080:1.0:1787826262.045486:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:1.0:1787826262.045513:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053cc00 uarg=0000000000000000 00800000:00000004:1.0:1787826262.045521:0:160582:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9181c5620000: age 8325, max_age 8332 00000100:00100000:1.0:1787826262.045568:0:160582:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:160582:1874663042994560:192.168.203.147@tcp:41:lfs.0 00000100:00100000:1.0:1787826262.046335:0:160582:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c83c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826262.051347:0:160582:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9181c5620000: committing for last_committed 347892350982 gen 1 00000100:00100000:2.0:1787826262.051401:0:160582:0:(client.c:3174:ptlrpc_free_committed()) @@@ keeping (FL_REPLAY) req@ffff9181c71f3b80 x1874663042993408/t347892350982(347892350982) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 0 dl 1787826277 ref 1 fl Complete:RPQU/604/ffffffff rc 301/-1 job:'lfs.0' uid:0 gid:0 projid:0 00000100:00100000:2.0:1787826262.051417:0:160582:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:160582:1874663042994560:192.168.203.147@tcp:41:lfs.0 00800000:00000004:2.0:1787826262.051436:0:160582:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0000-mdc-ffff9181c5620000: update ffff9181c354d178 cache blocks 317233/353529 objects 1023645/1024000 00000080:00200080:2.0:1787826262.051524:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.051543:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00800000:00000004:2.0:1787826262.051551:0:160582:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9181c5620000: age 8325, max_age 8332 00000100:00100000:2.0:1787826262.051592:0:160582:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f0e00 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:160582:1874663042994688:192.168.203.147@tcp:41:lfs.0 00000100:00100000:2.0:1787826262.051668:0:160582:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5256c00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826262.068438:0:160582:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff9181c5620000: committing for last_committed 17179869984 gen 1 00000100:00100000:2.0:1787826262.068460:0:160582:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f0e00 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:160582:1874663042994688:192.168.203.147@tcp:41:lfs.0 00800000:00000004:2.0:1787826262.068501:0:160582:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0001-mdc-ffff9181c5620000: update ffff9181c3548450 cache blocks 317323/353529 objects 1023702/1024000 00000080:00200080:2.0:1787826262.068571:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.068586:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.068594:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.068598:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.068602:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.068606:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.068610:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.068614:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.068627:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.068631:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.068635:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.068638:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.068641:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.068644:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.068647:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.068650:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.068653:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.068655:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.068658:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.068661:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.068664:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.068666:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.068673:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.068676:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.068680:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.068683:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.068686:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.068689:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.068692:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.068695:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.068698:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.068702:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.068706:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.068710:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.068713:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.068716:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.068720:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.068723:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.068727:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.068731:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.069047:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.069054:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.069058:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.069062:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.069065:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.069069:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.069073:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.069076:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.069080:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.069085:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.069088:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.069094:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.069097:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.069101:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.069104:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.069107:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.069111:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.069113:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.069117:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.069121:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.069124:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.069128:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.069132:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.069134:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.069138:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.069141:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.069145:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.069148:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.069152:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.069155:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.069159:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.069162:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.069165:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.069168:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.069172:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.069175:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.069178:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.069181:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.069185:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.069187:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.069191:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.069193:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.069197:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.069200:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.069203:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.069206:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.069209:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.069213:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.069216:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.069218:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.069223:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.069227:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.069231:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.069234:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.069237:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.069240:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.069243:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.069247:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.069251:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.069254:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.069257:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.069260:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.069264:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.069268:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.069271:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.069274:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.069277:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.069281:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.069284:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.069287:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.069291:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.069295:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.069299:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.069302:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.069305:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.069308:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.069312:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.069315:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.069318:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.069322:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.069325:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.069328:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.069332:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.069335:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.069339:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00800000:00000080:2.0:1787826262.069342:0:160582:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.069347:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.069366:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00020000:00000004:2.0:1787826262.069375:0:160582:0:(obd_class.h:1090:obd_statfs()) lustre-OST0000-osc-ffff9181c5620000: age 8325, max_age 8332 00000100:00100000:2.0:1787826262.069409:0:160582:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f0e00 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:160582:1874663042994816:192.168.203.147@tcp:13:lfs.0 00000100:00100000:2.0:1787826262.069569:0:160582:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5256c00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826262.077445:0:160582:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f0e00 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:160582:1874663042994816:192.168.203.147@tcp:13:lfs.0 00020000:00000004:2.0:1787826262.077479:0:160582:0:(obd_class.h:1116:obd_statfs()) lustre-OST0000-osc-ffff9181c5620000: update ffff9181c3549e08 cache blocks 901356/958279 objects 261711/262144 00000080:00200080:2.0:1787826262.077577:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.077594:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00020000:00000004:2.0:1787826262.077601:0:160582:0:(obd_class.h:1090:obd_statfs()) lustre-OST0001-osc-ffff9181c5620000: age 8325, max_age 8332 00000100:00100000:2.0:1787826262.077648:0:160582:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f0e00 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:160582:1874663042994944:192.168.203.147@tcp:13:lfs.0 00000100:00100000:2.0:1787826262.077740:0:160582:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5256c00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826262.081551:0:160582:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f0e00 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:160582:1874663042994944:192.168.203.147@tcp:13:lfs.0 00020000:00000004:2.0:1787826262.081583:0:160582:0:(obd_class.h:1116:obd_statfs()) lustre-OST0001-osc-ffff9181c5620000: update ffff9181c354b7c0 cache blocks 901360/958279 objects 261727/262144 00000080:00200080:2.0:1787826262.081880:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.081898:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.081905:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.081910:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.081914:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.081918:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.081922:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.081925:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.081930:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.081934:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.081938:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.081941:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.081945:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.081948:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.081952:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.081955:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.081959:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.081963:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.081967:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.081970:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.081974:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.081977:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.081982:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.081985:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.081989:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.081992:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.081996:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.081999:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.082003:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.082006:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.082010:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.082014:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.082018:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.082022:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.082026:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.082029:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.082032:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.082035:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.082039:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.082042:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.082046:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.082049:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.082052:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.082056:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.082060:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.082064:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.082067:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.082081:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.082087:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.082091:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.082095:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.082098:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.082102:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.082105:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.082109:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.082112:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.082116:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.082119:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.082123:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.082127:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.083241:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.083262:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.083268:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.083272:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.083275:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.083278:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.083282:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.083284:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.083288:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.083291:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.083294:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.083297:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.083301:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.083304:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.083308:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.083312:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.085460:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.085489:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.085498:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.085502:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.085506:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.085510:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.085514:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.085519:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.085522:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.085527:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.085531:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.085534:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.085538:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.085541:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.085544:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.085548:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.085552:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.085555:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.085559:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.085562:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.085566:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.085569:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.085573:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.085577:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.085581:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.085584:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.085589:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.085592:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.085596:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.085600:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.085604:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.085607:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.085611:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.085614:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.085618:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.085622:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.085625:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.085628:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.085632:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.085635:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.085639:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.085641:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.085645:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.085648:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.085652:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.085655:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.085659:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.085662:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200080:2.0:1787826262.085666:0:160582:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff254b44d0 00020000:00000080:2.0:1787826262.085669:0:160582:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2400 uarg=0000000000000000 00000080:00200008:2.0:1787826262.085684:0:160582:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9181d080cb08), flags 100000 00000080:00200000:3.0:1787826262.173263:0:160589:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:3.0:1787826262.173295:0:160589:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826262.173314:0:160589:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced963 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00010000:00010000:3.0:1787826262.173331:0:160589:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced963 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00000080:00200000:3.0:1787826262.173814:0:160589:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826262.173824:0:160589:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:3.0:1787826262.173828:0:160589:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f1f6 00000080:00010000:3.0:1787826262.173836:0:160589:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f1f6 from it ffffb975019cbbd0 00010000:00010000:3.0:1787826262.173841:0:160589:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced963 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00010000:00010000:3.0:1787826262.173922:0:160589:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced963 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00000080:00200000:3.0:1787826262.173940:0:160589:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826262.173957:0:160589:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=fsa-oleg347-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9181d080cb08), flags=1536 00000080:00200000:3.0:1787826262.173960:0:160589:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:3.0:1787826262.173968:0:160589:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826262.173976:0:160589:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced963 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00010000:00010000:3.0:1787826262.173990:0:160589:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced963 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00000080:00200000:3.0:1787826262.173997:0:160589:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826262.174001:0:160589:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:3.0:1787826262.174004:0:160589:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f1f6 00000080:00010000:3.0:1787826262.174007:0:160589:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f1f6 from it ffffb975019cbcd0 00010000:00010000:3.0:1787826262.174010:0:160589:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced963 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00010000:00010000:3.0:1787826262.174017:0:160589:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced963 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00000080:00200000:3.0:1787826262.174026:0:160589:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 3 00000080:00200000:3.0:1787826262.174031:0:160589:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:3.0:1787826262.174037:0:160589:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826262.174043:0:160589:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced963 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00010000:00010000:3.0:1787826262.174050:0:160589:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced963 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00000080:00200000:3.0:1787826262.174056:0:160589:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826262.174059:0:160589:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:3.0:1787826262.174062:0:160589:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f1f6 00000080:00010000:3.0:1787826262.174065:0:160589:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f1f6 from it ffffb975019cbdb0 00010000:00010000:3.0:1787826262.174069:0:160589:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced963 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00010000:00010000:3.0:1787826262.174075:0:160589:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced963 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 11 lru_type: 1 00000080:00200000:3.0:1787826262.174083:0:160589:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 3 00000080:00200000:3.0:1787826262.174088:0:160589:0:(namei.c:2129:ll_create_nd()) VFS Op:name=fsa-oleg347-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9181d080cb08), flags=33188, excl=1 00000080:00200000:3.0:1787826262.174092:0:160589:0:(namei.c:2084:ll_mknod()) VFS Op:name=fsa-oleg347-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9181d080cb08) mode 100644 dev 0 00010000:00010000:3.0:1787826262.174116:0:160589:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ced963 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 11 lru_type: 1 00010000:00010000:3.0:1787826262.174162:0:160589:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ced963 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 11 lru_type: 1 00010000:00010000:3.0:1787826262.174169:0:160589:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826262.174172:0:160589:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0512200/0xbef42c0221d6f1f6 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ced963 expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 11 lru_type: 1 00000020:00100000:3.0:1787826262.174195:0:160589:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:3.0:1787826262.174207:0:160589:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c5636300 x1874663042995200/t0(0) o36->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 528/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826262.174230:0:160589:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636300 pname:cluuid:pid:xid:nid:opc:job mcreate:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:160589:1874663042995200:192.168.203.147@tcp:36:mcreate.0 00000100:00100000:3.0:1787826262.174313:0:160589:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1801000 going to sleep for 11 seconds 00000100:00100000:3.0:1787826262.183258:0:160589:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636300 pname:cluuid:pid:xid:nid:opc:job mcreate:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:160589:1874663042995200:192.168.203.147@tcp:36:mcreate.0 00000080:00200000:3.0:1787826262.183310:0:160589:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200027cd1:0x3:0x0](ffff9181d0808908) 00000080:00200000:3.0:1787826262.183361:0:160589:0:(namei.c:2137:ll_create_nd()) VFS Op:name=fsa-oleg347-client.virtnet, unhashed 1 00000080:00200000:2.0:1787826262.201199:0:160592:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:2.0:1787826262.201229:0:160592:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826262.201287:0:160592:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7a00/0xbef42c0221d6f220 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 160592 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.201297:0:160592:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7a00/0xbef42c0221d6f220 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 160592 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.201312:0:160592:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7a00/0xbef42c0221d6f220 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 160592 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826262.201406:0:160592:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f0e00 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:160592:1874663042995328:192.168.203.147@tcp:101:rm.0 00000100:00100000:2.0:1787826262.201538:0:160592:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5256c00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826262.206900:0:160592:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f0e00 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:160592:1874663042995328:192.168.203.147@tcp:101:rm.0 00010000:00010000:2.0:1787826262.206929:0:160592:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7a00/0xbef42c0221d6f220 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced9fd expref: -99 pid: 160592 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.206941:0:160592:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7a00/0xbef42c0221d6f220 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced9fd expref: -99 pid: 160592 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.206951:0:160592:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7a00/0xbef42c0221d6f220 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced9fd expref: -99 pid: 160592 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.206963:0:160592:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7a00/0xbef42c0221d6f220 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced9fd expref: -99 pid: 160592 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.206975:0:160592:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7a00/0xbef42c0221d6f220 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced9fd expref: -99 pid: 160592 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.206983:0:160592:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7a00/0xbef42c0221d6f220 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced9fd expref: -99 pid: 160592 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.206990:0:160592:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7a00/0xbef42c0221d6f220 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced9fd expref: -99 pid: 160592 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787826262.207007:0:160592:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c71f0e00 x1874663042995328/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826278 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787826262.207020:0:160592:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7a00/0xbef42c0221d6f220 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced9fd expref: -99 pid: 160592 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.207034:0:160592:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d1a77c00 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:2.0:1787826262.207040:0:160592:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:2.0:1787826262.207067:0:160592:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 69632 00000080:00010000:2.0:1787826262.207074:0:160592:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f220 00000080:00010000:2.0:1787826262.207079:0:160592:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:2.0:1787826262.207082:0:160592:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f220 from it ffffb9750181bb58 00010000:00010000:2.0:1787826262.207085:0:160592:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7a00/0xbef42c0221d6f220 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced9fd expref: -99 pid: 160592 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.207093:0:160592:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7a00/0xbef42c0221d6f220 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced9fd expref: -99 pid: 160592 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826262.207118:0:160592:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:2.0:1787826262.207148:0:160592:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=fsa-oleg347-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9181d080cb08), flags=8 00000080:00200000:2.0:1787826262.207153:0:160592:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=fsa-oleg347-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9181d080cb08), intent=getattr 00000002:00010000:2.0:1787826262.207172:0:160592:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: fsa-oleg347-client.virtnet,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 01 00010000:00010000:2.0:1787826262.207211:0:160592:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7c00/0xbef42c0221d6f227 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 160592 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.207218:0:160592:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7c00/0xbef42c0221d6f227 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 160592 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.207229:0:160592:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7c00/0xbef42c0221d6f227 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 160592 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826262.207253:0:160592:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f0e00 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:160592:1874663042995456:192.168.203.147@tcp:101:rm.0 00000100:00100000:2.0:1787826262.207325:0:160592:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5256c00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826262.211684:0:160592:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f0e00 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:160592:1874663042995456:192.168.203.147@tcp:101:rm.0 00010000:00010000:2.0:1787826262.211723:0:160592:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7c00/0xbef42c0221d6f227 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceda19 expref: -99 pid: 160592 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.211777:0:160592:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7c00/0xbef42c0221d6f227 lrc: 4/1,0 mode: --/PR res: [0x200027cd1:0x3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceda19 expref: -99 pid: 160592 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.211786:0:160592:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7c00/0xbef42c0221d6f227 lrc: 4/1,0 mode: --/PR res: [0x200027cd1:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceda19 expref: -99 pid: 160592 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.211796:0:160592:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7c00/0xbef42c0221d6f227 lrc: 4/1,0 mode: PR/PR res: [0x200027cd1:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceda19 expref: -99 pid: 160592 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.211806:0:160592:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7c00/0xbef42c0221d6f227 lrc: 4/1,0 mode: PR/PR res: [0x200027cd1:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceda19 expref: -99 pid: 160592 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.211819:0:160592:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7c00/0xbef42c0221d6f227 lrc: 5/2,0 mode: PR/PR res: [0x200027cd1:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceda19 expref: -99 pid: 160592 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.211826:0:160592:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7c00/0xbef42c0221d6f227 lrc: 5/2,0 mode: PR/PR res: [0x200027cd1:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceda19 expref: -99 pid: 160592 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.211838:0:160592:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7c00/0xbef42c0221d6f227 lrc: 4/1,0 mode: PR/PR res: [0x200027cd1:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceda19 expref: -99 pid: 160592 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787826262.211850:0:160592:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c71f0e00 x1874663042995456/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 600/584 e 0 to 0 dl 1787826278 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787826262.211861:0:160592:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7c00/0xbef42c0221d6f227 lrc: 3/1,0 mode: PR/PR res: [0x200027cd1:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceda19 expref: -99 pid: 160592 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.211870:0:160592:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d1a77c00 type 13 mode 4 res 8590097617/3 (0 0) 00000080:00200000:2.0:1787826262.211877:0:160592:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1787826262.211900:0:160592:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200027cd1:0x3:0x0], updating i_size 0 00000080:00200000:2.0:1787826262.211903:0:160592:0:(namei.c:147:ll_iget()) got inode: [0x200027cd1:0x3:0x0](ffff9181d0808908): rc = 0 00000080:00200000:2.0:1787826262.211919:0:160592:0:(file.c:6843:ll_layout_conf()) [0x200027cd1:0x3:0x0]: layout version change: 4294967294 -> 4294967295 00000080:00010000:2.0:1787826262.211925:0:160592:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200027cd1:0x3:0x0](ffff9181d0808908) for lock 0xbef42c0221d6f227 00000080:00010000:2.0:1787826262.211935:0:160592:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200027cd1:0x3:0x0](ffff9181d0808908) 00000080:00010000:2.0:1787826262.211938:0:160592:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f227 from it ffffb9750181bb20 00010000:00010000:2.0:1787826262.211941:0:160592:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7c00/0xbef42c0221d6f227 lrc: 3/1,0 mode: PR/PR res: [0x200027cd1:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00ceda19 expref: -99 pid: 160592 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826262.211950:0:160592:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7c00/0xbef42c0221d6f227 lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00ceda19 expref: -99 pid: 160592 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:2.0:1787826262.211990:0:160592:0:(file.c:6296:ll_getattr_dentry()) START file fsa-oleg347-client.virtnet:[0x200027cd1:0x3:0x0](ffff9181d0808908), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1787826262.211997:0:160592:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200027cd1:0x3:0x0](ffff9181d0808908),name=fsa-oleg347-client.virtnet 00000002:00010000:2.0:1787826262.212017:0:160592:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200027cd1:0x3:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826262.212026:0:160592:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7c00/0xbef42c0221d6f227 lrc: 2/1,0 mode: PR/PR res: [0x200027cd1:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00ceda19 expref: -99 pid: 160592 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826262.212038:0:160592:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7c00/0xbef42c0221d6f227 lrc: 2/1,0 mode: PR/PR res: [0x200027cd1:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00ceda19 expref: -99 pid: 160592 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826262.212044:0:160592:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826262.212049:0:160592:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200027cd1:0x3:0x0](ffff9181d0808908) 00000080:00010000:2.0:1787826262.212053:0:160592:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200027cd1:0x3:0x0](ffff9181d0808908) for lock 0xbef42c0221d6f227 00000080:00010000:2.0:1787826262.212057:0:160592:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f227 from it ffffb9750181bcb8 00010000:00010000:2.0:1787826262.212060:0:160592:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7c00/0xbef42c0221d6f227 lrc: 3/1,0 mode: PR/PR res: [0x200027cd1:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00ceda19 expref: -99 pid: 160592 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826262.212067:0:160592:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7c00/0xbef42c0221d6f227 lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00ceda19 expref: -99 pid: 160592 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1787826262.212128:0:160592:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:2.0:1787826262.212848:0:160592:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826262.212858:0:160592:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7a00/0xbef42c0221d6f220 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced9fd expref: -99 pid: 160592 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826262.212867:0:160592:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7a00/0xbef42c0221d6f220 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced9fd expref: -99 pid: 160592 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826262.212874:0:160592:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826262.212879:0:160592:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:2.0:1787826262.212883:0:160592:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f220 00000080:00010000:2.0:1787826262.212887:0:160592:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f220 from it ffffb9750181bc10 00010000:00010000:2.0:1787826262.212890:0:160592:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7a00/0xbef42c0221d6f220 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced9fd expref: -99 pid: 160592 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826262.212898:0:160592:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7a00/0xbef42c0221d6f220 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced9fd expref: -99 pid: 160592 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1787826262.212907:0:160592:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:2.0:1787826262.212918:0:160592:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=fsa-oleg347-client.virtnet, flags=0 00000080:00200000:2.0:1787826262.212923:0:160592:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:2.0:1787826262.212929:0:160592:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826262.212935:0:160592:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7a00/0xbef42c0221d6f220 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced9fd expref: -99 pid: 160592 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787826262.212943:0:160592:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7a00/0xbef42c0221d6f220 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced9fd expref: -99 pid: 160592 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1787826262.212949:0:160592:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826262.212953:0:160592:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:2.0:1787826262.212955:0:160592:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f220 00000080:00010000:2.0:1787826262.212959:0:160592:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f220 from it ffffb9750181bd68 00010000:00010000:2.0:1787826262.212962:0:160592:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7a00/0xbef42c0221d6f220 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ced9fd expref: -99 pid: 160592 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787826262.212969:0:160592:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7a00/0xbef42c0221d6f220 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ced9fd expref: -99 pid: 160592 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:2.0:1787826262.212977:0:160592:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 3 00000080:00200000:2.0:1787826262.212983:0:160592:0:(namei.c:2418:ll_unlink()) VFS Op:name=fsa-oleg347-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9181d080cb08) 00010000:00010000:2.0:1787826262.213005:0:160592:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7a00/0xbef42c0221d6f220 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ced9fd expref: -99 pid: 160592 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1787826262.213029:0:160592:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7c00/0xbef42c0221d6f227 lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x878a5eea00ceda19 expref: -99 pid: 160592 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1787826262.213038:0:160592:0:(vvp_object.c:117:vvp_conf_set()) [0x200027cd1:0x3:0x0]: losing layout lock 00010000:00010000:2.0:1787826262.213071:0:160592:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7c00/0xbef42c0221d6f227 lrc: 1/0,0 mode: --/PR res: [0x200027cd1:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00ceda19 expref: -99 pid: 160592 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787826262.213079:0:160592:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7a00/0xbef42c0221d6f220 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ced9fd expref: -99 pid: 160592 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1787826262.213085:0:160592:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:2.0:1787826262.213088:0:160592:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7c00/0xbef42c0221d6f227 lrc: 0/0,0 mode: --/PR res: [0x200027cd1:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00ceda19 expref: -99 pid: 160592 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787826262.213106:0:160592:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7a00/0xbef42c0221d6f220 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ced9fd expref: -99 pid: 160592 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000020:00100000:2.0:1787826262.213122:0:160592:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787826262.213209:0:160592:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c71f0e00 x1874663042995712/t0(0) o36->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 512/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826262.213232:0:160592:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f0e00 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:160592:1874663042995712:192.168.203.147@tcp:36:rm.0 00000100:00100000:2.0:1787826262.213426:0:160592:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5256c00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826262.221203:0:160592:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f0e00 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:160592:1874663042995712:192.168.203.147@tcp:36:rm.0 00000080:00200000:2.0:1787826262.221262:0:160592:0:(vvp_io.c:1826:vvp_io_init()) [0x200027cd1:0x3:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:2.0:1787826262.221273:0:160592:0:(vvp_io.c:304:vvp_io_fini()) [0x200027cd1:0x3:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826262.221283:0:160592:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200027cd1:0x3:0x0](ffff9181d0808908) 00010000:00010000:3.0:1787826262.626167:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826263.649194:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826264.674831:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826265.697012:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826266.148213:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:f40147eb-9b53-47d4-b77c-99ceedd9664c:2389:1874663042995840:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826266.148291:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663042995968:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826266.148337:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663042996096:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826266.148355:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663042996224:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826266.148381:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663042996352:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826266.152555:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663042996352:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826266.152574:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:f40147eb-9b53-47d4-b77c-99ceedd9664c:2389:1874663042995840:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826266.152585:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663042996096:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826266.153163:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663042995968:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826266.153252:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663042996224:192.168.203.147@tcp:400:kworker.0 00010000:00010000:3.0:1787826266.721241:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826267.745799:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826268.769622:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:0.0:1787826269.582462:0:160664:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: local REPLAY BARRIER on lustre-MDT0000 00010000:00010000:3.0:1787826269.793028:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826270.825185:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826271.265118:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663042996608:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826271.265238:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663042996864:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826271.265277:0:2389:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:0.0:1787826271.265284:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:f40147eb-9b53-47d4-b77c-99ceedd9664c:2389:1874663042996480:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826271.265330:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663042996736:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826271.265355:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663042996992:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826271.294475:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:f40147eb-9b53-47d4-b77c-99ceedd9664c:2389:1874663042996480:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826271.294504:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663042996736:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826271.294517:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663042996992:192.168.203.147@tcp:400:kworker.0 00000100:00080000:1.0:1787826271.294531:0:2389:0:(recover.c:205:ptlrpc_request_handle_notconn()) import lustre-MDT0000-mdc-ffff9181c5620000 of lustre-MDT0000_UUID@192.168.203.147@tcp abruptly disconnected: reconnecting 00000100:00080000:1.0:1787826271.294538:0:2389:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from FULL to DISCONN 00000100:02000400:1.0:1787826271.294543:0:2389:0:(import.c:182:ptlrpc_set_import_discon()) lustre-MDT0000-mdc-ffff9181c5620000: Connection to lustre-MDT0000 (at 192.168.203.147@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00080000:1.0:1787826271.294552:0:2389:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826271.294556:0:2389:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8306 00000100:00080000:1.0:1787826271.294571:0:2389:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787826271.294583:0:2389:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787826271.294621:0:2389:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0700 x1874663042997120/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826271.294666:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663042996608:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826271.294682:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663042996864:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826271.294905:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663042997120:192.168.203.147@tcp:38:ptlrpcd_00_00.0 00000100:00080000:3.0:1787826271.315860:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826271.315879:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-19) 00000100:00100000:3.0:1787826271.315886:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663042997120:192.168.203.147@tcp:38:ptlrpcd_00_00.0 00010000:00010000:3.0:1787826271.841646:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826272.288995:0:160572:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5256540 going to sleep for 25 seconds 00010000:00010000:1.0:1787826272.865132:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826273.889137:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826274.913238:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826275.939331:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826276.386447:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:f40147eb-9b53-47d4-b77c-99ceedd9664c:2391:1874663042997248:192.168.203.147@tcp:400:kworker.0 00000100:00080000:2.0:1787826276.386600:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826276.386606:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787826276.386610:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826276.386617:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8343 00000100:00080000:2.0:1787826276.386625:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 5s 00000100:00080000:2.0:1787826276.386633:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826276.386646:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826276.386669:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c71f1880 x1874663042997376/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826276.386778:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663042997376:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787826276.394423:0:2391:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787826276.394519:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663042997504:192.168.203.147@tcp:400:kworker.0 00000100:00080000:3.0:1787826276.397579:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826276.397597:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-19) 00000100:00100000:3.0:1787826276.397603:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663042997376:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787826276.402207:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663042997760:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826276.409699:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663042997632:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826276.423207:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663042997504:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826276.423884:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663042997760:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826276.438124:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663042997632:192.168.203.147@tcp:400:kworker.0 00010000:00010000:1.0:1787826276.961172:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826277.985585:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826279.011170:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826280.034302:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826281.057083:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826281.505139:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:f40147eb-9b53-47d4-b77c-99ceedd9664c:2391:1874663042997888:192.168.203.147@tcp:400:kworker.0 00000100:00080000:2.0:1787826281.505587:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826281.505594:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787826281.505599:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826281.505607:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8348 00000100:00080000:2.0:1787826281.505616:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 10s 00000100:00080000:2.0:1787826281.505626:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826281.505641:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826281.505667:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c71f1500 x1874663042998016/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826281.505781:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663042998016:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787826281.506589:0:2391:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787826281.506604:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663042998144:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826281.507578:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663042998272:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826281.507614:0:2391:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1787826281.507621:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663042998400:192.168.203.147@tcp:400:kworker.0 00000100:00080000:3.0:1787826281.526865:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826281.526884:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-19) 00000100:00100000:3.0:1787826281.526890:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663042998016:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826281.532189:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663042998144:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826281.532250:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663042998400:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826281.532683:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663042998272:192.168.203.147@tcp:400:kworker.0 00010000:00010000:3.0:1787826282.081131:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826283.106672:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826284.133449:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826285.153183:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826286.177155:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787826286.624926:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826286.624935:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787826286.624938:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826286.624950:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8353 00000100:00080000:2.0:1787826286.624963:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 15s 00000100:00080000:2.0:1787826286.624971:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826286.624985:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826286.625009:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c71f2680 x1874663042998656/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787826286.625044:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc3800 lustre-OST0000_UUID: changing import state from FULL to CONNECTING 00000100:00080000:2.0:1787826286.625047:0:99984:0:(import.c:2101:ptlrpc_disconnect_and_idle_import()) lustre-OST0000-osc-ffff9181c5620000: disconnect after 24s idle 00000100:00080000:2.0:1787826286.625056:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc6000 lustre-OST0001_UUID: changing import state from FULL to CONNECTING 00000100:00080000:2.0:1787826286.625058:0:99984:0:(import.c:2101:ptlrpc_disconnect_and_idle_import()) lustre-OST0001-osc-ffff9181c5620000: disconnect after 24s idle 00000100:00100000:2.0:1787826286.625269:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663042998784:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826286.625386:0:2392:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1787826286.625395:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:f40147eb-9b53-47d4-b77c-99ceedd9664c:2392:1874663042998528:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826286.625442:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663042998656:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826286.625473:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663042998912:192.168.203.147@tcp:9:kworker.0 00000100:00100000:2.0:1787826286.625496:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663042999040:192.168.203.147@tcp:9:kworker.0 00000100:00100000:3.0:1787826286.631512:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663042998784:192.168.203.147@tcp:400:kworker.0 00000100:00080000:2.0:1787826286.632602:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826286.632614:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-19) 00000100:00100000:2.0:1787826286.632620:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663042998656:192.168.203.147@tcp:38:kworker.0 00000100:00080000:2.0:1787826286.632642:0:2388:0:(import.c:2011:ptlrpc_disconnect_idle_interpret()) @@@ inflight=1, refcount=3: rc = 0 req@ffff9181c71f2a00 x1874663042998912/t0(0) o9->lustre-OST0000-osc-ffff9181c5620000@192.168.203.147@tcp:28/4 lens 224/224 e 0 to 0 dl 1787826296 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826286.632652:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc3800 lustre-OST0000_UUID: changing import state from CONNECTING to IDLE 00000100:00100000:2.0:1787826286.632656:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663042998912:192.168.203.147@tcp:9:kworker.0 00000100:00080000:2.0:1787826286.632672:0:2388:0:(import.c:2011:ptlrpc_disconnect_idle_interpret()) @@@ inflight=1, refcount=3: rc = 0 req@ffff9181c71f0380 x1874663042999040/t0(0) o9->lustre-OST0001-osc-ffff9181c5620000@192.168.203.147@tcp:28/4 lens 224/224 e 0 to 0 dl 1787826296 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826286.632681:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc6000 lustre-OST0001_UUID: changing import state from CONNECTING to IDLE 00000100:00100000:2.0:1787826286.632685:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663042999040:192.168.203.147@tcp:9:kworker.0 00010000:00010000:1.0:1787826287.203512:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826288.224895:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826289.256948:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826290.273143:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826291.300629:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787826291.744900:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826291.744909:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787826291.744913:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826291.744926:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8358 00000100:00080000:2.0:1787826291.744938:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826291.744947:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826291.744961:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826291.744984:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:f40147eb-9b53-47d4-b77c-99ceedd9664c:2391:1874663042999168:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826291.744985:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c71f2a00 x1874663042999296/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826291.745064:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663042999424:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826291.745209:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663042999296:192.168.203.147@tcp:38:kworker.0 00000100:00080000:3.0:1787826291.756729:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826291.756891:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-19) 00000100:00100000:3.0:1787826291.756909:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663042999296:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826291.757554:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663042999424:192.168.203.147@tcp:400:kworker.0 00010000:00010000:3.0:1787826292.321452:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000400:3.0:1787826292.769003:0:2391:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787826276/real 1787826276] req@ffff9181c71f3480 x1874663042997248/t0(0) o400->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 224/224 e 0 to 1 dl 1787826292 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826292.832034:0:2391:0:(client.c:2547:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff9181c71f3480 x1874663042997248/t0(0) o400->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 224/224 e 0 to 1 dl 1787826292 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826292.832051:0:2391:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from FULL to DISCONN 00000100:02020000:3.0:1787826292.832055:0:2391:0:(import.c:186:ptlrpc_set_import_discon()) MGC192.168.203.147@tcp: Connection to MGS (at 192.168.203.147@tcp) was lost; in progress operations using this service will fail 00000100:00080000:3.0:1787826292.845013:0:2391:0:(import.c:192:ptlrpc_set_import_discon()) import MGS@192.168.203.147@tcp for MGC192.168.203.147@tcp not replayable, auto-deactivating 00000100:00080000:3.0:1787826292.845021:0:2391:0:(import.c:138:ptlrpc_deactivate_import_nolock()) setting import MGS INVALID 00000100:00100000:3.0:1787826292.845033:0:2391:0:(client.c:3589:ptlrpc_abort_inflight()) @@@ inflight req@ffff9181c71f3480 x1874663042997248/t0(0) o400->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 224/224 e 0 to 1 dl 1787826292 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826292.845049:0:2391:0:(client.c:3589:ptlrpc_abort_inflight()) @@@ inflight req@ffff9181c71f1c00 x1874663042997888/t0(0) o400->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 224/224 e 0 to 0 dl 1787826297 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826292.845059:0:2391:0:(client.c:3589:ptlrpc_abort_inflight()) @@@ inflight req@ffff9181c71f1180 x1874663042998528/t0(0) o400->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 224/224 e 0 to 0 dl 1787826302 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826292.845081:0:2391:0:(client.c:3589:ptlrpc_abort_inflight()) @@@ inflight req@ffff9181c71f0380 x1874663042999168/t0(0) o400->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 224/224 e 0 to 0 dl 1787826307 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 10000000:01000000:3.0:1787826292.845093:0:2391:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808001 00000100:00080000:3.0:1787826292.845097:0:2391:0:(pinger.c:193:ptlrpc_pinger_ir_down()) IR down 10000000:01000000:3.0:1787826292.845099:0:2391:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808002 00000100:00080000:3.0:1787826292.845101:0:2391:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:0.0:1787826292.845491:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826292.845499:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787826292.845502:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826292.845509:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8306 00000100:00080000:0.0:1787826292.845527:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787826292.845570:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6a00 x1874663042999552/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787826292.845599:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826292.845604:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787826292.845606:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826292.845610:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8363 00000100:00080000:0.0:1787826292.845615:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826292.845620:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787826292.845632:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826292.845646:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7100 x1874663042999680/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826292.845726:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663042999808:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826292.847728:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:f40147eb-9b53-47d4-b77c-99ceedd9664c:2392:1874663042998528:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826292.847832:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:f40147eb-9b53-47d4-b77c-99ceedd9664c:2391:1874663042999168:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826292.847842:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:f40147eb-9b53-47d4-b77c-99ceedd9664c:2391:1874663042997888:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826292.847847:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:f40147eb-9b53-47d4-b77c-99ceedd9664c:2391:1874663042997248:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826292.848389:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663042999552:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787826292.849279:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663042999680:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787826292.849659:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663042999808:192.168.203.147@tcp:400:kworker.0 00000100:00080000:3.0:1787826292.852447:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826292.852463:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-19) 00000100:00100000:3.0:1787826292.852470:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663042999680:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787826293.346481:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826294.368960:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826295.393327:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826296.417124:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826297.377129:0:160572:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5256540 going to sleep for 25 seconds 00010000:00010000:1.0:1787826297.441225:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787826297.889138:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826297.889151:0:160087:0:(pinger.c:247:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1787826297.889154:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826297.889158:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787826297.889162:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826297.889175:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8364 00000100:00080000:2.0:1787826297.889188:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826297.889196:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826297.889212:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826297.889255:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c71f3480 x1874663042999936/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826297.889326:0:2392:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1787826297.889341:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663043000064:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826297.889685:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663042999936:192.168.203.147@tcp:38:kworker.0 00000100:00080000:3.0:1787826297.892046:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826297.892061:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-19) 00000100:00100000:3.0:1787826297.892066:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663042999936:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826297.892175:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663043000064:192.168.203.147@tcp:400:kworker.0 00010000:00010000:3.0:1787826298.466881:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826299.489021:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826300.515583:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826301.539074:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826302.565940:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826303.009367:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787826292/real 1787826292] req@ffff9181c55a6a00 x1874663042999552/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826302 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826303.009410:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6a00 x1874663042999552/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826302 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826303.009434:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826303.011133:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826303.011143:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787826303.011149:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826303.011160:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8364 00000100:00080000:0.0:1787826303.011174:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 5s 00000100:00080000:0.0:1787826303.011183:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787826303.011232:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7480 x1874663043000192/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787826303.011250:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826303.011254:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787826303.011256:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826303.011260:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8369 00000100:00080000:0.0:1787826303.011263:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826303.011266:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787826303.011277:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826303.011286:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5880 x1874663043000320/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826303.011735:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043000448:192.168.203.147@tcp:400:kworker.0 00000100:00080000:3.0:1787826303.012045:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826303.012053:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663042999552:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787826303.012097:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043000192:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787826303.013380:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043000320:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787826303.019389:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043000448:192.168.203.147@tcp:400:kworker.0 00000100:00080000:3.0:1787826303.019403:0:2388:0:(import.c:1132:ptlrpc_connect_interpret()) MGC192.168.203.147@tcp: connect to target with instance 0 10000000:01000000:3.0:1787826303.019420:0:2388:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808005 00000100:02000400:3.0:1787826303.019425:0:2388:0:(import.c:1284:ptlrpc_connect_interpret()) Evicted from MGS (at 192.168.203.147@tcp) after server handle changed from 0x878a5eea00ced6ca to 0x878a5eea00cedc3b 00000100:00080000:3.0:1787826303.041232:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to EVICTED 00000100:00080000:3.0:1787826303.041251:0:2388:0:(import.c:1674:ptlrpc_import_recovery_state_machine()) evicted from MGS@192.168.203.147@tcp; invalidating 00000100:00100000:3.0:1787826303.043373:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043000192:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787826303.043419:0:160933:0:(import.c:1598:ptlrpc_invalidate_import_thread()) thread invalidate import MGC192.168.203.147@tcp to MGS@192.168.203.147@tcp 00000100:00100000:3.0:1787826303.043425:0:160933:0:(import.c:323:ptlrpc_invalidate_import()) Sleeping 20s for inflight to error out 10000000:01000000:3.0:1787826303.043430:0:160933:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808003 00010000:00010000:3.0:1787826303.043452:0:160933:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e06f6000/0xbef42c0221d6f147 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x110e400000000 nid: local remote: 0x878a5eea00ced6d1 expref: -99 pid: 159721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1787826303.043465:0:160933:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e06f6000/0xbef42c0221d6f147 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x110f400000000 nid: local remote: 0x878a5eea00ced6d1 expref: -99 pid: 159721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787826303.043472:0:160933:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:3.0:1787826303.043485:0:160933:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826303.043497:0:160933:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e06f6000/0xbef42c0221d6f147 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x590f400000000 nid: local remote: 0x878a5eea00ced6d1 expref: -99 pid: 159721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826303.043513:0:160933:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e06f6a00/0xbef42c0221d6f14e lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0x878a5eea00ced6d8 expref: -99 pid: 159721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1787826303.043518:0:160933:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e06f6a00/0xbef42c0221d6f14e lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0x878a5eea00ced6d8 expref: -99 pid: 159721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787826303.043523:0:160933:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:3.0:1787826303.043526:0:160933:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826303.043529:0:160933:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e06f6a00/0xbef42c0221d6f14e lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0x878a5eea00ced6d8 expref: -99 pid: 159721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826303.043541:0:160933:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b4600/0xbef42c0221d6f1b7 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0x878a5eea00ced7a3 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1787826303.043547:0:160933:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4600/0xbef42c0221d6f1b7 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0x878a5eea00ced7a3 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787826303.043551:0:160933:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:3.0:1787826303.043555:0:160933:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826303.043558:0:160933:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b4600/0xbef42c0221d6f1b7 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0x878a5eea00ced7a3 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826303.043570:0:160933:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e06f6e00/0xbef42c0221d6f186 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0x878a5eea00ced6e6 expref: -99 pid: 159721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1787826303.043576:0:160933:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e06f6e00/0xbef42c0221d6f186 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0x878a5eea00ced6e6 expref: -99 pid: 159721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787826303.043582:0:160933:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:3.0:1787826303.043586:0:160933:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826303.043589:0:160933:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e06f6e00/0xbef42c0221d6f186 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0x878a5eea00ced6e6 expref: -99 pid: 159721 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787826303.043604:0:160933:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from EVICTED to RECOVER 00000100:00080000:3.0:1787826303.043610:0:160933:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from RECOVER to FULL 10000000:01000000:3.0:1787826303.043612:0:160933:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808004 00000100:00080000:3.0:1787826303.043617:0:160933:0:(pinger.c:186:ptlrpc_pinger_ir_up()) IR up 00000100:02000000:3.0:1787826303.043621:0:160933:0:(import.c:1743:ptlrpc_import_recovery_state_machine()) MGC192.168.203.147@tcp: Connection restored to 192.168.203.147@tcp (at 192.168.203.147@tcp) 00010000:00080000:1.0:1787826303.050946:0:2388:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffff9181c55a5880 x1874663043000320/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/416 e 0 to 0 dl 1787826328 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787826303.050979:0:2388:0:(import.c:1132:ptlrpc_connect_interpret()) lustre-MDT0000-mdc-ffff9181c5620000: connect to target with instance 82 00000100:00080000:1.0:1787826303.051005:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to REPLAY 00000100:00080000:1.0:1787826303.051013:0:2388:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1787826303.051017:0:2388:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9181c5620000: committing for last_committed 347892350982 gen 1 00000100:00100000:1.0:1787826303.051024:0:2388:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff9181c5636300 x1874663042995200/t347892350984(347892350984) o36->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 528/448 e 0 to 0 dl 1787826278 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787826303.051045:0:2388:0:(recover.c:53:ptlrpc_replay_next()) import ffff9181c5bc0000 from lustre-MDT0000_UUID committed 347892350982 last 0 00000100:00080000:1.0:1787826303.051054:0:2388:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9181c71f3b80 x1874663042993408/t347892350982(347892350982) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 0 dl 1787826277 ref 1 fl New:RPQU/604/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:0 00000100:00100000:1.0:1787826303.051077:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043000320:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787826303.051134:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663042993408:192.168.203.147@tcp:101:lfs.0 00000100:00080000:3.0:1787826303.096858:0:2388:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9181c71f3b80 x1874663042993408/t347892350982(347892350982) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 0 dl 1787826319 ref 2 fl Interpret:RPQU/604/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00080000:3.0:1787826303.096880:0:2388:0:(mdc_request.c:696:mdc_replay_open()) updating handle from 0x878a5eea00ced9c5 to 0x878a5eea00cedc49 00000100:00020000:3.0:1787826303.096885:0:2388:0:(client.c:3448:ptlrpc_replay_interpret()) @@@ status 301, old was 0 req@ffff9181c71f3b80 x1874663042993408/t347892350982(347892350982) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 0 dl 1787826319 ref 2 fl Interpret:RPQU/604/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000100:00080000:3.0:1787826303.096896:0:2388:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:3.0:1787826303.096899:0:2388:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9181c5620000: committing for last_committed 347892350982 gen 1 00000100:00100000:3.0:1787826303.096906:0:2388:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff9181c5636300 x1874663042995200/t347892350984(347892350984) o36->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 528/448 e 0 to 0 dl 1787826278 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826303.096922:0:2388:0:(recover.c:53:ptlrpc_replay_next()) import ffff9181c5bc0000 from lustre-MDT0000_UUID committed 347892350982 last 347892350982 00000100:00080000:3.0:1787826303.096931:0:2388:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9181c5636300 x1874663042995200/t347892350984(347892350984) o36->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 528/448 e 0 to 0 dl 1787826278 ref 1 fl New:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826303.096944:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663042993408:192.168.203.147@tcp:101:lfs.0 00000100:00100000:2.0:1787826303.097214:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663042995200:192.168.203.147@tcp:36:mcreate.0 00010000:00010000:1.0:1787826303.585001:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787826304.481775:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:1.0:1787826304.481784:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff9181c5620000 from 336 10000000:01000000:1.0:1787826304.481791:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:1.0:1787826304.481858:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f22e lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826304.481871:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f22e lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826304.481890:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f22e lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826304.481924:0:159735:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043000576:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00100000:1.0:1787826304.482055:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826304.490385:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043000576:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787826304.490426:0:159735:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f22e lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00cedc50 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826304.490442:0:159735:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f22e lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x878a5eea00cedc50 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826304.490555:0:159735:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043000704:192.168.203.147@tcp:256:ll_cfg_requeue.0 00000100:00100000:1.0:1787826304.490664:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826304.499325:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043000704:192.168.203.147@tcp:256:ll_cfg_requeue.0 10000000:01000000:1.0:1787826304.499458:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-cliir' succeeded (0). 00010000:00010000:1.0:1787826304.500018:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f22e lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00cedc50 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826304.500033:0:159735:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f22e lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00cedc50 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826304.500046:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:1.0:1787826304.500050:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff9181c5620000 from 52 10000000:01000000:1.0:1787826304.500054:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:1.0:1787826304.500114:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f235 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826304.500124:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f235 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826304.500137:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f235 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826304.500167:0:159735:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043000832:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00100000:1.0:1787826304.500278:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826304.512278:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043000832:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787826304.512317:0:159735:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f235 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00cedc57 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826304.512333:0:159735:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f235 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x878a5eea00cedc57 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826304.512435:0:159735:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043000960:192.168.203.147@tcp:501:ll_cfg_requeue.0 00000100:00100000:1.0:1787826304.512547:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826304.518360:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043000960:192.168.203.147@tcp:501:ll_cfg_requeue.0 00000100:00100000:1.0:1787826304.518449:0:159735:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043001088:192.168.203.147@tcp:503:ll_cfg_requeue.0 00000100:00100000:1.0:1787826304.518792:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826304.528413:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043001088:192.168.203.147@tcp:503:ll_cfg_requeue.0 00000040:00080000:2.0:1787826304.532578:0:160934:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x3:0x0] index 64768 count 51 00000020:01000000:1.0:1787826304.532893:0:159735:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-client gen 52-51 (rc=0) 10000000:01000000:1.0:1787826304.532918:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-client' succeeded (0). 00010000:00010000:1.0:1787826304.532933:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f235 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00cedc57 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826304.532946:0:159735:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f235 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00cedc57 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826304.532958:0:159735:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:1.0:1787826304.532961:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff9181c5620000 from 7 10000000:01000000:1.0:1787826304.532964:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00010000:00010000:1.0:1787826304.533018:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f23c lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826304.533027:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f23c lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826304.533043:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f23c lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826304.533090:0:159735:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043001216:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00100000:1.0:1787826304.533221:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826304.545527:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043001216:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787826304.545568:0:159735:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f23c lrc: 4/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00cedc5e expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826304.545582:0:159735:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f23c lrc: 4/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x878a5eea00cedc5e expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826304.545682:0:159735:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043001344:192.168.203.147@tcp:501:ll_cfg_requeue.0 00000100:00100000:1.0:1787826304.545852:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826304.558101:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043001344:192.168.203.147@tcp:501:ll_cfg_requeue.0 00000100:00100000:1.0:1787826304.558197:0:159735:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043001472:192.168.203.147@tcp:503:ll_cfg_requeue.0 00000100:00100000:1.0:1787826304.559217:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826304.566343:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043001472:192.168.203.147@tcp:503:ll_cfg_requeue.0 00000040:00080000:3.0:1787826304.567716:0:160935:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x2:0x0] index 64768 count 7 00000020:01000000:1.0:1787826304.568267:0:159735:0:(obd_config.c:2145:class_config_parse_llog()) Processed log params gen 7-6 (rc=0) 10000000:01000000:1.0:1787826304.568291:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'params' succeeded (0). 00010000:00010000:1.0:1787826304.568304:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f23c lrc: 3/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00cedc5e expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826304.568318:0:159735:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f23c lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00cedc5e expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826304.568328:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:1.0:1787826304.568331:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff9181c87b19b8 from 1 10000000:01000000:1.0:1787826304.568334:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00010000:00010000:1.0:1787826304.568449:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5e00/0xbef42c0221d6f243 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826304.568458:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b5e00/0xbef42c0221d6f243 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826304.568471:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b5e00/0xbef42c0221d6f243 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826304.568503:0:159735:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043001600:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00100000:1.0:1787826304.568668:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826304.585166:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043001600:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787826304.585206:0:159735:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.147@tcp lock: ffff9181e01b5e00/0xbef42c0221d6f243 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00cedc65 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826304.585221:0:159735:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.147@tcp lock: ffff9181e01b5e00/0xbef42c0221d6f243 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x878a5eea00cedc65 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826304.585339:0:159735:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043001728:192.168.203.147@tcp:501:ll_cfg_requeue.0 00000100:00100000:1.0:1787826304.585448:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59c8cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826304.589416:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043001728:192.168.203.147@tcp:501:ll_cfg_requeue.0 10000000:01000000:1.0:1787826304.589478:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-sptlrpc' failed (-2). 00010000:00010000:1.0:1787826304.589492:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5e00/0xbef42c0221d6f243 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00cedc65 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826304.589506:0:159735:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.147@tcp lock: ffff9181e01b5e00/0xbef42c0221d6f243 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00cedc65 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826304.609195:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826305.633069:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826306.656995:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787826306.698464:0:2388:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9181c5636300 x1874663042995200/t347892350984(347892350984) o36->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 528/448 e 0 to 0 dl 1787826319 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826306.698480:0:2388:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:2.0:1787826306.698483:0:2388:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9181c5620000: committing for last_committed 347892350982 gen 1 00000100:00100000:2.0:1787826306.698488:0:2388:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff9181c5636300 x1874663042995200/t347892350984(347892350984) o36->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 528/448 e 0 to 0 dl 1787826319 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826306.698496:0:2388:0:(recover.c:53:ptlrpc_replay_next()) import ffff9181c5bc0000 from lustre-MDT0000_UUID committed 347892350982 last 347892350984 00000100:00080000:2.0:1787826306.698504:0:2388:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9181c71f0e00 x1874663042995712/t347892350985(347892350985) o36->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 512/456 e 0 to 0 dl 1787826278 ref 1 fl New:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826306.698516:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663042995200:192.168.203.147@tcp:36:mcreate.0 00000100:00100000:2.0:1787826306.698535:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663042995712:192.168.203.147@tcp:36:rm.0 00000100:00080000:2.0:1787826306.704243:0:2388:0:(client.c:3438:ptlrpc_replay_interpret()) @@@ got reply req@ffff9181c71f0e00 x1874663042995712/t347892350985(347892350985) o36->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 512/456 e 0 to 0 dl 1787826322 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826306.704258:0:2388:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:2.0:1787826306.704261:0:2388:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9181c5620000: committing for last_committed 347892350982 gen 1 00000100:00100000:2.0:1787826306.704267:0:2388:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff9181c5636300 x1874663042995200/t347892350984(347892350984) o36->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 528/448 e 0 to 0 dl 1787826319 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826306.704274:0:2388:0:(recover.c:53:ptlrpc_replay_next()) import ffff9181c5bc0000 from lustre-MDT0000_UUID committed 347892350982 last 347892350985 00000100:00080000:2.0:1787826306.704278:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from REPLAY to REPLAY_LOCKS 00000100:00100000:1.0:1787826306.704739:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663042995712:192.168.203.147@tcp:36:rm.0 00010000:00080000:2.0:1787826306.705122:0:160945:0:(ldlm_request.c:2972:ldlm_lock_replay_thread()) lock replay thread lustre-MDT0000-mdc-ffff9181c5620000 to lustre-MDT0000_UUID@192.168.203.147@tcp 00010000:00010000:2.0:1787826306.705134:0:160945:0:(ldlm_request.c:2888:ldlm_cancel_unused_locks_for_replay()) Dropping as many unused locks as possible before replay for namespace lustre-MDT0000-mdc-ffff9181c5620000 (1) 00010000:00010000:2.0:1787826306.705178:0:160945:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7000/0xbef42c0221d6f1fd lrc: 0/0,0 mode: --/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000000 nid: local remote: 0x878a5eea00ced97f expref: -99 pid: 160572 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787826306.705199:0:160945:0:(ldlm_request.c:2901:ldlm_cancel_unused_locks_for_replay()) Canceled 0 unused locks from namespace lustre-MDT0000-mdc-ffff9181c5620000 00000100:00080000:2.0:1787826306.705606:0:160945:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from REPLAY_LOCKS to REPLAY_WAIT 00000100:00100000:1.0:1787826306.705690:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043001856:192.168.203.147@tcp:400:ldlm_lock_repla.0 00000100:00100000:1.0:1787826306.718617:0:2388:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9181c5620000: committing for last_committed 347892350985 gen 1 00000100:00080000:1.0:1787826306.718651:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from REPLAY_WAIT to RECOVER 00000100:00080000:1.0:1787826306.718666:0:2388:0:(client.c:3247:ptlrpc_resend_req()) @@@ going to resend req@ffff9181c71f1c00 x1874663043001856/t0(0) o400->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 224/224 e 0 to 0 dl 1787826322 ref 1 fl Interpret:RQU/2c0/0 rc 0/0 job:'ldlm_lock_repla.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787826306.718677:0:2388:0:(client.c:3256:ptlrpc_resend_req()) @@@ it has reply, so skip it req@ffff9181c71f1c00 x1874663043001856/t0(0) o400->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 224/224 e 0 to 0 dl 1787826322 ref 1 fl Interpret:RQU/2c0/0 rc 0/0 job:'ldlm_lock_repla.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787826306.718685:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from RECOVER to FULL 00000100:02000000:1.0:1787826306.719288:0:2388:0:(import.c:1743:ptlrpc_import_recovery_state_machine()) lustre-MDT0000-mdc-ffff9181c5620000: Connection restored to 192.168.203.147@tcp (at 192.168.203.147@tcp) 00000100:00100000:1.0:1787826306.719295:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043001856:192.168.203.147@tcp:400:ldlm_lock_repla.0 00000100:00100000:2.0:1787826306.770077:0:160572:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f1f80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:160572:1874663042994304:192.168.203.147@tcp:36:lfs.0 00000080:00200000:2.0:1787826306.770131:0:160572:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988) 00000080:00200000:2.0:1787826306.770139:0:160572:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x240001f59:0x1:0x0], updating i_size 4096 00000080:00200008:2.0:1787826306.770181:0:160572:0:(file.c:479:ll_file_release()) START file d80a.replay-single:[0x200027cd1:0x2:0x0](ffff9181c83f3248), flags 300000 00000002:00100000:2.0:1787826306.770238:0:160572:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181c71f3b80 x1874663042993408/t347892350982(347892350982) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 0 dl 1787826319 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:0 00000020:00100000:2.0:1787826306.770327:0:160572:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1787826306.770334:0:160572:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c71f1f80 x1874663043002112/t0(0) o35->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:2.0:1787826306.770356:0:160572:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f1f80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:160572:1874663043002112:192.168.203.147@tcp:35:lfs.0 00000100:00100000:2.0:1787826306.770431:0:160572:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5256540 going to sleep for 11 seconds 00000100:00100000:2.0:1787826306.776662:0:160572:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f1f80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:160572:1874663043002112:192.168.203.147@tcp:35:lfs.0 00000002:00100000:2.0:1787826306.776703:0:160572:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181c71f3b80 x1874663042993408/t347892350982(347892350982) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 0 dl 1787826319 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:0 00010000:00010000:1.0:1787826307.681826:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826308.129105:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663043002368:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826308.129216:0:2391:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1787826308.129223:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:f40147eb-9b53-47d4-b77c-99ceedd9664c:2391:1874663043002240:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826308.129257:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663043002496:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826308.133963:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:f40147eb-9b53-47d4-b77c-99ceedd9664c:2391:1874663043002240:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826308.133993:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663043002496:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826308.134008:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663043002368:192.168.203.147@tcp:400:kworker.0 00010000:00010000:3.0:1787826308.705071:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200008:2.0:1787826309.702897:0:160988:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181d080cb08), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1787826309.702914:0:160988:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:2.0:1787826309.702938:0:160988:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826309.703001:0:160988:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 160988 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.703012:0:160988:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 160988 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.703033:0:160988:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 160988 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826309.703059:0:160988:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f1f80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:160988:1874663043002624:192.168.203.147@tcp:101:lfs.0 00000100:00100000:2.0:1787826309.703191:0:160988:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d38fc300 going to sleep for 62 seconds 00000100:00100000:2.0:1787826309.710459:0:160988:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f1f80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:160988:1874663043002624:192.168.203.147@tcp:101:lfs.0 00010000:00010000:2.0:1787826309.710487:0:160988:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.710499:0:160988:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.710511:0:160988:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.710524:0:160988:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.710537:0:160988:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.710545:0:160988:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.710558:0:160988:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787826309.710570:0:160988:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c71f1f80 x1874663043002624/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826376 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787826309.710583:0:160988:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.710595:0:160988:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d1a77c00 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:2.0:1787826309.710602:0:160988:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:2.0:1787826309.710631:0:160988:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 69632 00000080:00010000:2.0:1787826309.710637:0:160988:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f24a 00000080:00010000:2.0:1787826309.710643:0:160988:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:2.0:1787826309.710647:0:160988:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f24a from it ffffb97502bf7cb8 00010000:00010000:2.0:1787826309.710650:0:160988:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826309.710659:0:160988:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:2.0:1787826309.711373:0:160988:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181d080cb08), request_mask 0, flags 0, foreign 0 00000080:00200000:2.0:1787826309.711380:0:160988:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:2.0:1787826309.711396:0:160988:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826309.711406:0:160988:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826309.711423:0:160988:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826309.711430:0:160988:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826309.711434:0:160988:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:2.0:1787826309.711437:0:160988:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f24a 00000080:00010000:2.0:1787826309.711442:0:160988:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f24a from it ffffb97502bf7cb8 00010000:00010000:2.0:1787826309.711445:0:160988:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826309.711452:0:160988:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1787826309.711559:0:160988:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:2.0:1787826309.711569:0:160988:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826309.711576:0:160988:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787826309.711585:0:160988:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1787826309.711591:0:160988:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826309.711595:0:160988:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:2.0:1787826309.711598:0:160988:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f24a 00000080:00010000:2.0:1787826309.711602:0:160988:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f24a from it ffffb97502bf7bb0 00010000:00010000:2.0:1787826309.711605:0:160988:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787826309.711613:0:160988:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:2.0:1787826309.711621:0:160988:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 44 00000080:00200008:2.0:1787826309.711627:0:160988:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9181d080cb08), flags 100000 00000080:00200080:2.0:1787826309.711729:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.711764:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00800000:00000004:2.0:1787826309.711772:0:160988:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9181c5620000: age 8333, max_age 8380 00000100:00100000:2.0:1787826309.711807:0:160988:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f1f80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:160988:1874663043002752:192.168.203.147@tcp:41:lfs.0 00000100:00100000:2.0:1787826309.711871:0:160988:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d38fc300 going to sleep for 62 seconds 00000100:00100000:2.0:1787826309.715521:0:160988:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f1f80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:160988:1874663043002752:192.168.203.147@tcp:41:lfs.0 00800000:00000004:2.0:1787826309.715552:0:160988:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0000-mdc-ffff9181c5620000: update ffff9181c354d178 cache blocks 317223/353529 objects 1023643/1024000 00000080:00200080:2.0:1787826309.715625:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.715641:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00800000:00000004:2.0:1787826309.715649:0:160988:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9181c5620000: age 8333, max_age 8380 00000100:00100000:2.0:1787826309.715714:0:160988:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f1f80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:160988:1874663043002880:192.168.203.147@tcp:41:lfs.0 00000100:00100000:2.0:1787826309.715804:0:160988:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d38fc300 going to sleep for 62 seconds 00000100:00100000:2.0:1787826309.720850:0:160988:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f1f80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:160988:1874663043002880:192.168.203.147@tcp:41:lfs.0 00800000:00000004:2.0:1787826309.720882:0:160988:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0001-mdc-ffff9181c5620000: update ffff9181c3548450 cache blocks 317323/353529 objects 1023702/1024000 00000080:00200080:2.0:1787826309.720957:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.720974:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.720982:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.720986:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.720990:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.720994:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.720999:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721002:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721006:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721010:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721014:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721017:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721021:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721038:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721044:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721047:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721052:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721055:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721059:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721062:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721066:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721069:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721072:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721076:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721079:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721083:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721087:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721090:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721093:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721097:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721101:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721105:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721109:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721113:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721117:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721121:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721124:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721127:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721130:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721133:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721137:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721140:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721144:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721147:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721151:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721155:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721159:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721163:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721167:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721170:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721173:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721176:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721180:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721183:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721188:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721191:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721195:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721198:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721202:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721205:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721209:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721212:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721216:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721219:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721223:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721227:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721231:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721234:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721238:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721241:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721245:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721248:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721251:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721254:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721259:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721262:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721265:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721269:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721273:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721277:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721281:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721284:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721288:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721292:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721295:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721298:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721302:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721304:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721307:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721311:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721314:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721317:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721321:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721324:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721328:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721331:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721335:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721338:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721342:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721345:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721349:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721352:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721355:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721358:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721362:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721365:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721368:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721371:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721375:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721378:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721382:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721385:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721389:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721392:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721395:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721398:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721402:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721406:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721409:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721413:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721417:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721420:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721424:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721426:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721430:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00800000:00000080:2.0:1787826309.721433:0:160988:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00000080:00200080:2.0:1787826309.721438:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:2.0:1787826309.721443:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00020000:00000004:2.0:1787826309.721450:0:160988:0:(obd_class.h:1090:obd_statfs()) lustre-OST0000-osc-ffff9181c5620000: age 8333, max_age 8380 00000100:00080000:2.0:1787826309.721463:0:160988:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8322 00000100:00080000:2.0:1787826309.721473:0:160988:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9181c5620000: import ffff9181c5bc3800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826309.721483:0:160988:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683216 ocd_grant: 8437760, lost: 0. 00000100:00100000:2.0:1787826309.721506:0:160988:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c71f3b80 x1874663043003008/t0(0) o8->lustre-OST0000-osc-ffff9181c5620000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826309.721580:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043003008:192.168.203.147@tcp:8:lfs.0 00000100:00080000:2.0:1787826309.722141:0:160988:0:(client.c:1845:ptlrpc_send_new_req()) @@@ req waiting for recovery: (FULL != CONNECTING) req@ffff9181c71f1f80 x1874663043003136/t0(0) o13->lustre-OST0000-osc-ffff9181c5620000@192.168.203.147@tcp:7/4 lens 224/368 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826309.722151:0:160988:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d38fc300 going to sleep for 0 seconds 00010000:00010000:3.0:1787826309.732853:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826309.735304:0:2388:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff9181c5620000: committing for last_committed 38654705666 gen 2 00000100:00080000:2.0:1787826309.735325:0:2388:0:(import.c:1132:ptlrpc_connect_interpret()) lustre-OST0000-osc-ffff9181c5620000: connect to target with instance 9 00000100:00080000:2.0:1787826309.735337:0:2388:0:(import.c:956:ptlrpc_connect_set_flags()) lustre-OST0000-osc-ffff9181c5620000: Resetting ns_connect_flags to server flags: 0xa0427af2e3440038 00000100:00080000:2.0:1787826309.735352:0:2388:0:(import.c:1213:ptlrpc_connect_interpret()) connected to replayable target: lustre-OST0000_UUID 00000100:00080000:2.0:1787826309.735358:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc3800 lustre-OST0000_UUID: changing import state from CONNECTING to FULL 00000080:00000004:2.0:1787826309.735369:0:2388:0:(lcommon_misc.c:94:cl_ocd_update()) Changing connect_flags: 0xa0427af2e3440038 -> 0xa0427af2e3440038 00000080:00080000:2.0:1787826309.735375:0:2388:0:(lcommon_misc.c:56:cl_init_ea_size()) updating def/max_easize: 72/65536 00000100:00080000:2.0:1787826309.735394:0:2388:0:(recover.c:191:ptlrpc_wake_delayed()) @@@ waking (set 0000000039610416): req@ffff9181c71f1f80 x1874663043003136/t0(0) o13->lustre-OST0000-osc-ffff9181c5620000@192.168.203.147@tcp:7/4 lens 224/368 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826309.735421:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043003008:192.168.203.147@tcp:8:lfs.0 00000100:00100000:1.0:1787826309.735600:0:160988:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d38fc300 going to sleep for 11 seconds 00000100:00100000:2.0:1787826309.741850:0:160988:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f1f80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:160988:1874663043003136:192.168.203.147@tcp:13:lfs.0 00020000:00000004:2.0:1787826309.741903:0:160988:0:(obd_class.h:1116:obd_statfs()) lustre-OST0000-osc-ffff9181c5620000: update ffff9181c3549e08 cache blocks 901356/958279 objects 261710/262144 00000080:00200080:2.0:1787826309.741998:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:2.0:1787826309.742020:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df3800 uarg=0000000000000000 00020000:00000004:2.0:1787826309.742041:0:160988:0:(obd_class.h:1090:obd_statfs()) lustre-OST0001-osc-ffff9181c5620000: age 8333, max_age 8380 00000100:00080000:2.0:1787826309.742055:0:160988:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8306 00000100:00080000:2.0:1787826309.742069:0:160988:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9181c5620000: import ffff9181c5bc6000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826309.742092:0:160988:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683216 ocd_grant: 8437760, lost: 0. 00000100:00100000:2.0:1787826309.742124:0:160988:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c71f3b80 x1874663043003264/t0(0) o8->lustre-OST0001-osc-ffff9181c5620000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787826309.742170:0:160988:0:(client.c:1845:ptlrpc_send_new_req()) @@@ req waiting for recovery: (FULL != CONNECTING) req@ffff9181c71f1f80 x1874663043003392/t0(0) o13->lustre-OST0001-osc-ffff9181c5620000@192.168.203.147@tcp:7/4 lens 224/368 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826309.742179:0:160988:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d38fc300 going to sleep for 0 seconds 00000100:00100000:1.0:1787826309.742642:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043003264:192.168.203.147@tcp:8:lfs.0 00000100:00100000:2.0:1787826309.747400:0:2388:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-ffff9181c5620000: committing for last_committed 0 gen 2 00000100:00080000:2.0:1787826309.747425:0:2388:0:(import.c:1132:ptlrpc_connect_interpret()) lustre-OST0001-osc-ffff9181c5620000: connect to target with instance 1 00000100:00080000:2.0:1787826309.747437:0:2388:0:(import.c:956:ptlrpc_connect_set_flags()) lustre-OST0001-osc-ffff9181c5620000: Resetting ns_connect_flags to server flags: 0xa0427af2e3440038 00000100:00080000:2.0:1787826309.747454:0:2388:0:(import.c:1213:ptlrpc_connect_interpret()) connected to replayable target: lustre-OST0001_UUID 00000100:00080000:2.0:1787826309.747460:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc6000 lustre-OST0001_UUID: changing import state from CONNECTING to FULL 00000080:00000004:2.0:1787826309.747473:0:2388:0:(lcommon_misc.c:94:cl_ocd_update()) Changing connect_flags: 0xa0427af2e3440038 -> 0xa0427af2e3440038 00000080:00080000:2.0:1787826309.747479:0:2388:0:(lcommon_misc.c:56:cl_init_ea_size()) updating def/max_easize: 72/65536 00000100:00080000:2.0:1787826309.747498:0:2388:0:(recover.c:191:ptlrpc_wake_delayed()) @@@ waking (set 0000000039610416): req@ffff9181c71f1f80 x1874663043003392/t0(0) o13->lustre-OST0001-osc-ffff9181c5620000@192.168.203.147@tcp:7/4 lens 224/368 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826309.747523:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043003264:192.168.203.147@tcp:8:lfs.0 00000100:00100000:1.0:1787826309.747858:0:160988:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d38fc300 going to sleep for 11 seconds 00000100:00100000:1.0:1787826309.756723:0:160988:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f1f80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:160988:1874663043003392:192.168.203.147@tcp:13:lfs.0 00020000:00000004:1.0:1787826309.756799:0:160988:0:(obd_class.h:1116:obd_statfs()) lustre-OST0001-osc-ffff9181c5620000: update ffff9181c354b7c0 cache blocks 901360/958279 objects 261727/262144 00000080:00200080:1.0:1787826309.756890:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.756913:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.756921:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.756925:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.756929:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.756933:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.756937:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.756940:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.756944:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.756947:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.756951:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.756954:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.756958:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.756961:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.756965:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.756969:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.756973:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.756977:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.756981:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.756984:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.756989:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.756992:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.756996:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.756999:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.757003:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.757006:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.757010:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.757014:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.757017:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.757033:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.757039:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.757043:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.757047:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.757050:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.757054:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.757057:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.757061:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.757065:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.757069:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.757072:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.757149:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.757155:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.757160:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.757164:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.757168:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.757171:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.757176:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.757179:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.757183:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.757186:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.757190:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.757194:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.757198:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.757201:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.757205:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.757208:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.757211:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.757214:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.757218:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.757220:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.757224:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.757227:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.757231:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.757234:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.757238:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.757241:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.757245:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.757248:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.757251:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.757255:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.757258:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.757261:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.757265:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.757268:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.757271:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.757274:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.757278:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.757282:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.757287:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.757290:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.757294:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.757297:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.757301:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.757304:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.757308:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.757311:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.757315:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.757318:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.757322:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.757325:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.757328:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.757331:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.757335:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.757338:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.757342:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.757345:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.757348:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.757351:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.757355:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.757358:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.757361:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.757364:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.757368:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.757371:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.757374:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.757378:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.757381:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.757384:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.757388:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.757392:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.757396:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.757399:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.757404:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.757407:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.757411:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.757414:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.757418:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.757421:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.757425:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.757429:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.757433:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.757437:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.757441:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.757444:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200080:1.0:1787826309.757448:0:160988:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc79da1330 00020000:00000080:1.0:1787826309.757452:0:160988:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053e000 uarg=0000000000000000 00000080:00200008:1.0:1787826309.757469:0:160988:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9181d080cb08), flags 100000 00010000:00010000:3.0:1787826310.754181:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826311.777236:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826312.801073:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826313.249169:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663043003648:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826313.249256:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663043003904:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826313.249290:0:2392:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:2.0:1787826313.249295:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:f40147eb-9b53-47d4-b77c-99ceedd9664c:2392:1874663043003520:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826313.249721:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663043003776:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826313.249820:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663043004032:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826313.266233:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:f40147eb-9b53-47d4-b77c-99ceedd9664c:2392:1874663043003520:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826313.266257:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663043003776:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826313.266268:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663043004032:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826313.266277:0:2392:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9181c5620000: committing for last_committed 352187318275 gen 1 00000100:00100000:2.0:1787826313.266282:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663043003648:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826313.266291:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663043003904:192.168.203.147@tcp:400:kworker.0 00010000:00010000:3.0:1787826313.825753:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826314.850770:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:0.0:1787826315.168496:0:161269:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: oleg347-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 00010000:00010000:0.0:1787826315.885211:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:2.0:1787826316.699103:0:161417:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec 00010000:00010000:1.0:1787826316.899229:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826317.921648:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826318.369114:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043004288:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826318.369202:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043004544:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826318.369235:0:2390:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:1.0:1787826318.369242:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:f40147eb-9b53-47d4-b77c-99ceedd9664c:2390:1874663043004160:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826318.369270:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043004416:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826318.369290:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043004672:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826318.377507:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:f40147eb-9b53-47d4-b77c-99ceedd9664c:2390:1874663043004160:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826318.378343:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043004416:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826318.378358:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043004288:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826318.380181:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043004544:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826318.382934:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043004672:192.168.203.147@tcp:400:kworker.0 00000080:00200008:2.0:1787826318.839197:0:161456:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181d080cb08), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1787826318.839214:0:161456:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:2.0:1787826318.839238:0:161456:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826318.839256:0:161456:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1787826318.839271:0:161456:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:2.0:1787826318.839279:0:161456:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826318.839287:0:161456:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:2.0:1787826318.839290:0:161456:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f24a 00000080:00010000:2.0:1787826318.839298:0:161456:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f24a from it ffffb97500ddbcb8 00010000:00010000:2.0:1787826318.839301:0:161456:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1787826318.839316:0:161456:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00000080:00200008:2.0:1787826318.839922:0:161456:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181d080cb08), request_mask 0, flags 0, foreign 0 00000080:00200000:2.0:1787826318.839928:0:161456:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:2.0:1787826318.839938:0:161456:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826318.839946:0:161456:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:2.0:1787826318.839955:0:161456:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00000080:00200000:2.0:1787826318.839973:0:161456:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826318.839977:0:161456:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:2.0:1787826318.839980:0:161456:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f24a 00000080:00010000:2.0:1787826318.839984:0:161456:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f24a from it ffffb97500ddbcb8 00010000:00010000:2.0:1787826318.839988:0:161456:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:2.0:1787826318.839996:0:161456:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00000080:00200000:2.0:1787826318.840085:0:161456:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:2.0:1787826318.840094:0:161456:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826318.840102:0:161456:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00010000:00010000:2.0:1787826318.840115:0:161456:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00000080:00200000:2.0:1787826318.840121:0:161456:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826318.840125:0:161456:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:2.0:1787826318.840127:0:161456:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f24a 00000080:00010000:2.0:1787826318.840131:0:161456:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f24a from it ffffb97500ddbbb0 00010000:00010000:2.0:1787826318.840134:0:161456:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00010000:00010000:2.0:1787826318.840142:0:161456:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00000080:00200000:2.0:1787826318.840150:0:161456:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 44 00000080:00200008:2.0:1787826318.840156:0:161456:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9181d080cb08), flags 100000 00000080:00200080:2.0:1787826318.843475:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.846371:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00800000:00000004:2.0:1787826318.846395:0:161456:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9181c5620000: age 8381, max_age 8389 00000100:00100000:2.0:1787826318.846449:0:161456:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f2680 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161456:1874663043004800:192.168.203.147@tcp:41:lfs.0 00000100:00100000:2.0:1787826318.846550:0:161456:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c4aa6840 going to sleep for 11 seconds 00000100:00100000:2.0:1787826318.849731:0:161456:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f2680 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161456:1874663043004800:192.168.203.147@tcp:41:lfs.0 00800000:00000004:2.0:1787826318.849951:0:161456:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0000-mdc-ffff9181c5620000: update ffff9181c354d178 cache blocks 317223/353529 objects 1023643/1024000 00000080:00200080:2.0:1787826318.850166:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.850185:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00800000:00000004:2.0:1787826318.850192:0:161456:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9181c5620000: age 8381, max_age 8389 00000100:00100000:2.0:1787826318.850229:0:161456:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f2680 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161456:1874663043004928:192.168.203.147@tcp:41:lfs.0 00000100:00100000:2.0:1787826318.850441:0:161456:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c4aa6840 going to sleep for 11 seconds 00000100:00100000:2.0:1787826318.854222:0:161456:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f2680 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161456:1874663043004928:192.168.203.147@tcp:41:lfs.0 00800000:00000004:2.0:1787826318.854251:0:161456:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0001-mdc-ffff9181c5620000: update ffff9181c3548450 cache blocks 317323/353529 objects 1023702/1024000 00000080:00200080:2.0:1787826318.854370:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854386:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854394:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854398:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854402:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854406:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854409:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854413:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854417:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854420:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854423:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854427:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854430:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854433:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854437:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854440:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854444:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854448:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854452:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854455:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854462:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854466:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854470:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854474:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854478:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854481:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854485:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854488:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854492:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854495:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854499:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854502:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854505:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854509:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854513:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854516:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854520:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854524:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854528:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854531:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854535:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854538:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854542:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854545:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854549:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854553:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854557:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854560:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854564:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854568:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854572:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854576:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854580:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854596:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854602:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854605:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854609:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854612:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854615:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854618:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854622:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854625:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854628:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854631:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854635:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854638:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854641:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854644:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854648:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854652:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854656:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854659:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854663:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854667:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854671:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854674:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854678:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854682:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854686:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854689:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854693:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854697:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854700:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854704:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854707:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854711:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854715:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854719:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854722:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854727:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854730:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854733:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854737:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854740:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854803:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854809:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854813:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854817:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854821:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854825:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854828:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854832:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854835:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854838:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854842:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854845:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854849:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854852:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854856:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854859:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854862:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854865:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854869:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854872:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854875:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854878:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854882:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854885:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854888:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854891:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854896:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854899:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854903:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854906:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854910:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00800000:00000080:2.0:1787826318.854912:0:161456:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.854916:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.854922:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00020000:00000004:2.0:1787826318.854929:0:161456:0:(obd_class.h:1090:obd_statfs()) lustre-OST0000-osc-ffff9181c5620000: age 8381, max_age 8389 00000100:00100000:2.0:1787826318.854975:0:161456:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f2680 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161456:1874663043005056:192.168.203.147@tcp:13:lfs.0 00000100:00100000:2.0:1787826318.855057:0:161456:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c4aa6840 going to sleep for 11 seconds 00000100:00100000:2.0:1787826318.857796:0:161456:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f2680 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161456:1874663043005056:192.168.203.147@tcp:13:lfs.0 00020000:00000004:2.0:1787826318.857826:0:161456:0:(obd_class.h:1116:obd_statfs()) lustre-OST0000-osc-ffff9181c5620000: update ffff9181c3549e08 cache blocks 901356/958279 objects 261710/262144 00000080:00200080:2.0:1787826318.857883:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.857899:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00020000:00000004:2.0:1787826318.857907:0:161456:0:(obd_class.h:1090:obd_statfs()) lustre-OST0001-osc-ffff9181c5620000: age 8381, max_age 8389 00000100:00100000:2.0:1787826318.857940:0:161456:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f2680 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161456:1874663043005184:192.168.203.147@tcp:13:lfs.0 00000100:00100000:2.0:1787826318.858259:0:161456:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c4aa6840 going to sleep for 11 seconds 00000100:00100000:2.0:1787826318.861151:0:161456:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f2680 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161456:1874663043005184:192.168.203.147@tcp:13:lfs.0 00020000:00000004:2.0:1787826318.861183:0:161456:0:(obd_class.h:1116:obd_statfs()) lustre-OST0001-osc-ffff9181c5620000: update ffff9181c354b7c0 cache blocks 901360/958279 objects 261727/262144 00000080:00200080:2.0:1787826318.861300:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861318:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861326:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861330:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861335:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861338:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861342:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861345:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861349:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861352:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861356:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861359:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861362:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861365:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861368:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861371:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861375:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861378:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861382:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861385:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861389:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861392:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861395:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861398:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861402:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861405:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861408:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861412:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861415:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861419:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861422:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861425:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861429:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861432:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861435:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861437:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861441:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861444:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861447:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861450:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861453:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861455:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861459:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861461:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861465:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861468:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861471:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861474:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861477:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861480:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861483:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861486:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861489:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861491:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861495:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861498:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861501:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861503:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861506:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861510:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861514:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861518:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861522:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861525:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861528:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861531:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861534:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861537:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861540:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861542:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861545:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861547:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861550:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861552:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861555:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861557:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861560:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861563:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861566:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861569:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861572:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861575:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861578:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861645:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861654:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861659:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861662:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861665:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861668:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861670:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861673:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861676:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861679:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861682:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861685:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861687:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861690:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861693:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861696:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861699:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861701:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861704:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861708:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861711:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861714:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861717:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861721:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861723:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861727:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861730:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861733:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861736:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861740:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861768:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861774:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861777:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861781:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861784:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861787:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861790:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861794:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861796:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861800:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861803:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200080:2.0:1787826318.861806:0:161456:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffdadcb7c10 00020000:00000080:2.0:1787826318.861809:0:161456:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0400 uarg=0000000000000000 00000080:00200008:2.0:1787826318.861823:0:161456:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9181d080cb08), flags 100000 00000080:00200000:1.0:1787826318.907267:0:161472:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:1.0:1787826318.907302:0:161472:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826318.907320:0:161472:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00010000:00010000:1.0:1787826318.907337:0:161472:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00000080:00200000:1.0:1787826318.907346:0:161472:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826318.907353:0:161472:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:1.0:1787826318.907357:0:161472:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f24a 00000080:00010000:1.0:1787826318.907364:0:161472:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f24a from it ffffb97500da3bf8 00010000:00010000:1.0:1787826318.907369:0:161472:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00010000:00010000:1.0:1787826318.907378:0:161472:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00000080:00200000:1.0:1787826318.907393:0:161472:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826318.907407:0:161472:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=d80a.replay-single, dir=[0x200000007:0x1:0x0](ffff9181d080cb08), flags=27 00000080:00200000:1.0:1787826318.907411:0:161472:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=d80a.replay-single, dir=[0x200000007:0x1:0x0](ffff9181d080cb08), intent=0 00000002:00010000:1.0:1787826318.907424:0:161472:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: d80a.replay-single,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: lookup flags 0 00010000:00010000:1.0:1787826318.907470:0:161472:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4e00/0xbef42c0221d6f251 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 161472 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826318.907478:0:161472:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4e00/0xbef42c0221d6f251 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 161472 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826318.907488:0:161472:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4e00/0xbef42c0221d6f251 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 161472 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826318.907520:0:161472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161472:1874663043005312:192.168.203.147@tcp:101:lfs.0 00000100:00100000:1.0:1787826318.907766:0:161472:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d225fa80 going to sleep for 11 seconds 00000100:00100000:2.0:1787826318.915434:0:161472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161472:1874663043005312:192.168.203.147@tcp:101:lfs.0 00010000:00010000:2.0:1787826318.915460:0:161472:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4e00/0xbef42c0221d6f251 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedeaa expref: -99 pid: 161472 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826318.915483:0:161472:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4e00/0xbef42c0221d6f251 lrc: 4/1,0 mode: --/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedeaa expref: -99 pid: 161472 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826318.915492:0:161472:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4e00/0xbef42c0221d6f251 lrc: 4/1,0 mode: --/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedeaa expref: -99 pid: 161472 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826318.915504:0:161472:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4e00/0xbef42c0221d6f251 lrc: 4/1,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cedeaa expref: -99 pid: 161472 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826318.915515:0:161472:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4e00/0xbef42c0221d6f251 lrc: 4/1,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedeaa expref: -99 pid: 161472 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826318.915527:0:161472:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4e00/0xbef42c0221d6f251 lrc: 5/2,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cedeaa expref: -99 pid: 161472 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826318.915536:0:161472:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4e00/0xbef42c0221d6f251 lrc: 5/2,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cedeaa expref: -99 pid: 161472 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826318.915543:0:161472:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4e00/0xbef42c0221d6f251 lrc: 4/1,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cedeaa expref: -99 pid: 161472 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787826318.915552:0:161472:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x10 disposition=0xb, status=0 req@ffff9181c35b1180 x1874663043005312/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 592/688 e 0 to 0 dl 1787826334 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787826318.915565:0:161472:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4e00/0xbef42c0221d6f251 lrc: 3/1,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedeaa expref: -99 pid: 161472 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826318.915572:0:161472:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d1a77c00 type 13 mode 4 res 8590097617/2 (0 0) 00000080:00200000:2.0:1787826318.915579:0:161472:0:(namei.c:952:ll_intent_lock()) intent lock 16 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1787826318.915604:0:161472:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200027cd1:0x2:0x0], updating i_size 4096 00000080:00200000:2.0:1787826318.915607:0:161472:0:(namei.c:147:ll_iget()) got inode: [0x200027cd1:0x2:0x0](ffff9181c83f3248): rc = 0 00000080:00010000:2.0:1787826318.915613:0:161472:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200027cd1:0x2:0x0](ffff9181c83f3248) for lock 0xbef42c0221d6f251 00000080:00010000:2.0:1787826318.915622:0:161472:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200027cd1:0x2:0x0](ffff9181c83f3248) 00000080:00010000:2.0:1787826318.915625:0:161472:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f251 from it ffffb97500da3ac8 00010000:00010000:2.0:1787826318.915628:0:161472:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4e00/0xbef42c0221d6f251 lrc: 3/1,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cedeaa expref: -99 pid: 161472 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826318.915637:0:161472:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4e00/0xbef42c0221d6f251 lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedeaa expref: -99 pid: 161472 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826318.915665:0:161472:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x2:0x0](ffff9181c83f3248), inode mode 41ed mask 1 00000080:00200000:2.0:1787826318.915686:0:161472:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=remote_dir, dir=[0x200027cd1:0x2:0x0](ffff9181c83f3248), file ffff9181c38a3380, open_flags 18800, mode 0 opened 0 00000080:00200000:2.0:1787826318.915692:0:161472:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=remote_dir, dir=[0x200027cd1:0x2:0x0](ffff9181c83f3248), intent=open 00000002:00010000:2.0:1787826318.915712:0:161472:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: remote_dir,[0x0:0x0:0x0]) in obj [0x200027cd1:0x2:0x0], intent: open flags 0304001 00010000:00010000:2.0:1787826318.923907:0:161472:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7400/0xbef42c0221d6f258 lrc: 3/1,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 161472 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826318.923928:0:161472:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7400/0xbef42c0221d6f258 lrc: 3/1,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 161472 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1787826318.923943:0:161472:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1787826318.923957:0:161472:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c71f2680 x1874663043005568/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 656/131856 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1787826318.923981:0:161472:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7400/0xbef42c0221d6f258 lrc: 3/1,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 161472 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826318.924007:0:161472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f2680 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161472:1874663043005568:192.168.203.147@tcp:101:lfs.0 00000100:00100000:2.0:1787826318.924197:0:161472:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c4aa6840 going to sleep for 11 seconds 00000100:00100000:2.0:1787826318.936246:0:161472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f2680 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161472:1874663043005568:192.168.203.147@tcp:101:lfs.0 00010000:00010000:2.0:1787826318.936276:0:161472:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7400/0xbef42c0221d6f258 lrc: 4/1,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedec6 expref: -99 pid: 161472 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826318.936299:0:161472:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7400/0xbef42c0221d6f258 lrc: 4/1,0 mode: --/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedec6 expref: -99 pid: 161472 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826318.936308:0:161472:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7400/0xbef42c0221d6f258 lrc: 4/1,0 mode: --/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedec6 expref: -99 pid: 161472 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826318.936319:0:161472:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7400/0xbef42c0221d6f258 lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cedec6 expref: -99 pid: 161472 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826318.936330:0:161472:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7400/0xbef42c0221d6f258 lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedec6 expref: -99 pid: 161472 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826318.936343:0:161472:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7400/0xbef42c0221d6f258 lrc: 5/2,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cedec6 expref: -99 pid: 161472 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826318.936351:0:161472:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7400/0xbef42c0221d6f258 lrc: 5/2,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cedec6 expref: -99 pid: 161472 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826318.936358:0:161472:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7400/0xbef42c0221d6f258 lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cedec6 expref: -99 pid: 161472 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787826318.936369:0:161472:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x1 disposition=0xb, status=0 req@ffff9181c71f2680 x1874663043005568/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 656/608 e 0 to 0 dl 1787826334 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787826318.936382:0:161472:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7400/0xbef42c0221d6f258 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedec6 expref: -99 pid: 161472 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826318.936392:0:161472:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d1a77c00 type 13 mode 4 res 9663684441/1 (0 0) 00000100:00100000:2.0:1787826318.936435:0:161472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f0380 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161472:1874663043005696:192.168.203.147@tcp:900:lfs.0 00000100:00100000:2.0:1787826318.936505:0:161472:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c4aa6840 going to sleep for 11 seconds 00000100:00100000:2.0:1787826318.939002:0:161472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f0380 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161472:1874663043005696:192.168.203.147@tcp:900:lfs.0 00000002:00010000:2.0:1787826318.939041:0:161472:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200027cd1:0x2:0x0]) in obj [0x240001f59:0x1:0x0], intent: open flags 0304001 00010000:00010000:2.0:1787826318.939097:0:161472:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7e00/0xbef42c0221d6f25f lrc: 3/1,0 mode: --/CR res: [0x240001f59:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 161472 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826318.939107:0:161472:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7e00/0xbef42c0221d6f25f lrc: 3/1,0 mode: --/CR res: [0x240001f59:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 161472 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1787826318.939118:0:161472:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1787826318.939126:0:161472:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c71f0380 x1874663043005952/t0(0) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 592/131800 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1787826318.939134:0:161472:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7e00/0xbef42c0221d6f25f lrc: 3/1,0 mode: --/CR res: [0x240001f59:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 161472 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826318.939153:0:161472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f0380 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161472:1874663043005952:192.168.203.147@tcp:101:lfs.0 00000100:00100000:2.0:1787826318.939318:0:161472:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c4aa6840 going to sleep for 11 seconds 00010000:00010000:1.0:1787826318.946908:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826318.955591:0:161472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f0380 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161472:1874663043005952:192.168.203.147@tcp:101:lfs.0 00010000:00010000:3.0:1787826318.955614:0:161472:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7e00/0xbef42c0221d6f25f lrc: 4/1,0 mode: --/CR res: [0x240001f59:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 161472 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826318.955627:0:161472:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7e00/0xbef42c0221d6f25f lrc: 4/1,0 mode: --/CR res: [0x240001f59:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 161472 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826318.955636:0:161472:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7e00/0xbef42c0221d6f25f lrc: 4/1,0 mode: --/CR res: [0x240001f59:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 161472 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826318.955642:0:161472:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7e00/0xbef42c0221d6f25f lrc: 3/0,0 mode: --/CR res: [0x240001f59:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 161472 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826318.955651:0:161472:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7e00/0xbef42c0221d6f25f lrc: 4/0,0 mode: --/CR res: [0x240001f59:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 161472 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826318.955658:0:161472:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e06f7e00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:3.0:1787826318.955663:0:161472:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7e00/0xbef42c0221d6f25f lrc: 4/0,0 mode: --/CR res: [0x240001f59:0x1:0x0].0x0 bits 0x1/0x1 rrc: 2 type: IBT flags: 0x206400000000 pid: 161472 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826318.955669:0:161472:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7e00/0xbef42c0221d6f25f lrc: 4/0,0 mode: --/CR res: [0x240001f59:0x1:0x0].0x0 bits 0x1/0x1 rrc: 2 type: IBT flags: 0x206400000000 pid: 161472 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826318.955676:0:161472:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7e00/0xbef42c0221d6f25f lrc: 5/0,0 mode: --/CR res: [0x240001f59:0x1:0x0].0x0 bits 0x1/0x1 rrc: 2 type: IBT flags: 0x20e400000000 pid: 161472 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826318.955682:0:161472:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826318.955688:0:161472:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7e00/0xbef42c0221d6f25f lrc: 3/0,0 mode: --/CR res: [0x240001f59:0x1:0x0].0x0 bits 0x1/0x1 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 161472 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826318.955694:0:161472:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7e00/0xbef42c0221d6f25f lrc: 0/0,0 mode: --/CR res: [0x240001f59:0x1:0x0].0x0 bits 0x1/0x1 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 161472 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826318.955727:0:161472:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x1 disposition=0x2b, status=0 req@ffff9181c71f0380 x1874663043005952/t17179869986(17179869986) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826334 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00100000:3.0:1787826318.955740:0:161472:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c71f0380 x1874663043005952/t17179869986(17179869986) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826334 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:3.0:1787826318.955825:0:161472:0:(namei.c:952:ll_intent_lock()) intent lock 1 on i1 [0x200027cd1:0x2:0x0] suppgids 0 -1: rc 0 00000080:00200000:3.0:1787826318.955852:0:161472:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x240001f59:0x1:0x0], updating i_size 4096 00000080:00200000:3.0:1787826318.955855:0:161472:0:(namei.c:147:ll_iget()) got inode: [0x240001f59:0x1:0x0](ffff9181d0801988): rc = 0 00000080:00010000:3.0:1787826318.955861:0:161472:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001f59:0x1:0x0](ffff9181d0801988) for lock 0xbef42c0221d6f258 00000080:00010000:3.0:1787826318.955873:0:161472:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001f59:0x1:0x0](ffff9181d0801988) 00000080:00200008:3.0:1787826318.955886:0:161472:0:(file.c:1090:ll_file_open()) START file remote_dir:[0x240001f59:0x1:0x0](ffff9181d0801988), flags 304000 00000002:00100000:3.0:1787826318.955907:0:161472:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c71f0380 x1874663043005952/t17179869986(17179869986) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 0 dl 1787826334 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00010000:3.0:1787826318.955943:0:161472:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f258 from it ffff9181c553e060 00010000:00010000:3.0:1787826318.955948:0:161472:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7400/0xbef42c0221d6f258 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cedec6 expref: -99 pid: 161472 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826318.955968:0:161472:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7400/0xbef42c0221d6f258 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedec6 expref: -99 pid: 161472 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826318.955987:0:161472:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 44 00000080:00200000:3.0:1787826318.956037:0:161472:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:3.0:1787826318.956050:0:161472:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826318.956063:0:161472:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00010000:00010000:3.0:1787826318.956078:0:161472:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00000080:00200000:3.0:1787826318.956084:0:161472:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826318.956089:0:161472:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:3.0:1787826318.956092:0:161472:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f24a 00000080:00010000:3.0:1787826318.956096:0:161472:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f24a from it ffffb97500da3bf8 00010000:00010000:3.0:1787826318.956099:0:161472:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00010000:00010000:3.0:1787826318.956105:0:161472:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00000080:00200000:3.0:1787826318.956113:0:161472:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826318.956118:0:161472:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80a.replay-single, flags=24 00000080:00200000:3.0:1787826318.956122:0:161472:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x2:0x0](ffff9181c83f3248), inode mode 41ed mask 1 00000080:00200000:3.0:1787826318.956126:0:161472:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=remote_dir, flags=264 00000080:00200000:3.0:1787826318.956130:0:161472:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 44 00000080:00200008:3.0:1787826318.956133:0:161472:0:(file.c:1090:ll_file_open()) START file remote_dir:[0x240001f59:0x1:0x0](ffff9181d0801988), flags 504000 00000080:00200008:3.0:1787826318.956150:0:161472:0:(file.c:479:ll_file_release()) START file remote_dir:[0x240001f59:0x1:0x0](ffff9181d0801988), flags 304000 00000080:00200080:3.0:1787826318.956179:0:161472:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988) cmd=800466af arg=7fff068eba1c 00000080:00200008:3.0:1787826318.956264:0:161472:0:(file.c:479:ll_file_release()) START file remote_dir:[0x240001f59:0x1:0x0](ffff9181d0801988), flags 504000 00000002:00100000:3.0:1787826318.956299:0:161472:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181c71f0380 x1874663043005952/t17179869986(17179869986) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 0 dl 1787826334 ref 1 fl Complete:RPQU/604/ffffffff rc 301/-1 job:'lfs.0' uid:0 gid:0 projid:0 00000020:00100000:3.0:1787826318.956308:0:161472:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:3.0:1787826318.956313:0:161472:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c5636a00 x1874663043006208/t0(0) o35->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/568 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:3.0:1787826318.956333:0:161472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161472:1874663043006208:192.168.203.147@tcp:35:lfs.0 00000100:00100000:3.0:1787826318.956405:0:161472:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c1254a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826318.964178:0:161472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161472:1874663043006208:192.168.203.147@tcp:35:lfs.0 00000002:00100000:3.0:1787826318.964211:0:161472:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181c71f0380 x1874663043005952/t17179869986(17179869986) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 0 dl 1787826334 ref 1 fl Complete:RQU/604/ffffffff rc 301/-1 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1787826318.992834:0:161476:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:2.0:1787826318.992868:0:161476:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826318.992887:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00010000:00010000:2.0:1787826318.992908:0:161476:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00000080:00200000:2.0:1787826318.992916:0:161476:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826318.992925:0:161476:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:2.0:1787826318.992928:0:161476:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f24a 00000080:00010000:2.0:1787826318.992937:0:161476:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f24a from it ffffb97501c3bbf8 00010000:00010000:2.0:1787826318.992941:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00010000:00010000:2.0:1787826318.992949:0:161476:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00000080:00200000:2.0:1787826318.992976:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:2.0:1787826318.992985:0:161476:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80a.replay-single, flags=25 00000080:00200000:2.0:1787826318.992991:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x2:0x0](ffff9181c83f3248), inode mode 41ed mask 1 00000080:00200000:2.0:1787826318.992994:0:161476:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=remote_dir, flags=25 00000080:00200000:2.0:1787826318.992998:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 1 00000080:00200000:2.0:1787826318.993009:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 3 00000080:00200000:2.0:1787826318.993014:0:161476:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f-0, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), file ffff9181d23aa000, open_flags 8042, mode 81a4 opened 0 00000080:00200000:2.0:1787826318.993021:0:161476:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f-0, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), intent=open|creat 00000002:00010000:2.0:1787826318.993041:0:161476:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f-0,[0x240001f59:0x2:0x0]) in obj [0x240001f59:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:2.0:1787826318.993095:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7000/0xbef42c0221d6f266 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826318.993103:0:161476:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7000/0xbef42c0221d6f266 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1787826318.993116:0:161476:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1787826318.993130:0:161476:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c71f0000 x1874663043006464/t0(0) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 592/66264 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1787826318.993140:0:161476:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7000/0xbef42c0221d6f266 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826318.993169:0:161476:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f0000 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043006464:192.168.203.147@tcp:101:createmany.0 00000100:00100000:2.0:1787826318.993298:0:161476:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c4aa6600 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.020349:0:161476:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f0000 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043006464:192.168.203.147@tcp:101:createmany.0 00010000:00010000:2.0:1787826319.020375:0:161476:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7000/0xbef42c0221d6f266 lrc: 4/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedefe expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.020401:0:161476:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7000/0xbef42c0221d6f266 lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedefe expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.020410:0:161476:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7000/0xbef42c0221d6f266 lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedefe expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.020420:0:161476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7000/0xbef42c0221d6f266 lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cedefe expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.020433:0:161476:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7000/0xbef42c0221d6f266 lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedefe expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.020445:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7000/0xbef42c0221d6f266 lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cedefe expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.020476:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7000/0xbef42c0221d6f266 lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cedefe expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.020484:0:161476:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7000/0xbef42c0221d6f266 lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cedefe expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787826319.020495:0:161476:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9181c71f0000 x1874663043006464/t17179869988(17179869988) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826334 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:2.0:1787826319.020508:0:161476:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c71f0000 x1874663043006464/t17179869988(17179869988) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 592/664 e 0 to 0 dl 1787826334 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787826319.020521:0:161476:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7000/0xbef42c0221d6f266 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedefe expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:2.0:1787826319.020560:0:161476:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7000/0xbef42c0221d6f266 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedefe expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.020571:0:161476:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c77f3000 type 13 mode 4 res 9663684441/2 (0 0) 00000080:00200000:2.0:1787826319.020576:0:161476:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x240001f59:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1787826319.020588:0:161476:0:(namei.c:1628:ll_create_it()) VFS Op:name=f-0, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), intent=open|creat 00000080:00200000:2.0:1787826319.020620:0:161476:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001f59:0x2:0x0](ffff9181d0805b88) 00000080:00200000:2.0:1787826319.020691:0:161476:0:(file.c:6843:ll_layout_conf()) [0x240001f59:0x2:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:2.0:1787826319.020696:0:161476:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x240001f59:0x2:0x0](ffff9181d0805b88) 00000080:00010000:2.0:1787826319.020699:0:161476:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001f59:0x2:0x0](ffff9181d0805b88) for lock 0xbef42c0221d6f266 00000080:00010000:2.0:1787826319.020708:0:161476:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001f59:0x2:0x0](ffff9181d0805b88) 00000080:00200008:2.0:1787826319.020720:0:161476:0:(file.c:1090:ll_file_open()) START file f-0:[0x240001f59:0x2:0x0](ffff9181d0805b88), flags 100102 00000002:00100000:2.0:1787826319.020741:0:161476:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c71f0000 x1874663043006464/t17179869988(17179869988) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826334 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:2.0:1787826319.020790:0:161476:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f266 from it ffff9181c553e6c0 00010000:00010000:2.0:1787826319.020794:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7000/0xbef42c0221d6f266 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00cedefe expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.020804:0:161476:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7000/0xbef42c0221d6f266 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00cedefe expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826319.020823:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x2:0x0](ffff9181d0805b88), inode mode 81a4 mask 40 00000080:00200008:2.0:1787826319.020844:0:161476:0:(file.c:479:ll_file_release()) START file f-0:[0x240001f59:0x2:0x0](ffff9181d0805b88), flags 100002 00000002:00100000:2.0:1787826319.020885:0:161476:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181c71f0000 x1874663043006464/t17179869988(17179869988) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826334 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:2.0:1787826319.020896:0:161476:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1787826319.020901:0:161476:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c71f1180 x1874663043006720/t0(0) o35->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:2.0:1787826319.020920:0:161476:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f1180 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043006720:192.168.203.147@tcp:35:createmany.0 00000100:00100000:2.0:1787826319.021000:0:161476:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c4aa6300 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.034214:0:161476:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f1180 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043006720:192.168.203.147@tcp:35:createmany.0 00000002:00100000:2.0:1787826319.034249:0:161476:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181c71f0000 x1874663043006464/t17179869988(17179869988) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826334 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1787826319.034322:0:161476:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:2.0:1787826319.034339:0:161476:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826319.034355:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00010000:00010000:2.0:1787826319.034374:0:161476:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00000080:00200000:2.0:1787826319.034382:0:161476:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826319.034387:0:161476:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:2.0:1787826319.034390:0:161476:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f24a 00000080:00010000:2.0:1787826319.034398:0:161476:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f24a from it ffffb97501c3bbf8 00010000:00010000:2.0:1787826319.034402:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00010000:00010000:2.0:1787826319.034410:0:161476:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00000080:00200000:2.0:1787826319.034424:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.034430:0:161476:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80a.replay-single, flags=25 00000080:00200000:2.0:1787826319.034436:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x2:0x0](ffff9181c83f3248), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.034439:0:161476:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=remote_dir, flags=25 00000080:00200000:2.0:1787826319.034442:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.034454:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 3 00000080:00200000:2.0:1787826319.034458:0:161476:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f-1, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), file ffff9181c5457800, open_flags 8042, mode 81a4 opened 0 00000080:00200000:2.0:1787826319.034465:0:161476:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f-1, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), intent=open|creat 00000002:00010000:2.0:1787826319.034483:0:161476:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f-1,[0x240001f59:0x3:0x0]) in obj [0x240001f59:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:2.0:1787826319.034531:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6c00/0xbef42c0221d6f26d lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.034539:0:161476:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6c00/0xbef42c0221d6f26d lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1787826319.034550:0:161476:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1787826319.034556:0:161476:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c71f3b80 x1874663043006976/t0(0) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1787826319.034564:0:161476:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6c00/0xbef42c0221d6f26d lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826319.034581:0:161476:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f3b80 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043006976:192.168.203.147@tcp:101:createmany.0 00000100:00100000:2.0:1787826319.034704:0:161476:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c4aa6300 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.042173:0:161476:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f3b80 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043006976:192.168.203.147@tcp:101:createmany.0 00010000:00010000:2.0:1787826319.042198:0:161476:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6c00/0xbef42c0221d6f26d lrc: 4/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedf21 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.042228:0:161476:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6c00/0xbef42c0221d6f26d lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedf21 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.042237:0:161476:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6c00/0xbef42c0221d6f26d lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedf21 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.042247:0:161476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6c00/0xbef42c0221d6f26d lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cedf21 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.042257:0:161476:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6c00/0xbef42c0221d6f26d lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedf21 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.042270:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6c00/0xbef42c0221d6f26d lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cedf21 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.042284:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6c00/0xbef42c0221d6f26d lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cedf21 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.042292:0:161476:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6c00/0xbef42c0221d6f26d lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cedf21 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787826319.042302:0:161476:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9181c71f3b80 x1874663043006976/t17179869990(17179869990) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:2.0:1787826319.042315:0:161476:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c71f3b80 x1874663043006976/t17179869990(17179869990) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787826319.042328:0:161476:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6c00/0xbef42c0221d6f26d lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedf21 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:2.0:1787826319.042340:0:161476:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6c00/0xbef42c0221d6f26d lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedf21 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.042350:0:161476:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c77f3000 type 13 mode 4 res 9663684441/3 (0 0) 00000080:00200000:2.0:1787826319.042355:0:161476:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x240001f59:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1787826319.042367:0:161476:0:(namei.c:1628:ll_create_it()) VFS Op:name=f-1, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), intent=open|creat 00000080:00200000:2.0:1787826319.042400:0:161476:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001f59:0x3:0x0](ffff9181d0803a88) 00000080:00200000:2.0:1787826319.042474:0:161476:0:(file.c:6843:ll_layout_conf()) [0x240001f59:0x3:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:2.0:1787826319.042479:0:161476:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x240001f59:0x3:0x0](ffff9181d0803a88) 00000080:00010000:2.0:1787826319.042482:0:161476:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001f59:0x3:0x0](ffff9181d0803a88) for lock 0xbef42c0221d6f26d 00000080:00010000:2.0:1787826319.042491:0:161476:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001f59:0x3:0x0](ffff9181d0803a88) 00000080:00200008:2.0:1787826319.042503:0:161476:0:(file.c:1090:ll_file_open()) START file f-1:[0x240001f59:0x3:0x0](ffff9181d0803a88), flags 100102 00000002:00100000:2.0:1787826319.042523:0:161476:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c71f3b80 x1874663043006976/t17179869990(17179869990) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:2.0:1787826319.042558:0:161476:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f26d from it ffff9181c553e6c0 00010000:00010000:2.0:1787826319.042562:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6c00/0xbef42c0221d6f26d lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00cedf21 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.042572:0:161476:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6c00/0xbef42c0221d6f26d lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00cedf21 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826319.042593:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x3:0x0](ffff9181d0803a88), inode mode 81a4 mask 40 00000080:00200008:2.0:1787826319.042618:0:161476:0:(file.c:479:ll_file_release()) START file f-1:[0x240001f59:0x3:0x0](ffff9181d0803a88), flags 100002 00000002:00100000:2.0:1787826319.042660:0:161476:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181c71f3b80 x1874663043006976/t17179869990(17179869990) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:2.0:1787826319.042671:0:161476:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1787826319.042676:0:161476:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c71f0700 x1874663043007232/t0(0) o35->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:2.0:1787826319.042696:0:161476:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f0700 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043007232:192.168.203.147@tcp:35:createmany.0 00000100:00100000:2.0:1787826319.042798:0:161476:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c4aa63c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826319.051711:0:161476:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f0700 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043007232:192.168.203.147@tcp:35:createmany.0 00000002:00100000:3.0:1787826319.051781:0:161476:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181c71f3b80 x1874663043006976/t17179869990(17179869990) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:3.0:1787826319.051854:0:161476:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:3.0:1787826319.051871:0:161476:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826319.051889:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00010000:00010000:3.0:1787826319.051901:0:161476:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00000080:00200000:3.0:1787826319.051909:0:161476:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826319.051914:0:161476:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:3.0:1787826319.051918:0:161476:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f24a 00000080:00010000:3.0:1787826319.051925:0:161476:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f24a from it ffffb97501c3bbf8 00010000:00010000:3.0:1787826319.051929:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00010000:00010000:3.0:1787826319.051937:0:161476:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 11 lru_type: 1 00000080:00200000:3.0:1787826319.051951:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826319.051968:0:161476:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80a.replay-single, flags=25 00000080:00200000:3.0:1787826319.051974:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x2:0x0](ffff9181c83f3248), inode mode 41ed mask 1 00000080:00200000:3.0:1787826319.051978:0:161476:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=remote_dir, flags=25 00000080:00200000:3.0:1787826319.051982:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 1 00000080:00200000:3.0:1787826319.051994:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 3 00000080:00200000:3.0:1787826319.051998:0:161476:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f-2, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), file ffff9181c5149980, open_flags 8042, mode 81a4 opened 0 00000080:00200000:3.0:1787826319.052006:0:161476:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f-2, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), intent=open|creat 00000002:00010000:3.0:1787826319.052037:0:161476:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f-2,[0x240001f59:0x4:0x0]) in obj [0x240001f59:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:3.0:1787826319.052085:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481600/0xbef42c0221d6f274 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.052093:0:161476:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481600/0xbef42c0221d6f274 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:3.0:1787826319.052105:0:161476:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:3.0:1787826319.052111:0:161476:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c5636a00 x1874663043007488/t0(0) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:3.0:1787826319.052120:0:161476:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481600/0xbef42c0221d6f274 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826319.052139:0:161476:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043007488:192.168.203.147@tcp:101:createmany.0 00000100:00100000:3.0:1787826319.052252:0:161476:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c1254a80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826319.055582:0:161476:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043007488:192.168.203.147@tcp:101:createmany.0 00010000:00010000:3.0:1787826319.055608:0:161476:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481600/0xbef42c0221d6f274 lrc: 4/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedf44 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.055639:0:161476:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481600/0xbef42c0221d6f274 lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedf44 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.055648:0:161476:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481600/0xbef42c0221d6f274 lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedf44 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.055658:0:161476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481600/0xbef42c0221d6f274 lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cedf44 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.055673:0:161476:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481600/0xbef42c0221d6f274 lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedf44 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.055686:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481600/0xbef42c0221d6f274 lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cedf44 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.055695:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481600/0xbef42c0221d6f274 lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cedf44 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.055701:0:161476:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481600/0xbef42c0221d6f274 lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cedf44 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826319.055712:0:161476:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9181c5636a00 x1874663043007488/t17179869992(17179869992) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:3.0:1787826319.055725:0:161476:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c5636a00 x1874663043007488/t17179869992(17179869992) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826319.055739:0:161476:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481600/0xbef42c0221d6f274 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedf44 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:3.0:1787826319.055779:0:161476:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481600/0xbef42c0221d6f274 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedf44 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.055790:0:161476:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c77f3000 type 13 mode 4 res 9663684441/4 (0 0) 00000080:00200000:3.0:1787826319.055795:0:161476:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x240001f59:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:3.0:1787826319.055807:0:161476:0:(namei.c:1628:ll_create_it()) VFS Op:name=f-2, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), intent=open|creat 00000080:00200000:3.0:1787826319.055829:0:161476:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001f59:0x4:0x0](ffff9181d080c2c8) 00000080:00200000:3.0:1787826319.055898:0:161476:0:(file.c:6843:ll_layout_conf()) [0x240001f59:0x4:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:3.0:1787826319.055904:0:161476:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x240001f59:0x4:0x0](ffff9181d080c2c8) 00000080:00010000:3.0:1787826319.055907:0:161476:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001f59:0x4:0x0](ffff9181d080c2c8) for lock 0xbef42c0221d6f274 00000080:00010000:3.0:1787826319.055916:0:161476:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001f59:0x4:0x0](ffff9181d080c2c8) 00000080:00200008:3.0:1787826319.055928:0:161476:0:(file.c:1090:ll_file_open()) START file f-2:[0x240001f59:0x4:0x0](ffff9181d080c2c8), flags 100102 00000002:00100000:3.0:1787826319.055950:0:161476:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c5636a00 x1874663043007488/t17179869992(17179869992) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:3.0:1787826319.055991:0:161476:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f274 from it ffff9181c6edea20 00010000:00010000:3.0:1787826319.055997:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481600/0xbef42c0221d6f274 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00cedf44 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.056007:0:161476:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481600/0xbef42c0221d6f274 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00cedf44 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826319.056040:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x4:0x0](ffff9181d080c2c8), inode mode 81a4 mask 40 00000080:00200008:3.0:1787826319.056066:0:161476:0:(file.c:479:ll_file_release()) START file f-2:[0x240001f59:0x4:0x0](ffff9181d080c2c8), flags 100002 00000002:00100000:3.0:1787826319.056109:0:161476:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181c5636a00 x1874663043007488/t17179869992(17179869992) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:3.0:1787826319.056127:0:161476:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:3.0:1787826319.056132:0:161476:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c5635f80 x1874663043007744/t0(0) o35->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:3.0:1787826319.056153:0:161476:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043007744:192.168.203.147@tcp:35:createmany.0 00000100:00100000:3.0:1787826319.056222:0:161476:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c1254780 going to sleep for 11 seconds 00000100:00100000:3.0:1787826319.070048:0:161476:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043007744:192.168.203.147@tcp:35:createmany.0 00000002:00100000:3.0:1787826319.070084:0:161476:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181c5636a00 x1874663043007488/t17179869992(17179869992) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:3.0:1787826319.070150:0:161476:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:3.0:1787826319.070167:0:161476:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826319.070184:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 11 lru_type: 1 00010000:00010000:3.0:1787826319.070197:0:161476:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 11 lru_type: 1 00000080:00200000:3.0:1787826319.070203:0:161476:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826319.070208:0:161476:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:3.0:1787826319.070211:0:161476:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f24a 00000080:00010000:3.0:1787826319.070217:0:161476:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f24a from it ffffb97501c3bbf8 00010000:00010000:3.0:1787826319.070221:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 11 lru_type: 1 00010000:00010000:3.0:1787826319.070230:0:161476:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 12 lru_type: 1 00000080:00200000:3.0:1787826319.070244:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826319.070250:0:161476:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80a.replay-single, flags=25 00000080:00200000:3.0:1787826319.070254:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x2:0x0](ffff9181c83f3248), inode mode 41ed mask 1 00000080:00200000:3.0:1787826319.070258:0:161476:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=remote_dir, flags=25 00000080:00200000:3.0:1787826319.070261:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 1 00000080:00200000:3.0:1787826319.070272:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 3 00000080:00200000:3.0:1787826319.070276:0:161476:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f-3, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), file ffff9181c5148a80, open_flags 8042, mode 81a4 opened 0 00000080:00200000:3.0:1787826319.070282:0:161476:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f-3, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), intent=open|creat 00000002:00010000:3.0:1787826319.070300:0:161476:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f-3,[0x240001f59:0x5:0x0]) in obj [0x240001f59:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:3.0:1787826319.070346:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f27b lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.070353:0:161476:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f27b lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:3.0:1787826319.070364:0:161476:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:3.0:1787826319.070370:0:161476:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c5637100 x1874663043008000/t0(0) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:3.0:1787826319.070378:0:161476:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f27b lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826319.070395:0:161476:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043008000:192.168.203.147@tcp:101:createmany.0 00000100:00100000:3.0:1787826319.070515:0:161476:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c1254780 going to sleep for 11 seconds 00000100:00100000:3.0:1787826319.081982:0:161476:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043008000:192.168.203.147@tcp:101:createmany.0 00010000:00010000:3.0:1787826319.082010:0:161476:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f27b lrc: 4/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedf67 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.082045:0:161476:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f27b lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedf67 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.082053:0:161476:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f27b lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedf67 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.082063:0:161476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f27b lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cedf67 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.082074:0:161476:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f27b lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedf67 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.082088:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f27b lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cedf67 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.082095:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f27b lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cedf67 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.082103:0:161476:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f27b lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cedf67 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826319.082115:0:161476:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9181c5637100 x1874663043008000/t17179869994(17179869994) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:3.0:1787826319.082128:0:161476:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c5637100 x1874663043008000/t17179869994(17179869994) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826319.082143:0:161476:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f27b lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedf67 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:3.0:1787826319.082156:0:161476:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f27b lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedf67 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.082166:0:161476:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c77f3000 type 13 mode 4 res 9663684441/5 (0 0) 00000080:00200000:3.0:1787826319.082173:0:161476:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x240001f59:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:3.0:1787826319.082186:0:161476:0:(namei.c:1628:ll_create_it()) VFS Op:name=f-3, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), intent=open|creat 00000080:00200000:3.0:1787826319.082213:0:161476:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001f59:0x5:0x0](ffff9181d080d348) 00000080:00200000:3.0:1787826319.082327:0:161476:0:(file.c:6843:ll_layout_conf()) [0x240001f59:0x5:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:3.0:1787826319.082333:0:161476:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x240001f59:0x5:0x0](ffff9181d080d348) 00000080:00010000:3.0:1787826319.082336:0:161476:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001f59:0x5:0x0](ffff9181d080d348) for lock 0xbef42c0221d6f27b 00000080:00010000:3.0:1787826319.082344:0:161476:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001f59:0x5:0x0](ffff9181d080d348) 00000080:00200008:3.0:1787826319.082355:0:161476:0:(file.c:1090:ll_file_open()) START file f-3:[0x240001f59:0x5:0x0](ffff9181d080d348), flags 100102 00000002:00100000:3.0:1787826319.082375:0:161476:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c5637100 x1874663043008000/t17179869994(17179869994) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:3.0:1787826319.082408:0:161476:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f27b from it ffff9181c6edea20 00010000:00010000:3.0:1787826319.082413:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f27b lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00cedf67 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.082422:0:161476:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f27b lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00cedf67 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826319.082442:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x5:0x0](ffff9181d080d348), inode mode 81a4 mask 40 00000080:00200008:3.0:1787826319.082462:0:161476:0:(file.c:479:ll_file_release()) START file f-3:[0x240001f59:0x5:0x0](ffff9181d080d348), flags 100002 00000002:00100000:3.0:1787826319.082503:0:161476:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181c5637100 x1874663043008000/t17179869994(17179869994) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:3.0:1787826319.082517:0:161476:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:3.0:1787826319.082522:0:161476:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c5637800 x1874663043008256/t0(0) o35->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:3.0:1787826319.082543:0:161476:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043008256:192.168.203.147@tcp:35:createmany.0 00000100:00100000:3.0:1787826319.082616:0:161476:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5c053c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826319.092647:0:161476:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043008256:192.168.203.147@tcp:35:createmany.0 00000002:00100000:3.0:1787826319.092683:0:161476:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181c5637100 x1874663043008000/t17179869994(17179869994) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:3.0:1787826319.094035:0:161476:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:3.0:1787826319.094296:0:161476:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826319.094325:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 12 lru_type: 1 00010000:00010000:3.0:1787826319.094340:0:161476:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 12 lru_type: 1 00000080:00200000:3.0:1787826319.094348:0:161476:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826319.094355:0:161476:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:3.0:1787826319.094358:0:161476:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f24a 00000080:00010000:3.0:1787826319.094365:0:161476:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f24a from it ffffb97501c3bbf8 00010000:00010000:3.0:1787826319.094368:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 12 lru_type: 1 00010000:00010000:3.0:1787826319.094377:0:161476:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 13 lru_type: 1 00000080:00200000:3.0:1787826319.094393:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826319.094400:0:161476:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80a.replay-single, flags=25 00000080:00200000:3.0:1787826319.094405:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x2:0x0](ffff9181c83f3248), inode mode 41ed mask 1 00000080:00200000:3.0:1787826319.094409:0:161476:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=remote_dir, flags=25 00000080:00200000:3.0:1787826319.094412:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 1 00000080:00200000:3.0:1787826319.094425:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 3 00000080:00200000:3.0:1787826319.094429:0:161476:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f-4, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), file ffff9181c4d7fc80, open_flags 8042, mode 81a4 opened 0 00000080:00200000:3.0:1787826319.094436:0:161476:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f-4, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), intent=open|creat 00000002:00010000:3.0:1787826319.094453:0:161476:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f-4,[0x240001f59:0x6:0x0]) in obj [0x240001f59:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:3.0:1787826319.094500:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f282 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.094508:0:161476:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f282 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:3.0:1787826319.094519:0:161476:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:3.0:1787826319.094529:0:161476:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c5635500 x1874663043008512/t0(0) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:3.0:1787826319.094718:0:161476:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f282 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826319.094741:0:161476:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043008512:192.168.203.147@tcp:101:createmany.0 00000100:00100000:3.0:1787826319.094986:0:161476:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5c053c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.103928:0:161476:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043008512:192.168.203.147@tcp:101:createmany.0 00010000:00010000:2.0:1787826319.103956:0:161476:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f282 lrc: 4/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedf8a expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.103991:0:161476:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f282 lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedf8a expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.104000:0:161476:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f282 lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedf8a expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.104011:0:161476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f282 lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cedf8a expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.104023:0:161476:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f282 lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedf8a expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.104036:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f282 lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cedf8a expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.104045:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f282 lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cedf8a expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.104052:0:161476:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f282 lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cedf8a expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787826319.104066:0:161476:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9181c5635500 x1874663043008512/t17179869996(17179869996) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:2.0:1787826319.104079:0:161476:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c5635500 x1874663043008512/t17179869996(17179869996) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787826319.104091:0:161476:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f282 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedf8a expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:2.0:1787826319.104103:0:161476:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f282 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedf8a expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.104112:0:161476:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c77f3000 type 13 mode 4 res 9663684441/6 (0 0) 00000080:00200000:2.0:1787826319.104118:0:161476:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x240001f59:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1787826319.105237:0:161476:0:(namei.c:1628:ll_create_it()) VFS Op:name=f-4, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), intent=open|creat 00000080:00200000:2.0:1787826319.105283:0:161476:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001f59:0x6:0x0](ffff9181d0803248) 00000080:00200000:2.0:1787826319.105360:0:161476:0:(file.c:6843:ll_layout_conf()) [0x240001f59:0x6:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:2.0:1787826319.105367:0:161476:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x240001f59:0x6:0x0](ffff9181d0803248) 00000080:00010000:2.0:1787826319.105370:0:161476:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001f59:0x6:0x0](ffff9181d0803248) for lock 0xbef42c0221d6f282 00000080:00010000:2.0:1787826319.105379:0:161476:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001f59:0x6:0x0](ffff9181d0803248) 00000080:00200008:2.0:1787826319.105392:0:161476:0:(file.c:1090:ll_file_open()) START file f-4:[0x240001f59:0x6:0x0](ffff9181d0803248), flags 100102 00000002:00100000:2.0:1787826319.105416:0:161476:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c5635500 x1874663043008512/t17179869996(17179869996) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:2.0:1787826319.105450:0:161476:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f282 from it ffff9181c6edea20 00010000:00010000:2.0:1787826319.105455:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f282 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00cedf8a expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.105464:0:161476:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f282 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00cedf8a expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826319.105484:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x6:0x0](ffff9181d0803248), inode mode 81a4 mask 40 00000080:00200008:2.0:1787826319.105512:0:161476:0:(file.c:479:ll_file_release()) START file f-4:[0x240001f59:0x6:0x0](ffff9181d0803248), flags 100002 00000002:00100000:2.0:1787826319.105556:0:161476:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181c5635500 x1874663043008512/t17179869996(17179869996) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:2.0:1787826319.105567:0:161476:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1787826319.105572:0:161476:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c71f3800 x1874663043008768/t0(0) o35->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:2.0:1787826319.105593:0:161476:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f3800 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043008768:192.168.203.147@tcp:35:createmany.0 00000100:00100000:0.0:1787826319.107448:0:161476:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c4aa6840 going to sleep for 11 seconds 00000100:00100000:3.0:1787826319.112884:0:161476:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f3800 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043008768:192.168.203.147@tcp:35:createmany.0 00000002:00100000:3.0:1787826319.112920:0:161476:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181c5635500 x1874663043008512/t17179869996(17179869996) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:3.0:1787826319.113002:0:161476:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:3.0:1787826319.113020:0:161476:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826319.113040:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 13 lru_type: 1 00010000:00010000:3.0:1787826319.113054:0:161476:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 13 lru_type: 1 00000080:00200000:3.0:1787826319.113061:0:161476:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826319.113067:0:161476:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:3.0:1787826319.113070:0:161476:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f24a 00000080:00010000:3.0:1787826319.113076:0:161476:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f24a from it ffffb97501c3bbf8 00010000:00010000:3.0:1787826319.113080:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 13 lru_type: 1 00010000:00010000:3.0:1787826319.113089:0:161476:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 14 lru_type: 1 00000080:00200000:3.0:1787826319.113103:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826319.113110:0:161476:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80a.replay-single, flags=25 00000080:00200000:3.0:1787826319.113115:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x2:0x0](ffff9181c83f3248), inode mode 41ed mask 1 00000080:00200000:3.0:1787826319.113119:0:161476:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=remote_dir, flags=25 00000080:00200000:3.0:1787826319.113122:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 1 00000080:00200000:3.0:1787826319.113132:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 3 00000080:00200000:3.0:1787826319.113143:0:161476:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f-5, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), file ffff9181c5148a80, open_flags 8042, mode 81a4 opened 0 00000080:00200000:3.0:1787826319.113151:0:161476:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f-5, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), intent=open|creat 00000002:00010000:3.0:1787826319.113168:0:161476:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f-5,[0x240001f59:0x7:0x0]) in obj [0x240001f59:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:3.0:1787826319.113219:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481200/0xbef42c0221d6f289 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.113226:0:161476:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481200/0xbef42c0221d6f289 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:3.0:1787826319.113237:0:161476:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:3.0:1787826319.113243:0:161476:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c5637b80 x1874663043009024/t0(0) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:3.0:1787826319.113251:0:161476:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481200/0xbef42c0221d6f289 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826319.113268:0:161476:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043009024:192.168.203.147@tcp:101:createmany.0 00000100:00100000:3.0:1787826319.113615:0:161476:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5c059c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826319.120001:0:161476:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043009024:192.168.203.147@tcp:101:createmany.0 00010000:00010000:3.0:1787826319.120044:0:161476:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481200/0xbef42c0221d6f289 lrc: 4/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedfad expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.120073:0:161476:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481200/0xbef42c0221d6f289 lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedfad expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.120083:0:161476:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481200/0xbef42c0221d6f289 lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedfad expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.120095:0:161476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481200/0xbef42c0221d6f289 lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cedfad expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.120106:0:161476:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481200/0xbef42c0221d6f289 lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedfad expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.120121:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481200/0xbef42c0221d6f289 lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cedfad expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.120129:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481200/0xbef42c0221d6f289 lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cedfad expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.120137:0:161476:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481200/0xbef42c0221d6f289 lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cedfad expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826319.120148:0:161476:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9181c5637b80 x1874663043009024/t17179869998(17179869998) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:3.0:1787826319.120162:0:161476:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c5637b80 x1874663043009024/t17179869998(17179869998) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826319.120176:0:161476:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481200/0xbef42c0221d6f289 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedfad expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:3.0:1787826319.120190:0:161476:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481200/0xbef42c0221d6f289 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedfad expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.120200:0:161476:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c77f3000 type 13 mode 4 res 9663684441/7 (0 0) 00000080:00200000:3.0:1787826319.120206:0:161476:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x240001f59:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:3.0:1787826319.120220:0:161476:0:(namei.c:1628:ll_create_it()) VFS Op:name=f-5, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), intent=open|creat 00000080:00200000:3.0:1787826319.120248:0:161476:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001f59:0x7:0x0](ffff9181d080ec08) 00000080:00200000:3.0:1787826319.120317:0:161476:0:(file.c:6843:ll_layout_conf()) [0x240001f59:0x7:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:3.0:1787826319.120322:0:161476:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x240001f59:0x7:0x0](ffff9181d080ec08) 00000080:00010000:3.0:1787826319.120325:0:161476:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001f59:0x7:0x0](ffff9181d080ec08) for lock 0xbef42c0221d6f289 00000080:00010000:3.0:1787826319.120334:0:161476:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001f59:0x7:0x0](ffff9181d080ec08) 00000080:00200008:3.0:1787826319.120347:0:161476:0:(file.c:1090:ll_file_open()) START file f-5:[0x240001f59:0x7:0x0](ffff9181d080ec08), flags 100102 00000002:00100000:3.0:1787826319.120369:0:161476:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c5637b80 x1874663043009024/t17179869998(17179869998) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:3.0:1787826319.120404:0:161476:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f289 from it ffff9181c6ede9c0 00010000:00010000:3.0:1787826319.120409:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481200/0xbef42c0221d6f289 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00cedfad expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.120419:0:161476:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481200/0xbef42c0221d6f289 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00cedfad expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826319.120440:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x7:0x0](ffff9181d080ec08), inode mode 81a4 mask 40 00000080:00200008:3.0:1787826319.120469:0:161476:0:(file.c:479:ll_file_release()) START file f-5:[0x240001f59:0x7:0x0](ffff9181d080ec08), flags 100002 00000002:00100000:3.0:1787826319.120690:0:161476:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181c5637b80 x1874663043009024/t17179869998(17179869998) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:3.0:1787826319.120711:0:161476:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:3.0:1787826319.120716:0:161476:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c5636300 x1874663043009280/t0(0) o35->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:3.0:1787826319.120736:0:161476:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636300 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043009280:192.168.203.147@tcp:35:createmany.0 00000100:00100000:3.0:1787826319.120849:0:161476:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5c05840 going to sleep for 11 seconds 00000100:00100000:3.0:1787826319.130503:0:161476:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636300 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043009280:192.168.203.147@tcp:35:createmany.0 00000002:00100000:3.0:1787826319.130555:0:161476:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181c5637b80 x1874663043009024/t17179869998(17179869998) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:3.0:1787826319.130639:0:161476:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:3.0:1787826319.130659:0:161476:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826319.130679:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 14 lru_type: 1 00010000:00010000:3.0:1787826319.130695:0:161476:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 14 lru_type: 1 00000080:00200000:3.0:1787826319.130703:0:161476:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826319.130710:0:161476:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:3.0:1787826319.130713:0:161476:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f24a 00000080:00010000:3.0:1787826319.130721:0:161476:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f24a from it ffffb97501c3bbf8 00010000:00010000:3.0:1787826319.130726:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 14 lru_type: 1 00010000:00010000:3.0:1787826319.130736:0:161476:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 15 lru_type: 1 00000080:00200000:3.0:1787826319.130786:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826319.130795:0:161476:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80a.replay-single, flags=25 00000080:00200000:3.0:1787826319.130801:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x2:0x0](ffff9181c83f3248), inode mode 41ed mask 1 00000080:00200000:3.0:1787826319.130805:0:161476:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=remote_dir, flags=25 00000080:00200000:3.0:1787826319.130808:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 1 00000080:00200000:3.0:1787826319.130823:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 3 00000080:00200000:3.0:1787826319.130827:0:161476:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f-6, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), file ffff9181c5148480, open_flags 8042, mode 81a4 opened 0 00000080:00200000:3.0:1787826319.130836:0:161476:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f-6, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), intent=open|creat 00000002:00010000:3.0:1787826319.130855:0:161476:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f-6,[0x240001f59:0x8:0x0]) in obj [0x240001f59:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:3.0:1787826319.130908:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f290 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.130921:0:161476:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f290 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:3.0:1787826319.130933:0:161476:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:3.0:1787826319.130940:0:161476:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c5634a80 x1874663043009536/t0(0) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:3.0:1787826319.130950:0:161476:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f290 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826319.130980:0:161476:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043009536:192.168.203.147@tcp:101:createmany.0 00000100:00100000:3.0:1787826319.131124:0:161476:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5c05840 going to sleep for 11 seconds 00000100:00100000:3.0:1787826319.136628:0:161476:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043009536:192.168.203.147@tcp:101:createmany.0 00010000:00010000:3.0:1787826319.136656:0:161476:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f290 lrc: 4/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedfd0 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.136686:0:161476:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f290 lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedfd0 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.136708:0:161476:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f290 lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedfd0 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.136720:0:161476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f290 lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cedfd0 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.136731:0:161476:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f290 lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedfd0 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.136791:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f290 lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cedfd0 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.136801:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f290 lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cedfd0 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.136808:0:161476:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f290 lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cedfd0 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826319.136819:0:161476:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9181c5634a80 x1874663043009536/t17179870000(17179870000) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:3.0:1787826319.136831:0:161476:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c5634a80 x1874663043009536/t17179870000(17179870000) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826319.136844:0:161476:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f290 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedfd0 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:3.0:1787826319.136857:0:161476:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f290 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedfd0 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.136868:0:161476:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c77f3000 type 13 mode 4 res 9663684441/8 (0 0) 00000080:00200000:3.0:1787826319.136874:0:161476:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x240001f59:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:3.0:1787826319.136888:0:161476:0:(namei.c:1628:ll_create_it()) VFS Op:name=f-6, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), intent=open|creat 00000080:00200000:3.0:1787826319.136916:0:161476:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001f59:0x8:0x0](ffff9181d08080c8) 00000080:00200000:3.0:1787826319.137068:0:161476:0:(file.c:6843:ll_layout_conf()) [0x240001f59:0x8:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:3.0:1787826319.137073:0:161476:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x240001f59:0x8:0x0](ffff9181d08080c8) 00000080:00010000:3.0:1787826319.137077:0:161476:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001f59:0x8:0x0](ffff9181d08080c8) for lock 0xbef42c0221d6f290 00000080:00010000:3.0:1787826319.137087:0:161476:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001f59:0x8:0x0](ffff9181d08080c8) 00000080:00200008:3.0:1787826319.137101:0:161476:0:(file.c:1090:ll_file_open()) START file f-6:[0x240001f59:0x8:0x0](ffff9181d08080c8), flags 100102 00000002:00100000:3.0:1787826319.137122:0:161476:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c5634a80 x1874663043009536/t17179870000(17179870000) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:3.0:1787826319.137159:0:161476:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f290 from it ffff9181c6ede9c0 00010000:00010000:3.0:1787826319.137163:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f290 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00cedfd0 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.137173:0:161476:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f290 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00cedfd0 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826319.137193:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x8:0x0](ffff9181d08080c8), inode mode 81a4 mask 40 00000080:00200008:3.0:1787826319.137223:0:161476:0:(file.c:479:ll_file_release()) START file f-6:[0x240001f59:0x8:0x0](ffff9181d08080c8), flags 100002 00000002:00100000:3.0:1787826319.137266:0:161476:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181c5634a80 x1874663043009536/t17179870000(17179870000) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:3.0:1787826319.137277:0:161476:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:3.0:1787826319.137281:0:161476:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c5636d80 x1874663043009792/t0(0) o35->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:3.0:1787826319.137302:0:161476:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043009792:192.168.203.147@tcp:35:createmany.0 00000100:00100000:3.0:1787826319.137386:0:161476:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c32f6d80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826319.141377:0:161476:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043009792:192.168.203.147@tcp:35:createmany.0 00000002:00100000:3.0:1787826319.141418:0:161476:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181c5634a80 x1874663043009536/t17179870000(17179870000) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:3.0:1787826319.141489:0:161476:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:2.0:1787826319.142801:0:161476:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826319.142828:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 15 lru_type: 1 00010000:00010000:2.0:1787826319.142847:0:161476:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 15 lru_type: 1 00000080:00200000:2.0:1787826319.142856:0:161476:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826319.142866:0:161476:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:2.0:1787826319.142870:0:161476:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f24a 00000080:00010000:2.0:1787826319.142878:0:161476:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f24a from it ffffb97501c3bbf8 00010000:00010000:2.0:1787826319.142882:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 15 lru_type: 1 00010000:00010000:2.0:1787826319.142891:0:161476:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 16 lru_type: 1 00000080:00200000:2.0:1787826319.142909:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.142918:0:161476:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80a.replay-single, flags=25 00000080:00200000:2.0:1787826319.142926:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x2:0x0](ffff9181c83f3248), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.142930:0:161476:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=remote_dir, flags=25 00000080:00200000:2.0:1787826319.142933:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.142976:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 3 00000080:00200000:2.0:1787826319.142981:0:161476:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f-7, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), file ffff9181c5148a80, open_flags 8042, mode 81a4 opened 0 00000080:00200000:2.0:1787826319.142989:0:161476:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f-7, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), intent=open|creat 00000002:00010000:2.0:1787826319.143012:0:161476:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f-7,[0x240001f59:0x9:0x0]) in obj [0x240001f59:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:2.0:1787826319.143069:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7200/0xbef42c0221d6f297 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.143099:0:161476:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7200/0xbef42c0221d6f297 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1787826319.143113:0:161476:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1787826319.143124:0:161476:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c71f0380 x1874663043010048/t0(0) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1787826319.143134:0:161476:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7200/0xbef42c0221d6f297 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826319.143158:0:161476:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f0380 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043010048:192.168.203.147@tcp:101:createmany.0 00000100:00100000:2.0:1787826319.143306:0:161476:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c4aa6a80 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.148537:0:161476:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f0380 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043010048:192.168.203.147@tcp:101:createmany.0 00010000:00010000:2.0:1787826319.148566:0:161476:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7200/0xbef42c0221d6f297 lrc: 4/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedff3 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.148608:0:161476:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7200/0xbef42c0221d6f297 lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedff3 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.148618:0:161476:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7200/0xbef42c0221d6f297 lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedff3 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.148628:0:161476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7200/0xbef42c0221d6f297 lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cedff3 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.148639:0:161476:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7200/0xbef42c0221d6f297 lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedff3 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.148657:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7200/0xbef42c0221d6f297 lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cedff3 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.148666:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7200/0xbef42c0221d6f297 lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cedff3 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.148674:0:161476:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7200/0xbef42c0221d6f297 lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cedff3 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787826319.148684:0:161476:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9181c71f0380 x1874663043010048/t17179870002(17179870002) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:2.0:1787826319.148952:0:161476:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c71f0380 x1874663043010048/t17179870002(17179870002) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787826319.148978:0:161476:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7200/0xbef42c0221d6f297 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedff3 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:2.0:1787826319.148992:0:161476:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7200/0xbef42c0221d6f297 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedff3 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.149003:0:161476:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c77f3000 type 13 mode 4 res 9663684441/9 (0 0) 00000080:00200000:2.0:1787826319.149018:0:161476:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x240001f59:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1787826319.149031:0:161476:0:(namei.c:1628:ll_create_it()) VFS Op:name=f-7, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), intent=open|creat 00000080:00200000:2.0:1787826319.149057:0:161476:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001f59:0x9:0x0](ffff9181d0802a08) 00000080:00200000:2.0:1787826319.149128:0:161476:0:(file.c:6843:ll_layout_conf()) [0x240001f59:0x9:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:2.0:1787826319.149133:0:161476:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x240001f59:0x9:0x0](ffff9181d0802a08) 00000080:00010000:2.0:1787826319.149136:0:161476:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001f59:0x9:0x0](ffff9181d0802a08) for lock 0xbef42c0221d6f297 00000080:00010000:2.0:1787826319.149145:0:161476:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001f59:0x9:0x0](ffff9181d0802a08) 00000080:00200008:2.0:1787826319.149158:0:161476:0:(file.c:1090:ll_file_open()) START file f-7:[0x240001f59:0x9:0x0](ffff9181d0802a08), flags 100102 00000002:00100000:2.0:1787826319.149180:0:161476:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c71f0380 x1874663043010048/t17179870002(17179870002) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:2.0:1787826319.149215:0:161476:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f297 from it ffff9181c2b50420 00010000:00010000:2.0:1787826319.149220:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7200/0xbef42c0221d6f297 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00cedff3 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.149230:0:161476:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7200/0xbef42c0221d6f297 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00cedff3 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826319.149252:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x9:0x0](ffff9181d0802a08), inode mode 81a4 mask 40 00000080:00200008:2.0:1787826319.149281:0:161476:0:(file.c:479:ll_file_release()) START file f-7:[0x240001f59:0x9:0x0](ffff9181d0802a08), flags 100002 00000002:00100000:2.0:1787826319.149321:0:161476:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181c71f0380 x1874663043010048/t17179870002(17179870002) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:2.0:1787826319.149332:0:161476:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1787826319.149337:0:161476:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c71f2680 x1874663043010304/t0(0) o35->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:2.0:1787826319.149357:0:161476:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f2680 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043010304:192.168.203.147@tcp:35:createmany.0 00000100:00100000:2.0:1787826319.149433:0:161476:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c4aa6240 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.154387:0:161476:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f2680 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043010304:192.168.203.147@tcp:35:createmany.0 00000002:00100000:2.0:1787826319.154423:0:161476:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181c71f0380 x1874663043010048/t17179870002(17179870002) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1787826319.154499:0:161476:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:2.0:1787826319.154518:0:161476:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826319.154537:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 16 lru_type: 1 00010000:00010000:2.0:1787826319.154553:0:161476:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 16 lru_type: 1 00000080:00200000:2.0:1787826319.154562:0:161476:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826319.154567:0:161476:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:2.0:1787826319.154571:0:161476:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f24a 00000080:00010000:2.0:1787826319.154579:0:161476:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f24a from it ffffb97501c3bbf8 00010000:00010000:2.0:1787826319.154663:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 16 lru_type: 1 00010000:00010000:2.0:1787826319.154674:0:161476:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 17 lru_type: 1 00000080:00200000:2.0:1787826319.154690:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.154901:0:161476:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80a.replay-single, flags=25 00000080:00200000:2.0:1787826319.154908:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x2:0x0](ffff9181c83f3248), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.154913:0:161476:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=remote_dir, flags=25 00000080:00200000:2.0:1787826319.154916:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.154931:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 3 00000080:00200000:2.0:1787826319.154936:0:161476:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f-8, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), file ffff9181c5457500, open_flags 8042, mode 81a4 opened 0 00000080:00200000:2.0:1787826319.154944:0:161476:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f-8, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), intent=open|creat 00000002:00010000:2.0:1787826319.154976:0:161476:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f-8,[0x240001f59:0xa:0x0]) in obj [0x240001f59:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:2.0:1787826319.155099:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6800/0xbef42c0221d6f29e lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.155113:0:161476:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6800/0xbef42c0221d6f29e lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1787826319.155126:0:161476:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1787826319.155133:0:161476:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c71f1f80 x1874663043010560/t0(0) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1787826319.155144:0:161476:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6800/0xbef42c0221d6f29e lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826319.155166:0:161476:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f1f80 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043010560:192.168.203.147@tcp:101:createmany.0 00000100:00100000:2.0:1787826319.155351:0:161476:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c4aa6240 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.159233:0:161476:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f1f80 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043010560:192.168.203.147@tcp:101:createmany.0 00010000:00010000:2.0:1787826319.159262:0:161476:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6800/0xbef42c0221d6f29e lrc: 4/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee016 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.159291:0:161476:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6800/0xbef42c0221d6f29e lrc: 4/0,1 mode: --/PR res: [0x240001f59:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee016 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.159300:0:161476:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6800/0xbef42c0221d6f29e lrc: 4/0,1 mode: --/PR res: [0x240001f59:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee016 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.159311:0:161476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6800/0xbef42c0221d6f29e lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee016 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.159322:0:161476:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6800/0xbef42c0221d6f29e lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee016 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.159342:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6800/0xbef42c0221d6f29e lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee016 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.159351:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6800/0xbef42c0221d6f29e lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee016 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.159358:0:161476:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6800/0xbef42c0221d6f29e lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee016 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787826319.159369:0:161476:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9181c71f1f80 x1874663043010560/t17179870004(17179870004) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:2.0:1787826319.159382:0:161476:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c71f1f80 x1874663043010560/t17179870004(17179870004) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787826319.159394:0:161476:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6800/0xbef42c0221d6f29e lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee016 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:2.0:1787826319.159406:0:161476:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6800/0xbef42c0221d6f29e lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee016 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.159416:0:161476:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c77f3000 type 13 mode 4 res 9663684441/10 (0 0) 00000080:00200000:2.0:1787826319.159422:0:161476:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x240001f59:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1787826319.159434:0:161476:0:(namei.c:1628:ll_create_it()) VFS Op:name=f-8, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), intent=open|creat 00000080:00200000:2.0:1787826319.159462:0:161476:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001f59:0xa:0x0](ffff9181d0804b08) 00000080:00200000:2.0:1787826319.159526:0:161476:0:(file.c:6843:ll_layout_conf()) [0x240001f59:0xa:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:2.0:1787826319.159529:0:161476:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x240001f59:0xa:0x0](ffff9181d0804b08) 00000080:00010000:2.0:1787826319.159533:0:161476:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001f59:0xa:0x0](ffff9181d0804b08) for lock 0xbef42c0221d6f29e 00000080:00010000:2.0:1787826319.159541:0:161476:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001f59:0xa:0x0](ffff9181d0804b08) 00000080:00200008:2.0:1787826319.159554:0:161476:0:(file.c:1090:ll_file_open()) START file f-8:[0x240001f59:0xa:0x0](ffff9181d0804b08), flags 100102 00000002:00100000:2.0:1787826319.159571:0:161476:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c71f1f80 x1874663043010560/t17179870004(17179870004) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:2.0:1787826319.159605:0:161476:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f29e from it ffff9181c2b50420 00010000:00010000:2.0:1787826319.159610:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6800/0xbef42c0221d6f29e lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00cee016 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.159619:0:161476:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6800/0xbef42c0221d6f29e lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00cee016 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826319.159637:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0xa:0x0](ffff9181d0804b08), inode mode 81a4 mask 40 00000080:00200008:2.0:1787826319.159663:0:161476:0:(file.c:479:ll_file_release()) START file f-8:[0x240001f59:0xa:0x0](ffff9181d0804b08), flags 100002 00000002:00100000:2.0:1787826319.159697:0:161476:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181c71f1f80 x1874663043010560/t17179870004(17179870004) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:2.0:1787826319.159706:0:161476:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1787826319.159710:0:161476:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c71f1c00 x1874663043010816/t0(0) o35->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:2.0:1787826319.159729:0:161476:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f1c00 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043010816:192.168.203.147@tcp:35:createmany.0 00000100:00100000:2.0:1787826319.159822:0:161476:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c4aa6480 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.163575:0:161476:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f1c00 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043010816:192.168.203.147@tcp:35:createmany.0 00000002:00100000:2.0:1787826319.163621:0:161476:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181c71f1f80 x1874663043010560/t17179870004(17179870004) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1787826319.163698:0:161476:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:2.0:1787826319.163717:0:161476:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826319.163785:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 17 lru_type: 1 00010000:00010000:2.0:1787826319.163801:0:161476:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 17 lru_type: 1 00000080:00200000:2.0:1787826319.163808:0:161476:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826319.163813:0:161476:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:2.0:1787826319.163817:0:161476:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f24a 00000080:00010000:2.0:1787826319.163824:0:161476:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f24a from it ffffb97501c3bbf8 00010000:00010000:2.0:1787826319.163827:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 17 lru_type: 1 00010000:00010000:2.0:1787826319.163836:0:161476:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 18 lru_type: 1 00000080:00200000:2.0:1787826319.163850:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.163857:0:161476:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80a.replay-single, flags=25 00000080:00200000:2.0:1787826319.163862:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x2:0x0](ffff9181c83f3248), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.163866:0:161476:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=remote_dir, flags=25 00000080:00200000:2.0:1787826319.163870:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.163883:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 3 00000080:00200000:2.0:1787826319.163888:0:161476:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f-9, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), file ffff9181c5456d80, open_flags 8042, mode 81a4 opened 0 00000080:00200000:2.0:1787826319.163895:0:161476:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f-9, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), intent=open|creat 00000002:00010000:2.0:1787826319.163912:0:161476:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f-9,[0x240001f59:0xb:0x0]) in obj [0x240001f59:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:2.0:1787826319.163970:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f2a5 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.163983:0:161476:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f2a5 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1787826319.163993:0:161476:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1787826319.164000:0:161476:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c71f0e00 x1874663043011072/t0(0) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1787826319.164008:0:161476:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f2a5 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826319.164025:0:161476:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f0e00 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043011072:192.168.203.147@tcp:101:createmany.0 00000100:00100000:2.0:1787826319.164162:0:161476:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c4aa6480 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.169229:0:161476:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f0e00 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043011072:192.168.203.147@tcp:101:createmany.0 00010000:00010000:2.0:1787826319.169259:0:161476:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f2a5 lrc: 4/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee039 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.169291:0:161476:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f2a5 lrc: 4/0,1 mode: --/PR res: [0x240001f59:0xb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee039 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.169300:0:161476:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f2a5 lrc: 4/0,1 mode: --/PR res: [0x240001f59:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee039 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.169311:0:161476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f2a5 lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee039 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.169322:0:161476:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f2a5 lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee039 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.169340:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f2a5 lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee039 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.169349:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f2a5 lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee039 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.169355:0:161476:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f2a5 lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee039 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787826319.169367:0:161476:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9181c71f0e00 x1874663043011072/t17179870006(17179870006) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:2.0:1787826319.169379:0:161476:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c71f0e00 x1874663043011072/t17179870006(17179870006) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787826319.169392:0:161476:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f2a5 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee039 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:2.0:1787826319.169404:0:161476:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f2a5 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee039 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.169415:0:161476:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c77f3000 type 13 mode 4 res 9663684441/11 (0 0) 00000080:00200000:2.0:1787826319.169420:0:161476:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x240001f59:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1787826319.169433:0:161476:0:(namei.c:1628:ll_create_it()) VFS Op:name=f-9, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), intent=open|creat 00000080:00200000:2.0:1787826319.169460:0:161476:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001f59:0xb:0x0](ffff9181d0800908) 00000080:00200000:2.0:1787826319.169534:0:161476:0:(file.c:6843:ll_layout_conf()) [0x240001f59:0xb:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:2.0:1787826319.169539:0:161476:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x240001f59:0xb:0x0](ffff9181d0800908) 00000080:00010000:2.0:1787826319.169544:0:161476:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001f59:0xb:0x0](ffff9181d0800908) for lock 0xbef42c0221d6f2a5 00000080:00010000:2.0:1787826319.169553:0:161476:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001f59:0xb:0x0](ffff9181d0800908) 00000080:00200008:2.0:1787826319.169567:0:161476:0:(file.c:1090:ll_file_open()) START file f-9:[0x240001f59:0xb:0x0](ffff9181d0800908), flags 100102 00000002:00100000:2.0:1787826319.169601:0:161476:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c71f0e00 x1874663043011072/t17179870006(17179870006) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:2.0:1787826319.169637:0:161476:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f2a5 from it ffff9181c2b50420 00010000:00010000:2.0:1787826319.169642:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f2a5 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00cee039 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.169651:0:161476:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f2a5 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00cee039 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826319.169672:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0xb:0x0](ffff9181d0800908), inode mode 81a4 mask 40 00000080:00200008:2.0:1787826319.169713:0:161476:0:(file.c:479:ll_file_release()) START file f-9:[0x240001f59:0xb:0x0](ffff9181d0800908), flags 100002 00000002:00100000:2.0:1787826319.169802:0:161476:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181c71f0e00 x1874663043011072/t17179870006(17179870006) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:2.0:1787826319.169815:0:161476:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1787826319.169820:0:161476:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c71f3480 x1874663043011328/t0(0) o35->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:2.0:1787826319.169841:0:161476:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f3480 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043011328:192.168.203.147@tcp:35:createmany.0 00000100:00100000:2.0:1787826319.169917:0:161476:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c4aa6d80 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.173873:0:161476:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f3480 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043011328:192.168.203.147@tcp:35:createmany.0 00000002:00100000:2.0:1787826319.173915:0:161476:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181c71f0e00 x1874663043011072/t17179870006(17179870006) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1787826319.174002:0:161476:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:2.0:1787826319.174090:0:161476:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826319.174109:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 18 lru_type: 1 00010000:00010000:2.0:1787826319.174124:0:161476:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 18 lru_type: 1 00000080:00200000:2.0:1787826319.174132:0:161476:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826319.174138:0:161476:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:2.0:1787826319.174141:0:161476:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f24a 00000080:00010000:2.0:1787826319.174148:0:161476:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f24a from it ffffb97501c3bbf8 00010000:00010000:2.0:1787826319.174152:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 18 lru_type: 1 00010000:00010000:2.0:1787826319.174160:0:161476:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 19 lru_type: 1 00000080:00200000:2.0:1787826319.174174:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.174181:0:161476:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80a.replay-single, flags=25 00000080:00200000:2.0:1787826319.174187:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x2:0x0](ffff9181c83f3248), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.174190:0:161476:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=remote_dir, flags=25 00000080:00200000:2.0:1787826319.174194:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.174207:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 3 00000080:00200000:2.0:1787826319.174212:0:161476:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f-10, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), file ffff9181c5457500, open_flags 8042, mode 81a4 opened 0 00000080:00200000:2.0:1787826319.174219:0:161476:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f-10, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), intent=open|creat 00000002:00010000:2.0:1787826319.174236:0:161476:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f-10,[0x240001f59:0xc:0x0]) in obj [0x240001f59:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:2.0:1787826319.174287:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6e00/0xbef42c0221d6f2ac lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.174298:0:161476:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6e00/0xbef42c0221d6f2ac lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1787826319.174308:0:161476:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1787826319.174315:0:161476:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c71f2a00 x1874663043011584/t0(0) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1787826319.174322:0:161476:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6e00/0xbef42c0221d6f2ac lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826319.174340:0:161476:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f2a00 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043011584:192.168.203.147@tcp:101:createmany.0 00000100:00100000:2.0:1787826319.174513:0:161476:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c4aa6d80 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.179563:0:161476:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f2a00 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043011584:192.168.203.147@tcp:101:createmany.0 00010000:00010000:2.0:1787826319.179601:0:161476:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6e00/0xbef42c0221d6f2ac lrc: 4/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee05c expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.179631:0:161476:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6e00/0xbef42c0221d6f2ac lrc: 4/0,1 mode: --/PR res: [0x240001f59:0xc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee05c expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.179640:0:161476:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6e00/0xbef42c0221d6f2ac lrc: 4/0,1 mode: --/PR res: [0x240001f59:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee05c expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.179653:0:161476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6e00/0xbef42c0221d6f2ac lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee05c expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.179665:0:161476:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6e00/0xbef42c0221d6f2ac lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee05c expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.179683:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6e00/0xbef42c0221d6f2ac lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee05c expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.179692:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6e00/0xbef42c0221d6f2ac lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee05c expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.179712:0:161476:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6e00/0xbef42c0221d6f2ac lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee05c expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787826319.179725:0:161476:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9181c71f2a00 x1874663043011584/t17179870008(17179870008) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:2.0:1787826319.179739:0:161476:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c71f2a00 x1874663043011584/t17179870008(17179870008) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787826319.179789:0:161476:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6e00/0xbef42c0221d6f2ac lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee05c expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:2.0:1787826319.179803:0:161476:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6e00/0xbef42c0221d6f2ac lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee05c expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.179814:0:161476:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c77f3000 type 13 mode 4 res 9663684441/12 (0 0) 00000080:00200000:2.0:1787826319.179820:0:161476:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x240001f59:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1787826319.179834:0:161476:0:(namei.c:1628:ll_create_it()) VFS Op:name=f-10, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), intent=open|creat 00000080:00200000:2.0:1787826319.179860:0:161476:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001f59:0xc:0x0](ffff9181d0801148) 00000080:00200000:2.0:1787826319.179935:0:161476:0:(file.c:6843:ll_layout_conf()) [0x240001f59:0xc:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:2.0:1787826319.179941:0:161476:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x240001f59:0xc:0x0](ffff9181d0801148) 00000080:00010000:2.0:1787826319.179945:0:161476:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001f59:0xc:0x0](ffff9181d0801148) for lock 0xbef42c0221d6f2ac 00000080:00010000:2.0:1787826319.179953:0:161476:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001f59:0xc:0x0](ffff9181d0801148) 00000080:00200008:2.0:1787826319.179978:0:161476:0:(file.c:1090:ll_file_open()) START file f-10:[0x240001f59:0xc:0x0](ffff9181d0801148), flags 100102 00000002:00100000:2.0:1787826319.179999:0:161476:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c71f2a00 x1874663043011584/t17179870008(17179870008) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:2.0:1787826319.180045:0:161476:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f2ac from it ffff9181c2b50420 00010000:00010000:2.0:1787826319.180051:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6e00/0xbef42c0221d6f2ac lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00cee05c expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.180060:0:161476:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6e00/0xbef42c0221d6f2ac lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00cee05c expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826319.180080:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0xc:0x0](ffff9181d0801148), inode mode 81a4 mask 40 00000080:00200008:2.0:1787826319.180109:0:161476:0:(file.c:479:ll_file_release()) START file f-10:[0x240001f59:0xc:0x0](ffff9181d0801148), flags 100002 00000002:00100000:2.0:1787826319.180151:0:161476:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181c71f2a00 x1874663043011584/t17179870008(17179870008) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:2.0:1787826319.180162:0:161476:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1787826319.180167:0:161476:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c71f2300 x1874663043011840/t0(0) o35->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:2.0:1787826319.180187:0:161476:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f2300 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043011840:192.168.203.147@tcp:35:createmany.0 00000100:00100000:2.0:1787826319.180262:0:161476:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c4aa6840 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.183929:0:161476:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f2300 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043011840:192.168.203.147@tcp:35:createmany.0 00000002:00100000:2.0:1787826319.183979:0:161476:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181c71f2a00 x1874663043011584/t17179870008(17179870008) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1787826319.184067:0:161476:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:2.0:1787826319.184087:0:161476:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826319.184107:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 19 lru_type: 1 00010000:00010000:2.0:1787826319.184122:0:161476:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 19 lru_type: 1 00000080:00200000:2.0:1787826319.184130:0:161476:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826319.184136:0:161476:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:2.0:1787826319.184139:0:161476:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f24a 00000080:00010000:2.0:1787826319.184146:0:161476:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f24a from it ffffb97501c3bbf8 00010000:00010000:2.0:1787826319.184149:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 19 lru_type: 1 00010000:00010000:2.0:1787826319.184157:0:161476:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 20 lru_type: 1 00000080:00200000:2.0:1787826319.184170:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.184177:0:161476:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80a.replay-single, flags=25 00000080:00200000:2.0:1787826319.184182:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x2:0x0](ffff9181c83f3248), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.184186:0:161476:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=remote_dir, flags=25 00000080:00200000:2.0:1787826319.184189:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.184201:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 3 00000080:00200000:2.0:1787826319.184205:0:161476:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f-11, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), file ffff9181c5457980, open_flags 8042, mode 81a4 opened 0 00000080:00200000:2.0:1787826319.184213:0:161476:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f-11, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), intent=open|creat 00000002:00010000:2.0:1787826319.184231:0:161476:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f-11,[0x240001f59:0xd:0x0]) in obj [0x240001f59:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:2.0:1787826319.184280:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7a00/0xbef42c0221d6f2b3 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.184292:0:161476:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7a00/0xbef42c0221d6f2b3 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1787826319.184303:0:161476:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1787826319.184310:0:161476:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c71f1500 x1874663043012096/t0(0) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1787826319.184319:0:161476:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7a00/0xbef42c0221d6f2b3 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826319.184341:0:161476:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f1500 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043012096:192.168.203.147@tcp:101:createmany.0 00000100:00100000:2.0:1787826319.184475:0:161476:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c4aa6840 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.190216:0:161476:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f1500 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043012096:192.168.203.147@tcp:101:createmany.0 00010000:00010000:2.0:1787826319.190243:0:161476:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7a00/0xbef42c0221d6f2b3 lrc: 4/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee07f expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.190270:0:161476:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7a00/0xbef42c0221d6f2b3 lrc: 4/0,1 mode: --/PR res: [0x240001f59:0xd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee07f expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.190279:0:161476:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7a00/0xbef42c0221d6f2b3 lrc: 4/0,1 mode: --/PR res: [0x240001f59:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee07f expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.190289:0:161476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7a00/0xbef42c0221d6f2b3 lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee07f expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.190300:0:161476:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7a00/0xbef42c0221d6f2b3 lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee07f expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.190319:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7a00/0xbef42c0221d6f2b3 lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee07f expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.190327:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7a00/0xbef42c0221d6f2b3 lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee07f expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.190334:0:161476:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7a00/0xbef42c0221d6f2b3 lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee07f expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787826319.190346:0:161476:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9181c71f1500 x1874663043012096/t17179870010(17179870010) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:2.0:1787826319.190358:0:161476:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c71f1500 x1874663043012096/t17179870010(17179870010) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787826319.190372:0:161476:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7a00/0xbef42c0221d6f2b3 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee07f expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:2.0:1787826319.190385:0:161476:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7a00/0xbef42c0221d6f2b3 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee07f expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.190395:0:161476:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c77f3000 type 13 mode 4 res 9663684441/13 (0 0) 00000080:00200000:2.0:1787826319.190401:0:161476:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x240001f59:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1787826319.190416:0:161476:0:(namei.c:1628:ll_create_it()) VFS Op:name=f-11, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), intent=open|creat 00000080:00200000:2.0:1787826319.190447:0:161476:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001f59:0xd:0x0](ffff9181d08000c8) 00000080:00200000:2.0:1787826319.190520:0:161476:0:(file.c:6843:ll_layout_conf()) [0x240001f59:0xd:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:2.0:1787826319.190526:0:161476:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x240001f59:0xd:0x0](ffff9181d08000c8) 00000080:00010000:2.0:1787826319.190529:0:161476:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001f59:0xd:0x0](ffff9181d08000c8) for lock 0xbef42c0221d6f2b3 00000080:00010000:2.0:1787826319.190538:0:161476:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001f59:0xd:0x0](ffff9181d08000c8) 00000080:00200008:2.0:1787826319.190552:0:161476:0:(file.c:1090:ll_file_open()) START file f-11:[0x240001f59:0xd:0x0](ffff9181d08000c8), flags 100102 00000002:00100000:2.0:1787826319.190572:0:161476:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c71f1500 x1874663043012096/t17179870010(17179870010) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:2.0:1787826319.190606:0:161476:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f2b3 from it ffff9181c553e6c0 00010000:00010000:2.0:1787826319.190612:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7a00/0xbef42c0221d6f2b3 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00cee07f expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.190621:0:161476:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7a00/0xbef42c0221d6f2b3 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00cee07f expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826319.190641:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0xd:0x0](ffff9181d08000c8), inode mode 81a4 mask 40 00000080:00200008:2.0:1787826319.190670:0:161476:0:(file.c:479:ll_file_release()) START file f-11:[0x240001f59:0xd:0x0](ffff9181d08000c8), flags 100002 00000002:00100000:2.0:1787826319.190885:0:161476:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181c71f1500 x1874663043012096/t17179870010(17179870010) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:2.0:1787826319.190896:0:161476:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1787826319.190901:0:161476:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c71f0a80 x1874663043012352/t0(0) o35->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:2.0:1787826319.190922:0:161476:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f0a80 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043012352:192.168.203.147@tcp:35:createmany.0 00000100:00100000:2.0:1787826319.191054:0:161476:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c513bd80 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.194145:0:161476:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f0a80 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043012352:192.168.203.147@tcp:35:createmany.0 00000002:00100000:2.0:1787826319.194187:0:161476:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181c71f1500 x1874663043012096/t17179870010(17179870010) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1787826319.194260:0:161476:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:2.0:1787826319.194278:0:161476:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826319.194295:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 20 lru_type: 1 00010000:00010000:2.0:1787826319.194310:0:161476:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 20 lru_type: 1 00000080:00200000:2.0:1787826319.194317:0:161476:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826319.194323:0:161476:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:2.0:1787826319.194327:0:161476:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f24a 00000080:00010000:2.0:1787826319.194334:0:161476:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f24a from it ffffb97501c3bbf8 00010000:00010000:2.0:1787826319.194338:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 20 lru_type: 1 00010000:00010000:2.0:1787826319.194347:0:161476:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 21 lru_type: 1 00000080:00200000:2.0:1787826319.194362:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.194369:0:161476:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80a.replay-single, flags=25 00000080:00200000:2.0:1787826319.194375:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x2:0x0](ffff9181c83f3248), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.194379:0:161476:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=remote_dir, flags=25 00000080:00200000:2.0:1787826319.194382:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.194395:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 3 00000080:00200000:2.0:1787826319.194400:0:161476:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f-12, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), file ffff9181c5457500, open_flags 8042, mode 81a4 opened 0 00000080:00200000:2.0:1787826319.194407:0:161476:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f-12, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), intent=open|creat 00000002:00010000:2.0:1787826319.194427:0:161476:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f-12,[0x240001f59:0xe:0x0]) in obj [0x240001f59:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:2.0:1787826319.194478:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6a00/0xbef42c0221d6f2ba lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.194490:0:161476:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6a00/0xbef42c0221d6f2ba lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1787826319.194501:0:161476:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1787826319.194507:0:161476:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c71f1880 x1874663043012608/t0(0) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1787826319.194516:0:161476:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6a00/0xbef42c0221d6f2ba lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826319.194535:0:161476:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f1880 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043012608:192.168.203.147@tcp:101:createmany.0 00000100:00100000:2.0:1787826319.194692:0:161476:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c513bd80 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.200499:0:161476:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f1880 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043012608:192.168.203.147@tcp:101:createmany.0 00010000:00010000:2.0:1787826319.200528:0:161476:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6a00/0xbef42c0221d6f2ba lrc: 4/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee0a2 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.200558:0:161476:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6a00/0xbef42c0221d6f2ba lrc: 4/0,1 mode: --/PR res: [0x240001f59:0xe:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee0a2 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.200567:0:161476:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6a00/0xbef42c0221d6f2ba lrc: 4/0,1 mode: --/PR res: [0x240001f59:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee0a2 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.200578:0:161476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6a00/0xbef42c0221d6f2ba lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee0a2 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.200599:0:161476:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6a00/0xbef42c0221d6f2ba lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee0a2 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.200617:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6a00/0xbef42c0221d6f2ba lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee0a2 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.200625:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6a00/0xbef42c0221d6f2ba lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee0a2 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.200633:0:161476:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6a00/0xbef42c0221d6f2ba lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee0a2 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787826319.200645:0:161476:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9181c71f1880 x1874663043012608/t17179870012(17179870012) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:2.0:1787826319.200658:0:161476:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c71f1880 x1874663043012608/t17179870012(17179870012) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787826319.200671:0:161476:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6a00/0xbef42c0221d6f2ba lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee0a2 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:2.0:1787826319.200684:0:161476:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6a00/0xbef42c0221d6f2ba lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee0a2 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.200694:0:161476:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c77f3000 type 13 mode 4 res 9663684441/14 (0 0) 00000080:00200000:2.0:1787826319.200834:0:161476:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x240001f59:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1787826319.200848:0:161476:0:(namei.c:1628:ll_create_it()) VFS Op:name=f-12, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), intent=open|creat 00000080:00200000:2.0:1787826319.200875:0:161476:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001f59:0xe:0x0](ffff9181d0806c08) 00000080:00200000:2.0:1787826319.201104:0:161476:0:(file.c:6843:ll_layout_conf()) [0x240001f59:0xe:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:2.0:1787826319.201110:0:161476:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x240001f59:0xe:0x0](ffff9181d0806c08) 00000080:00010000:2.0:1787826319.201114:0:161476:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001f59:0xe:0x0](ffff9181d0806c08) for lock 0xbef42c0221d6f2ba 00000080:00010000:2.0:1787826319.201124:0:161476:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001f59:0xe:0x0](ffff9181d0806c08) 00000080:00200008:2.0:1787826319.201140:0:161476:0:(file.c:1090:ll_file_open()) START file f-12:[0x240001f59:0xe:0x0](ffff9181d0806c08), flags 100102 00000002:00100000:2.0:1787826319.201163:0:161476:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c71f1880 x1874663043012608/t17179870012(17179870012) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:2.0:1787826319.201198:0:161476:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f2ba from it ffff9181e003aba0 00010000:00010000:2.0:1787826319.201203:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6a00/0xbef42c0221d6f2ba lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00cee0a2 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.201213:0:161476:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6a00/0xbef42c0221d6f2ba lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00cee0a2 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826319.201234:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0xe:0x0](ffff9181d0806c08), inode mode 81a4 mask 40 00000080:00200008:2.0:1787826319.201262:0:161476:0:(file.c:479:ll_file_release()) START file f-12:[0x240001f59:0xe:0x0](ffff9181d0806c08), flags 100002 00000002:00100000:2.0:1787826319.201304:0:161476:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181c71f1880 x1874663043012608/t17179870012(17179870012) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:2.0:1787826319.201315:0:161476:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1787826319.201320:0:161476:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c71f3100 x1874663043012864/t0(0) o35->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:2.0:1787826319.201340:0:161476:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f3100 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043012864:192.168.203.147@tcp:35:createmany.0 00000100:00100000:2.0:1787826319.201465:0:161476:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59033c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.207856:0:161476:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f3100 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043012864:192.168.203.147@tcp:35:createmany.0 00000002:00100000:2.0:1787826319.207896:0:161476:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181c71f1880 x1874663043012608/t17179870012(17179870012) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1787826319.207985:0:161476:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:2.0:1787826319.208016:0:161476:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826319.208035:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 21 lru_type: 1 00010000:00010000:2.0:1787826319.208052:0:161476:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 21 lru_type: 1 00000080:00200000:2.0:1787826319.208060:0:161476:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826319.208066:0:161476:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:2.0:1787826319.208069:0:161476:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f24a 00000080:00010000:2.0:1787826319.208077:0:161476:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f24a from it ffffb97501c3bbf8 00010000:00010000:2.0:1787826319.208081:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 21 lru_type: 1 00010000:00010000:2.0:1787826319.208091:0:161476:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 22 lru_type: 1 00000080:00200000:2.0:1787826319.208106:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.208113:0:161476:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80a.replay-single, flags=25 00000080:00200000:2.0:1787826319.208119:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x2:0x0](ffff9181c83f3248), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.208123:0:161476:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=remote_dir, flags=25 00000080:00200000:2.0:1787826319.208127:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.208140:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 3 00000080:00200000:2.0:1787826319.208144:0:161476:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f-13, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), file ffff9181c5457980, open_flags 8042, mode 81a4 opened 0 00000080:00200000:2.0:1787826319.208152:0:161476:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f-13, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), intent=open|creat 00000002:00010000:2.0:1787826319.208170:0:161476:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f-13,[0x240001f59:0xf:0x0]) in obj [0x240001f59:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:2.0:1787826319.208217:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7c00/0xbef42c0221d6f2c1 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.208229:0:161476:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7c00/0xbef42c0221d6f2c1 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1787826319.208240:0:161476:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1787826319.208247:0:161476:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c71f2d80 x1874663043013120/t0(0) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1787826319.208256:0:161476:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7c00/0xbef42c0221d6f2c1 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826319.208273:0:161476:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c71f2d80 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043013120:192.168.203.147@tcp:101:createmany.0 00000100:00100000:2.0:1787826319.208401:0:161476:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59033c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.212551:0:161476:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c71f2d80 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043013120:192.168.203.147@tcp:101:createmany.0 00010000:00010000:2.0:1787826319.212597:0:161476:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7c00/0xbef42c0221d6f2c1 lrc: 4/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee0c5 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.212628:0:161476:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7c00/0xbef42c0221d6f2c1 lrc: 4/0,1 mode: --/PR res: [0x240001f59:0xf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee0c5 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.212635:0:161476:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7c00/0xbef42c0221d6f2c1 lrc: 4/0,1 mode: --/PR res: [0x240001f59:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee0c5 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.212646:0:161476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7c00/0xbef42c0221d6f2c1 lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee0c5 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.212655:0:161476:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7c00/0xbef42c0221d6f2c1 lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee0c5 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.212671:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7c00/0xbef42c0221d6f2c1 lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee0c5 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.212679:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7c00/0xbef42c0221d6f2c1 lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee0c5 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.212684:0:161476:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7c00/0xbef42c0221d6f2c1 lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee0c5 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787826319.212693:0:161476:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9181c71f2d80 x1874663043013120/t17179870014(17179870014) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:2.0:1787826319.212715:0:161476:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c71f2d80 x1874663043013120/t17179870014(17179870014) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787826319.212727:0:161476:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7c00/0xbef42c0221d6f2c1 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee0c5 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:2.0:1787826319.212739:0:161476:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7c00/0xbef42c0221d6f2c1 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee0c5 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.212783:0:161476:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c77f3000 type 13 mode 4 res 9663684441/15 (0 0) 00000080:00200000:2.0:1787826319.212789:0:161476:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x240001f59:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1787826319.212803:0:161476:0:(namei.c:1628:ll_create_it()) VFS Op:name=f-13, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), intent=open|creat 00000080:00200000:2.0:1787826319.212830:0:161476:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001f59:0xf:0x0](ffff9181d0805348) 00000080:00200000:2.0:1787826319.212901:0:161476:0:(file.c:6843:ll_layout_conf()) [0x240001f59:0xf:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:2.0:1787826319.212906:0:161476:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x240001f59:0xf:0x0](ffff9181d0805348) 00000080:00010000:2.0:1787826319.212909:0:161476:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001f59:0xf:0x0](ffff9181d0805348) for lock 0xbef42c0221d6f2c1 00000080:00010000:2.0:1787826319.212919:0:161476:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001f59:0xf:0x0](ffff9181d0805348) 00000080:00200008:2.0:1787826319.212933:0:161476:0:(file.c:1090:ll_file_open()) START file f-13:[0x240001f59:0xf:0x0](ffff9181d0805348), flags 100102 00000002:00100000:2.0:1787826319.212954:0:161476:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c71f2d80 x1874663043013120/t17179870014(17179870014) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:2.0:1787826319.213002:0:161476:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f2c1 from it ffff9181e003aba0 00010000:00010000:2.0:1787826319.213007:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7c00/0xbef42c0221d6f2c1 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00cee0c5 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.213025:0:161476:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7c00/0xbef42c0221d6f2c1 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00cee0c5 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826319.213047:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0xf:0x0](ffff9181d0805348), inode mode 81a4 mask 40 00000080:00200008:2.0:1787826319.213078:0:161476:0:(file.c:479:ll_file_release()) START file f-13:[0x240001f59:0xf:0x0](ffff9181d0805348), flags 100002 00000002:00100000:2.0:1787826319.213125:0:161476:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181c71f2d80 x1874663043013120/t17179870014(17179870014) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:2.0:1787826319.213134:0:161476:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1787826319.213139:0:161476:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c49e7480 x1874663043013376/t0(0) o35->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:2.0:1787826319.213158:0:161476:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c49e7480 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043013376:192.168.203.147@tcp:35:createmany.0 00000100:00100000:2.0:1787826319.213219:0:161476:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5903300 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.216508:0:161476:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c49e7480 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043013376:192.168.203.147@tcp:35:createmany.0 00000002:00100000:2.0:1787826319.216542:0:161476:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181c71f2d80 x1874663043013120/t17179870014(17179870014) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1787826319.216620:0:161476:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:2.0:1787826319.216637:0:161476:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826319.216653:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 22 lru_type: 1 00010000:00010000:2.0:1787826319.216665:0:161476:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 22 lru_type: 1 00000080:00200000:2.0:1787826319.216671:0:161476:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826319.216677:0:161476:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:2.0:1787826319.216680:0:161476:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f24a 00000080:00010000:2.0:1787826319.216686:0:161476:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f24a from it ffffb97501c3bbf8 00010000:00010000:2.0:1787826319.216689:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 22 lru_type: 1 00010000:00010000:2.0:1787826319.216707:0:161476:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 23 lru_type: 1 00000080:00200000:2.0:1787826319.216719:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.216725:0:161476:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80a.replay-single, flags=25 00000080:00200000:2.0:1787826319.216730:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x2:0x0](ffff9181c83f3248), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.216733:0:161476:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=remote_dir, flags=25 00000080:00200000:2.0:1787826319.216737:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.216783:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 3 00000080:00200000:2.0:1787826319.216788:0:161476:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f-14, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), file ffff9181c5457500, open_flags 8042, mode 81a4 opened 0 00000080:00200000:2.0:1787826319.216795:0:161476:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f-14, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), intent=open|creat 00000002:00010000:2.0:1787826319.216815:0:161476:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f-14,[0x240001f59:0x10:0x0]) in obj [0x240001f59:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:2.0:1787826319.216860:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6000/0xbef42c0221d6f2c8 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.216871:0:161476:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6000/0xbef42c0221d6f2c8 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1787826319.216882:0:161476:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1787826319.216888:0:161476:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c49e7100 x1874663043013632/t0(0) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1787826319.216897:0:161476:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6000/0xbef42c0221d6f2c8 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826319.216914:0:161476:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c49e7100 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043013632:192.168.203.147@tcp:101:createmany.0 00000100:00100000:2.0:1787826319.217188:0:161476:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5903300 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.222133:0:161476:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c49e7100 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043013632:192.168.203.147@tcp:101:createmany.0 00010000:00010000:2.0:1787826319.222162:0:161476:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6000/0xbef42c0221d6f2c8 lrc: 4/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee0e8 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.222191:0:161476:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6000/0xbef42c0221d6f2c8 lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x10:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee0e8 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.222200:0:161476:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6000/0xbef42c0221d6f2c8 lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee0e8 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.222211:0:161476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6000/0xbef42c0221d6f2c8 lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee0e8 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.222221:0:161476:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6000/0xbef42c0221d6f2c8 lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee0e8 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.222240:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6000/0xbef42c0221d6f2c8 lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee0e8 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.222248:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6000/0xbef42c0221d6f2c8 lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee0e8 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.222254:0:161476:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6000/0xbef42c0221d6f2c8 lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee0e8 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787826319.222265:0:161476:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9181c49e7100 x1874663043013632/t17179870016(17179870016) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:2.0:1787826319.222277:0:161476:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c49e7100 x1874663043013632/t17179870016(17179870016) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787826319.222290:0:161476:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6000/0xbef42c0221d6f2c8 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee0e8 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:2.0:1787826319.222303:0:161476:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6000/0xbef42c0221d6f2c8 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee0e8 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.222314:0:161476:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c77f3000 type 13 mode 4 res 9663684441/16 (0 0) 00000080:00200000:2.0:1787826319.222320:0:161476:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x240001f59:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1787826319.222333:0:161476:0:(namei.c:1628:ll_create_it()) VFS Op:name=f-14, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), intent=open|creat 00000080:00200000:2.0:1787826319.222362:0:161476:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001f59:0x10:0x0](ffff9181d0807448) 00000080:00200000:2.0:1787826319.222440:0:161476:0:(file.c:6843:ll_layout_conf()) [0x240001f59:0x10:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:2.0:1787826319.222446:0:161476:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x240001f59:0x10:0x0](ffff9181d0807448) 00000080:00010000:2.0:1787826319.222449:0:161476:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001f59:0x10:0x0](ffff9181d0807448) for lock 0xbef42c0221d6f2c8 00000080:00010000:2.0:1787826319.222458:0:161476:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001f59:0x10:0x0](ffff9181d0807448) 00000080:00200008:2.0:1787826319.222473:0:161476:0:(file.c:1090:ll_file_open()) START file f-14:[0x240001f59:0x10:0x0](ffff9181d0807448), flags 100102 00000002:00100000:2.0:1787826319.222493:0:161476:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c49e7100 x1874663043013632/t17179870016(17179870016) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:2.0:1787826319.222527:0:161476:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f2c8 from it ffff9181e003aba0 00010000:00010000:2.0:1787826319.222532:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6000/0xbef42c0221d6f2c8 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00cee0e8 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.222541:0:161476:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6000/0xbef42c0221d6f2c8 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00cee0e8 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826319.222562:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x10:0x0](ffff9181d0807448), inode mode 81a4 mask 40 00000080:00200008:2.0:1787826319.222589:0:161476:0:(file.c:479:ll_file_release()) START file f-14:[0x240001f59:0x10:0x0](ffff9181d0807448), flags 100002 00000002:00100000:2.0:1787826319.222633:0:161476:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181c49e7100 x1874663043013632/t17179870016(17179870016) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:2.0:1787826319.222643:0:161476:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1787826319.222648:0:161476:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c49e4e00 x1874663043013888/t0(0) o35->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:2.0:1787826319.222669:0:161476:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c49e4e00 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043013888:192.168.203.147@tcp:35:createmany.0 00000100:00100000:2.0:1787826319.222791:0:161476:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5903180 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.226607:0:161476:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c49e4e00 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043013888:192.168.203.147@tcp:35:createmany.0 00000002:00100000:2.0:1787826319.226646:0:161476:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181c49e7100 x1874663043013632/t17179870016(17179870016) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1787826319.226727:0:161476:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:2.0:1787826319.226777:0:161476:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826319.226795:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 23 lru_type: 1 00010000:00010000:2.0:1787826319.226809:0:161476:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 23 lru_type: 1 00000080:00200000:2.0:1787826319.226817:0:161476:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826319.226822:0:161476:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:2.0:1787826319.226826:0:161476:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f24a 00000080:00010000:2.0:1787826319.226832:0:161476:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f24a from it ffffb97501c3bbf8 00010000:00010000:2.0:1787826319.226836:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 23 lru_type: 1 00010000:00010000:2.0:1787826319.226844:0:161476:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 24 lru_type: 1 00000080:00200000:2.0:1787826319.226859:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.226866:0:161476:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80a.replay-single, flags=25 00000080:00200000:2.0:1787826319.226871:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x2:0x0](ffff9181c83f3248), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.226875:0:161476:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=remote_dir, flags=25 00000080:00200000:2.0:1787826319.226878:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.226890:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 3 00000080:00200000:2.0:1787826319.226895:0:161476:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f-15, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), file ffff9181c5457980, open_flags 8042, mode 81a4 opened 0 00000080:00200000:2.0:1787826319.226902:0:161476:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f-15, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), intent=open|creat 00000002:00010000:2.0:1787826319.226921:0:161476:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f-15,[0x240001f59:0x11:0x0]) in obj [0x240001f59:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:2.0:1787826319.226980:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7e00/0xbef42c0221d6f2cf lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.226994:0:161476:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7e00/0xbef42c0221d6f2cf lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1787826319.227005:0:161476:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1787826319.227022:0:161476:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c49e4a80 x1874663043014144/t0(0) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1787826319.227031:0:161476:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7e00/0xbef42c0221d6f2cf lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826319.227051:0:161476:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c49e4a80 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043014144:192.168.203.147@tcp:101:createmany.0 00000100:00100000:2.0:1787826319.227168:0:161476:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5903180 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.231566:0:161476:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c49e4a80 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043014144:192.168.203.147@tcp:101:createmany.0 00010000:00010000:2.0:1787826319.231608:0:161476:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7e00/0xbef42c0221d6f2cf lrc: 4/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee10b expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.231643:0:161476:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7e00/0xbef42c0221d6f2cf lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x11:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee10b expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.231651:0:161476:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7e00/0xbef42c0221d6f2cf lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee10b expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.231662:0:161476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7e00/0xbef42c0221d6f2cf lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee10b expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.231673:0:161476:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7e00/0xbef42c0221d6f2cf lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee10b expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.231692:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7e00/0xbef42c0221d6f2cf lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee10b expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.231700:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7e00/0xbef42c0221d6f2cf lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee10b expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.231707:0:161476:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7e00/0xbef42c0221d6f2cf lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee10b expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787826319.231719:0:161476:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9181c49e4a80 x1874663043014144/t17179870018(17179870018) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:2.0:1787826319.231731:0:161476:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c49e4a80 x1874663043014144/t17179870018(17179870018) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787826319.231776:0:161476:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7e00/0xbef42c0221d6f2cf lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee10b expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:2.0:1787826319.231789:0:161476:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7e00/0xbef42c0221d6f2cf lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee10b expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.231800:0:161476:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c77f3000 type 13 mode 4 res 9663684441/17 (0 0) 00000080:00200000:2.0:1787826319.231806:0:161476:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x240001f59:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1787826319.231818:0:161476:0:(namei.c:1628:ll_create_it()) VFS Op:name=f-15, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), intent=open|creat 00000080:00200000:2.0:1787826319.231845:0:161476:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001f59:0x11:0x0](ffff9181d08021c8) 00000080:00200000:2.0:1787826319.231917:0:161476:0:(file.c:6843:ll_layout_conf()) [0x240001f59:0x11:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:2.0:1787826319.231922:0:161476:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x240001f59:0x11:0x0](ffff9181d08021c8) 00000080:00010000:2.0:1787826319.231926:0:161476:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001f59:0x11:0x0](ffff9181d08021c8) for lock 0xbef42c0221d6f2cf 00000080:00010000:2.0:1787826319.231935:0:161476:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001f59:0x11:0x0](ffff9181d08021c8) 00000080:00200008:2.0:1787826319.231948:0:161476:0:(file.c:1090:ll_file_open()) START file f-15:[0x240001f59:0x11:0x0](ffff9181d08021c8), flags 100102 00000002:00100000:2.0:1787826319.231980:0:161476:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c49e4a80 x1874663043014144/t17179870018(17179870018) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:2.0:1787826319.232024:0:161476:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f2cf from it ffff9181e003aba0 00010000:00010000:2.0:1787826319.232029:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7e00/0xbef42c0221d6f2cf lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00cee10b expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.232038:0:161476:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7e00/0xbef42c0221d6f2cf lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00cee10b expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826319.232058:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x11:0x0](ffff9181d08021c8), inode mode 81a4 mask 40 00000080:00200008:2.0:1787826319.232086:0:161476:0:(file.c:479:ll_file_release()) START file f-15:[0x240001f59:0x11:0x0](ffff9181d08021c8), flags 100002 00000002:00100000:2.0:1787826319.232126:0:161476:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181c49e4a80 x1874663043014144/t17179870018(17179870018) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:2.0:1787826319.232136:0:161476:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1787826319.232140:0:161476:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c49e5c00 x1874663043014400/t0(0) o35->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:2.0:1787826319.232160:0:161476:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c49e5c00 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043014400:192.168.203.147@tcp:35:createmany.0 00000100:00100000:2.0:1787826319.232223:0:161476:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5903000 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.235082:0:161476:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c49e5c00 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043014400:192.168.203.147@tcp:35:createmany.0 00000002:00100000:2.0:1787826319.235115:0:161476:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181c49e4a80 x1874663043014144/t17179870018(17179870018) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1787826319.235185:0:161476:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:2.0:1787826319.235201:0:161476:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826319.235217:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 24 lru_type: 1 00010000:00010000:2.0:1787826319.235230:0:161476:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 24 lru_type: 1 00000080:00200000:2.0:1787826319.235238:0:161476:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826319.235243:0:161476:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:2.0:1787826319.235246:0:161476:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f24a 00000080:00010000:2.0:1787826319.235252:0:161476:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f24a from it ffffb97501c3bbf8 00010000:00010000:2.0:1787826319.235256:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 24 lru_type: 1 00010000:00010000:2.0:1787826319.235265:0:161476:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 25 lru_type: 1 00000080:00200000:2.0:1787826319.235278:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.235284:0:161476:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80a.replay-single, flags=25 00000080:00200000:2.0:1787826319.235289:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x2:0x0](ffff9181c83f3248), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.235291:0:161476:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=remote_dir, flags=25 00000080:00200000:2.0:1787826319.235294:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.235304:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 3 00000080:00200000:2.0:1787826319.235308:0:161476:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f-16, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), file ffff9181c5457500, open_flags 8042, mode 81a4 opened 0 00000080:00200000:2.0:1787826319.235315:0:161476:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f-16, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), intent=open|creat 00000002:00010000:2.0:1787826319.235332:0:161476:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f-16,[0x240001f59:0x12:0x0]) in obj [0x240001f59:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:2.0:1787826319.235379:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7800/0xbef42c0221d6f2d6 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.235392:0:161476:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7800/0xbef42c0221d6f2d6 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1787826319.235402:0:161476:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1787826319.235407:0:161476:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c49e4380 x1874663043014656/t0(0) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1787826319.235416:0:161476:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7800/0xbef42c0221d6f2d6 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826319.235431:0:161476:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c49e4380 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043014656:192.168.203.147@tcp:101:createmany.0 00000100:00100000:2.0:1787826319.235534:0:161476:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5903000 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.240022:0:161476:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c49e4380 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043014656:192.168.203.147@tcp:101:createmany.0 00010000:00010000:2.0:1787826319.240052:0:161476:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7800/0xbef42c0221d6f2d6 lrc: 4/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee12e expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.240081:0:161476:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7800/0xbef42c0221d6f2d6 lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x12:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee12e expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.240089:0:161476:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7800/0xbef42c0221d6f2d6 lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee12e expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.240100:0:161476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7800/0xbef42c0221d6f2d6 lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee12e expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.240111:0:161476:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7800/0xbef42c0221d6f2d6 lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee12e expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.240130:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7800/0xbef42c0221d6f2d6 lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee12e expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.240137:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7800/0xbef42c0221d6f2d6 lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee12e expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.240144:0:161476:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7800/0xbef42c0221d6f2d6 lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee12e expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787826319.240156:0:161476:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9181c49e4380 x1874663043014656/t17179870020(17179870020) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:2.0:1787826319.240169:0:161476:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c49e4380 x1874663043014656/t17179870020(17179870020) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787826319.240181:0:161476:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7800/0xbef42c0221d6f2d6 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee12e expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:2.0:1787826319.240194:0:161476:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7800/0xbef42c0221d6f2d6 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee12e expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.240204:0:161476:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c77f3000 type 13 mode 4 res 9663684441/18 (0 0) 00000080:00200000:2.0:1787826319.240210:0:161476:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x240001f59:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1787826319.240224:0:161476:0:(namei.c:1628:ll_create_it()) VFS Op:name=f-16, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), intent=open|creat 00000080:00200000:2.0:1787826319.240251:0:161476:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001f59:0x12:0x0](ffff9181d08063c8) 00000080:00200000:2.0:1787826319.240322:0:161476:0:(file.c:6843:ll_layout_conf()) [0x240001f59:0x12:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:2.0:1787826319.240328:0:161476:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x240001f59:0x12:0x0](ffff9181d08063c8) 00000080:00010000:2.0:1787826319.240332:0:161476:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001f59:0x12:0x0](ffff9181d08063c8) for lock 0xbef42c0221d6f2d6 00000080:00010000:2.0:1787826319.240340:0:161476:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001f59:0x12:0x0](ffff9181d08063c8) 00000080:00200008:2.0:1787826319.240353:0:161476:0:(file.c:1090:ll_file_open()) START file f-16:[0x240001f59:0x12:0x0](ffff9181d08063c8), flags 100102 00000002:00100000:2.0:1787826319.240373:0:161476:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c49e4380 x1874663043014656/t17179870020(17179870020) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:2.0:1787826319.240409:0:161476:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f2d6 from it ffff9181e003aba0 00010000:00010000:2.0:1787826319.240413:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7800/0xbef42c0221d6f2d6 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00cee12e expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.240423:0:161476:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7800/0xbef42c0221d6f2d6 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00cee12e expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826319.240445:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x12:0x0](ffff9181d08063c8), inode mode 81a4 mask 40 00000080:00200008:2.0:1787826319.240477:0:161476:0:(file.c:479:ll_file_release()) START file f-16:[0x240001f59:0x12:0x0](ffff9181d08063c8), flags 100002 00000002:00100000:2.0:1787826319.240519:0:161476:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181c49e4380 x1874663043014656/t17179870020(17179870020) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:2.0:1787826319.240529:0:161476:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1787826319.240533:0:161476:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c49e6300 x1874663043014912/t0(0) o35->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:2.0:1787826319.240554:0:161476:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c49e6300 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043014912:192.168.203.147@tcp:35:createmany.0 00000100:00100000:2.0:1787826319.240632:0:161476:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5903540 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.244085:0:161476:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c49e6300 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043014912:192.168.203.147@tcp:35:createmany.0 00000002:00100000:2.0:1787826319.244123:0:161476:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181c49e4380 x1874663043014656/t17179870020(17179870020) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1787826319.244198:0:161476:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:2.0:1787826319.244218:0:161476:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826319.244235:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 25 lru_type: 1 00010000:00010000:2.0:1787826319.244249:0:161476:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 25 lru_type: 1 00000080:00200000:2.0:1787826319.244256:0:161476:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826319.244262:0:161476:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:2.0:1787826319.244265:0:161476:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f24a 00000080:00010000:2.0:1787826319.244273:0:161476:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f24a from it ffffb97501c3bbf8 00010000:00010000:2.0:1787826319.244276:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 25 lru_type: 1 00010000:00010000:2.0:1787826319.244285:0:161476:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 26 lru_type: 1 00000080:00200000:2.0:1787826319.244298:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.244305:0:161476:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80a.replay-single, flags=25 00000080:00200000:2.0:1787826319.244310:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x2:0x0](ffff9181c83f3248), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.244314:0:161476:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=remote_dir, flags=25 00000080:00200000:2.0:1787826319.244318:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.244329:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 3 00000080:00200000:2.0:1787826319.244334:0:161476:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f-17, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), file ffff9181c5456d80, open_flags 8042, mode 81a4 opened 0 00000080:00200000:2.0:1787826319.244342:0:161476:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f-17, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), intent=open|creat 00000002:00010000:2.0:1787826319.244361:0:161476:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f-17,[0x240001f59:0x13:0x0]) in obj [0x240001f59:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:2.0:1787826319.244412:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6200/0xbef42c0221d6f2dd lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.244425:0:161476:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6200/0xbef42c0221d6f2dd lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1787826319.244436:0:161476:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1787826319.244442:0:161476:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c49e4000 x1874663043015168/t0(0) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1787826319.244450:0:161476:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6200/0xbef42c0221d6f2dd lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826319.244469:0:161476:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c49e4000 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043015168:192.168.203.147@tcp:101:createmany.0 00000100:00100000:2.0:1787826319.244602:0:161476:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5903540 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.248504:0:161476:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c49e4000 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043015168:192.168.203.147@tcp:101:createmany.0 00010000:00010000:2.0:1787826319.248534:0:161476:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6200/0xbef42c0221d6f2dd lrc: 4/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee151 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.248566:0:161476:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6200/0xbef42c0221d6f2dd lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x13:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee151 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.248575:0:161476:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6200/0xbef42c0221d6f2dd lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee151 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.248597:0:161476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6200/0xbef42c0221d6f2dd lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee151 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.248608:0:161476:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6200/0xbef42c0221d6f2dd lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee151 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.248627:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6200/0xbef42c0221d6f2dd lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee151 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.248635:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6200/0xbef42c0221d6f2dd lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee151 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.248642:0:161476:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6200/0xbef42c0221d6f2dd lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee151 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787826319.248654:0:161476:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9181c49e4000 x1874663043015168/t17179870022(17179870022) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:2.0:1787826319.248667:0:161476:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c49e4000 x1874663043015168/t17179870022(17179870022) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787826319.248681:0:161476:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6200/0xbef42c0221d6f2dd lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee151 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:2.0:1787826319.248695:0:161476:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6200/0xbef42c0221d6f2dd lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee151 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.248705:0:161476:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c77f3000 type 13 mode 4 res 9663684441/19 (0 0) 00000080:00200000:2.0:1787826319.248711:0:161476:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x240001f59:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1787826319.248727:0:161476:0:(namei.c:1628:ll_create_it()) VFS Op:name=f-17, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), intent=open|creat 00000080:00200000:2.0:1787826319.248802:0:161476:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001f59:0x13:0x0](ffff9181d08042c8) 00000080:00200000:2.0:1787826319.248876:0:161476:0:(file.c:6843:ll_layout_conf()) [0x240001f59:0x13:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:2.0:1787826319.248881:0:161476:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x240001f59:0x13:0x0](ffff9181d08042c8) 00000080:00010000:2.0:1787826319.248886:0:161476:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001f59:0x13:0x0](ffff9181d08042c8) for lock 0xbef42c0221d6f2dd 00000080:00010000:2.0:1787826319.248896:0:161476:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001f59:0x13:0x0](ffff9181d08042c8) 00000080:00200008:2.0:1787826319.248909:0:161476:0:(file.c:1090:ll_file_open()) START file f-17:[0x240001f59:0x13:0x0](ffff9181d08042c8), flags 100102 00000002:00100000:2.0:1787826319.248929:0:161476:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c49e4000 x1874663043015168/t17179870022(17179870022) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:2.0:1787826319.248974:0:161476:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f2dd from it ffff9181e003aba0 00010000:00010000:2.0:1787826319.248979:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6200/0xbef42c0221d6f2dd lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00cee151 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.248988:0:161476:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6200/0xbef42c0221d6f2dd lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00cee151 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826319.249019:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x13:0x0](ffff9181d08042c8), inode mode 81a4 mask 40 00000080:00200008:2.0:1787826319.249048:0:161476:0:(file.c:479:ll_file_release()) START file f-17:[0x240001f59:0x13:0x0](ffff9181d08042c8), flags 100002 00000002:00100000:2.0:1787826319.249090:0:161476:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181c49e4000 x1874663043015168/t17179870022(17179870022) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:2.0:1787826319.249101:0:161476:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1787826319.249106:0:161476:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c49e6680 x1874663043015424/t0(0) o35->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:2.0:1787826319.249127:0:161476:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c49e6680 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043015424:192.168.203.147@tcp:35:createmany.0 00000100:00100000:2.0:1787826319.249192:0:161476:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5903c00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.254212:0:161476:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c49e6680 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043015424:192.168.203.147@tcp:35:createmany.0 00000002:00100000:2.0:1787826319.254252:0:161476:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181c49e4000 x1874663043015168/t17179870022(17179870022) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1787826319.254328:0:161476:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:2.0:1787826319.254346:0:161476:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826319.254364:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 26 lru_type: 1 00010000:00010000:2.0:1787826319.254380:0:161476:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 26 lru_type: 1 00000080:00200000:2.0:1787826319.254388:0:161476:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826319.254394:0:161476:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:2.0:1787826319.254397:0:161476:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f24a 00000080:00010000:2.0:1787826319.254404:0:161476:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f24a from it ffffb97501c3bbf8 00010000:00010000:2.0:1787826319.254409:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 26 lru_type: 1 00010000:00010000:2.0:1787826319.254418:0:161476:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 27 lru_type: 1 00000080:00200000:2.0:1787826319.254432:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.254439:0:161476:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80a.replay-single, flags=25 00000080:00200000:2.0:1787826319.254446:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x2:0x0](ffff9181c83f3248), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.254450:0:161476:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=remote_dir, flags=25 00000080:00200000:2.0:1787826319.254453:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.254465:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 3 00000080:00200000:2.0:1787826319.254469:0:161476:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f-18, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), file ffff9181c5457500, open_flags 8042, mode 81a4 opened 0 00000080:00200000:2.0:1787826319.254478:0:161476:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f-18, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), intent=open|creat 00000002:00010000:2.0:1787826319.254497:0:161476:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f-18,[0x240001f59:0x14:0x0]) in obj [0x240001f59:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:2.0:1787826319.254548:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f2e4 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.254561:0:161476:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f2e4 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1787826319.254572:0:161476:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1787826319.254579:0:161476:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c49e7800 x1874663043015680/t0(0) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1787826319.254602:0:161476:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f2e4 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826319.254621:0:161476:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c49e7800 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043015680:192.168.203.147@tcp:101:createmany.0 00000100:00100000:2.0:1787826319.254795:0:161476:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5903c00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.259169:0:161476:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c49e7800 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043015680:192.168.203.147@tcp:101:createmany.0 00010000:00010000:2.0:1787826319.259197:0:161476:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f2e4 lrc: 4/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee174 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.259226:0:161476:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f2e4 lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x14:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee174 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.259234:0:161476:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f2e4 lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x14:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee174 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.259245:0:161476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f2e4 lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x14:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee174 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.259255:0:161476:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f2e4 lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x14:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee174 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.259275:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f2e4 lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x14:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee174 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.259283:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f2e4 lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x14:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee174 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.259290:0:161476:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f2e4 lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x14:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee174 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787826319.259302:0:161476:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9181c49e7800 x1874663043015680/t17179870024(17179870024) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:2.0:1787826319.259316:0:161476:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c49e7800 x1874663043015680/t17179870024(17179870024) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787826319.259328:0:161476:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f2e4 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x14:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee174 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:2.0:1787826319.259341:0:161476:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f2e4 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x14:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee174 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.259351:0:161476:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c77f3000 type 13 mode 4 res 9663684441/20 (0 0) 00000080:00200000:2.0:1787826319.259357:0:161476:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x240001f59:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1787826319.259371:0:161476:0:(namei.c:1628:ll_create_it()) VFS Op:name=f-18, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), intent=open|creat 00000080:00200000:2.0:1787826319.259402:0:161476:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001f59:0x14:0x0](ffff9181c830aa08) 00000080:00200000:2.0:1787826319.259481:0:161476:0:(file.c:6843:ll_layout_conf()) [0x240001f59:0x14:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:2.0:1787826319.259487:0:161476:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x240001f59:0x14:0x0](ffff9181c830aa08) 00000080:00010000:2.0:1787826319.259491:0:161476:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001f59:0x14:0x0](ffff9181c830aa08) for lock 0xbef42c0221d6f2e4 00000080:00010000:2.0:1787826319.259502:0:161476:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001f59:0x14:0x0](ffff9181c830aa08) 00000080:00200008:2.0:1787826319.259517:0:161476:0:(file.c:1090:ll_file_open()) START file f-18:[0x240001f59:0x14:0x0](ffff9181c830aa08), flags 100102 00000002:00100000:2.0:1787826319.259539:0:161476:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c49e7800 x1874663043015680/t17179870024(17179870024) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:2.0:1787826319.259576:0:161476:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f2e4 from it ffff9181e003aba0 00010000:00010000:2.0:1787826319.259581:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f2e4 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x14:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00cee174 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.259589:0:161476:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f2e4 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x14:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00cee174 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826319.259613:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x14:0x0](ffff9181c830aa08), inode mode 81a4 mask 40 00000080:00200008:2.0:1787826319.259643:0:161476:0:(file.c:479:ll_file_release()) START file f-18:[0x240001f59:0x14:0x0](ffff9181c830aa08), flags 100002 00000002:00100000:2.0:1787826319.259684:0:161476:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181c49e7800 x1874663043015680/t17179870024(17179870024) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:2.0:1787826319.259694:0:161476:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1787826319.259711:0:161476:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c49e6a00 x1874663043015936/t0(0) o35->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:2.0:1787826319.259734:0:161476:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c49e6a00 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043015936:192.168.203.147@tcp:35:createmany.0 00000100:00100000:2.0:1787826319.259831:0:161476:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5903600 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.263423:0:161476:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c49e6a00 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043015936:192.168.203.147@tcp:35:createmany.0 00000002:00100000:2.0:1787826319.263460:0:161476:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181c49e7800 x1874663043015680/t17179870024(17179870024) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1787826319.263535:0:161476:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:2.0:1787826319.263554:0:161476:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826319.263572:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 27 lru_type: 1 00010000:00010000:2.0:1787826319.263597:0:161476:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 27 lru_type: 1 00000080:00200000:2.0:1787826319.263605:0:161476:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826319.263611:0:161476:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:2.0:1787826319.263614:0:161476:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f24a 00000080:00010000:2.0:1787826319.263622:0:161476:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f24a from it ffffb97501c3bbf8 00010000:00010000:2.0:1787826319.263627:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 27 lru_type: 1 00010000:00010000:2.0:1787826319.263636:0:161476:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 28 lru_type: 1 00000080:00200000:2.0:1787826319.263652:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.263659:0:161476:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80a.replay-single, flags=25 00000080:00200000:2.0:1787826319.263665:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x2:0x0](ffff9181c83f3248), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.263669:0:161476:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=remote_dir, flags=25 00000080:00200000:2.0:1787826319.263672:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.263684:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 3 00000080:00200000:2.0:1787826319.263689:0:161476:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f-19, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), file ffff9181c5456d80, open_flags 8042, mode 81a4 opened 0 00000080:00200000:2.0:1787826319.263708:0:161476:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f-19, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), intent=open|creat 00000002:00010000:2.0:1787826319.263728:0:161476:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f-19,[0x240001f59:0x15:0x0]) in obj [0x240001f59:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:2.0:1787826319.263810:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842600/0xbef42c0221d6f2eb lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.263825:0:161476:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842600/0xbef42c0221d6f2eb lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1787826319.263835:0:161476:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1787826319.263841:0:161476:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c49e5500 x1874663043016192/t0(0) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1787826319.263851:0:161476:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842600/0xbef42c0221d6f2eb lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 161476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826319.263870:0:161476:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c49e5500 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043016192:192.168.203.147@tcp:101:createmany.0 00000100:00100000:2.0:1787826319.263995:0:161476:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5903600 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.269460:0:161476:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c49e5500 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043016192:192.168.203.147@tcp:101:createmany.0 00010000:00010000:2.0:1787826319.269487:0:161476:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842600/0xbef42c0221d6f2eb lrc: 4/0,1 mode: --/CW res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee197 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.269515:0:161476:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842600/0xbef42c0221d6f2eb lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x15:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee197 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.269523:0:161476:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842600/0xbef42c0221d6f2eb lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x15:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee197 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.269534:0:161476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842600/0xbef42c0221d6f2eb lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x15:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee197 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.269544:0:161476:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842600/0xbef42c0221d6f2eb lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x15:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee197 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.269562:0:161476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842600/0xbef42c0221d6f2eb lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x15:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee197 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.269570:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842600/0xbef42c0221d6f2eb lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x15:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee197 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.269578:0:161476:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842600/0xbef42c0221d6f2eb lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x15:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee197 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787826319.269600:0:161476:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9181c49e5500 x1874663043016192/t17179870026(17179870026) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:2.0:1787826319.269614:0:161476:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c49e5500 x1874663043016192/t17179870026(17179870026) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787826319.269626:0:161476:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842600/0xbef42c0221d6f2eb lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x15:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee197 expref: -99 pid: 161476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:2.0:1787826319.269638:0:161476:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842600/0xbef42c0221d6f2eb lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x15:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee197 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.269647:0:161476:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c77f3000 type 13 mode 4 res 9663684441/21 (0 0) 00000080:00200000:2.0:1787826319.269654:0:161476:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x240001f59:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1787826319.269666:0:161476:0:(namei.c:1628:ll_create_it()) VFS Op:name=f-19, dir=[0x240001f59:0x1:0x0](ffff9181d0801988), intent=open|creat 00000080:00200000:2.0:1787826319.269690:0:161476:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001f59:0x15:0x0](ffff9181c830ec08) 00000080:00200000:2.0:1787826319.269817:0:161476:0:(file.c:6843:ll_layout_conf()) [0x240001f59:0x15:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:2.0:1787826319.269822:0:161476:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x240001f59:0x15:0x0](ffff9181c830ec08) 00000080:00010000:2.0:1787826319.269826:0:161476:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001f59:0x15:0x0](ffff9181c830ec08) for lock 0xbef42c0221d6f2eb 00000080:00010000:2.0:1787826319.269836:0:161476:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001f59:0x15:0x0](ffff9181c830ec08) 00000080:00200008:2.0:1787826319.269850:0:161476:0:(file.c:1090:ll_file_open()) START file f-19:[0x240001f59:0x15:0x0](ffff9181c830ec08), flags 100102 00000002:00100000:2.0:1787826319.269873:0:161476:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c49e5500 x1874663043016192/t17179870026(17179870026) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:2.0:1787826319.269911:0:161476:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f2eb from it ffff9181e003aba0 00010000:00010000:2.0:1787826319.269916:0:161476:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842600/0xbef42c0221d6f2eb lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x15:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00cee197 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.269924:0:161476:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842600/0xbef42c0221d6f2eb lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x15:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00cee197 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826319.269945:0:161476:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x15:0x0](ffff9181c830ec08), inode mode 81a4 mask 40 00000080:00200008:2.0:1787826319.269985:0:161476:0:(file.c:479:ll_file_release()) START file f-19:[0x240001f59:0x15:0x0](ffff9181c830ec08), flags 100002 00000002:00100000:2.0:1787826319.270038:0:161476:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181c49e5500 x1874663043016192/t17179870026(17179870026) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:2.0:1787826319.270050:0:161476:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1787826319.270055:0:161476:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c49e5880 x1874663043016448/t0(0) o35->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:2.0:1787826319.270074:0:161476:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c49e5880 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043016448:192.168.203.147@tcp:35:createmany.0 00000100:00100000:2.0:1787826319.270137:0:161476:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5903780 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.273504:0:161476:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c49e5880 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161476:1874663043016448:192.168.203.147@tcp:35:createmany.0 00000002:00100000:2.0:1787826319.273538:0:161476:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181c49e5500 x1874663043016192/t17179870026(17179870026) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:3.0:1787826319.287659:0:161484:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:3.0:1787826319.287693:0:161484:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826319.287989:0:161484:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 28 lru_type: 1 00010000:00010000:3.0:1787826319.288005:0:161484:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 28 lru_type: 1 00000080:00200000:3.0:1787826319.288023:0:161484:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826319.288031:0:161484:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:3.0:1787826319.288034:0:161484:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f24a 00000080:00010000:3.0:1787826319.288043:0:161484:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f24a from it ffffb97500da3bf8 00010000:00010000:3.0:1787826319.288047:0:161484:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 28 lru_type: 1 00010000:00010000:3.0:1787826319.288056:0:161484:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 29 lru_type: 1 00000080:00200000:3.0:1787826319.288072:0:161484:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826319.288080:0:161484:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80a.replay-single, flags=27 00000080:00200000:3.0:1787826319.288086:0:161484:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x2:0x0](ffff9181c83f3248), inode mode 41ed mask 1 00000080:00200000:3.0:1787826319.288090:0:161484:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=remote_dir, flags=27 00000080:00200000:3.0:1787826319.288093:0:161484:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 1 00000080:00200000:3.0:1787826319.288098:0:161484:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f-1, flags=267 00000080:00200000:3.0:1787826319.288127:0:161484:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:3.0:1787826319.288136:0:161484:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826319.288143:0:161484:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 29 lru_type: 1 00010000:00010000:3.0:1787826319.288150:0:161484:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 29 lru_type: 1 00000080:00200000:3.0:1787826319.288157:0:161484:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826319.288160:0:161484:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:3.0:1787826319.288163:0:161484:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f24a 00000080:00010000:3.0:1787826319.288166:0:161484:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f24a from it ffffb97500da3bf8 00010000:00010000:3.0:1787826319.288170:0:161484:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 29 lru_type: 1 00010000:00010000:3.0:1787826319.288178:0:161484:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 30 lru_type: 1 00000080:00200000:3.0:1787826319.288186:0:161484:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826319.288190:0:161484:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80a.replay-single, flags=27 00000080:00200000:3.0:1787826319.288193:0:161484:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x2:0x0](ffff9181c83f3248), inode mode 41ed mask 1 00000080:00200000:3.0:1787826319.288196:0:161484:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=remote_dir, flags=267 00000080:00200000:3.0:1787826319.288199:0:161484:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 44 00000080:00200008:3.0:1787826319.288205:0:161484:0:(file.c:1090:ll_file_open()) START file remote_dir:[0x240001f59:0x1:0x0](ffff9181d0801988), flags 304000 00000002:00010000:3.0:1787826319.288220:0:161484:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001f59:0x1:0x0]) in obj [0x200027cd1:0x2:0x0], intent: open flags 0100240304001 00010000:00010000:3.0:1787826319.288267:0:161484:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481800/0xbef42c0221d6f2f2 lrc: 3/1,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 161484 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.288274:0:161484:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481800/0xbef42c0221d6f2f2 lrc: 3/1,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 161484 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:3.0:1787826319.288287:0:161484:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:3.0:1787826319.288567:0:161484:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c5634700 x1874663043016704/t0(0) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/131856 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:3.0:1787826319.288585:0:161484:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481800/0xbef42c0221d6f2f2 lrc: 3/1,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 161484 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826319.288607:0:161484:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634700 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161484:1874663043016704:192.168.203.147@tcp:101:lfs.0 00000100:00100000:3.0:1787826319.289198:0:161484:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c32f6d80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826319.292276:0:161484:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634700 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161484:1874663043016704:192.168.203.147@tcp:101:lfs.0 00010000:00010000:3.0:1787826319.292299:0:161484:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481800/0xbef42c0221d6f2f2 lrc: 4/1,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 161484 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.292311:0:161484:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481800/0xbef42c0221d6f2f2 lrc: 4/1,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 161484 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.292318:0:161484:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481800/0xbef42c0221d6f2f2 lrc: 4/1,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 161484 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.292324:0:161484:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481800/0xbef42c0221d6f2f2 lrc: 3/0,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 161484 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.292332:0:161484:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481800/0xbef42c0221d6f2f2 lrc: 4/0,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 161484 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.292337:0:161484:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3481800 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:3.0:1787826319.292340:0:161484:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481800/0xbef42c0221d6f2f2 lrc: 4/0,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x1/0x1 rrc: 2 type: IBT flags: 0x206400000000 pid: 161484 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.292346:0:161484:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481800/0xbef42c0221d6f2f2 lrc: 4/0,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x1/0x1 rrc: 2 type: IBT flags: 0x206400000000 pid: 161484 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.292352:0:161484:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481800/0xbef42c0221d6f2f2 lrc: 5/0,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x1/0x1 rrc: 2 type: IBT flags: 0x20e400000000 pid: 161484 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.292358:0:161484:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826319.292365:0:161484:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481800/0xbef42c0221d6f2f2 lrc: 3/0,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x1/0x1 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 161484 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.292371:0:161484:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481800/0xbef42c0221d6f2f2 lrc: 0/0,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x1/0x1 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 161484 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826319.292392:0:161484:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x1 disposition=0x2b, status=0 req@ffff9181c5634700 x1874663043016704/t17179870028(17179870028) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00100000:3.0:1787826319.292404:0:161484:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c5634700 x1874663043016704/t17179870028(17179870028) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:3.0:1787826319.292419:0:161484:0:(namei.c:952:ll_intent_lock()) intent lock 1 on i1 [0x200027cd1:0x2:0x0] suppgids 0 0: rc 0 00000080:00200000:3.0:1787826319.292443:0:161484:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x240001f59:0x1:0x0], updating i_size 4096 00000002:00100000:3.0:1787826319.292461:0:161484:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c5634700 x1874663043016704/t17179870028(17179870028) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:3.0:1787826319.292551:0:161484:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:3.0:1787826319.292565:0:161484:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826319.292578:0:161484:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 30 lru_type: 1 00010000:00010000:3.0:1787826319.292590:0:161484:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 30 lru_type: 1 00000080:00200000:3.0:1787826319.292598:0:161484:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826319.292603:0:161484:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:3.0:1787826319.292606:0:161484:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f24a 00000080:00010000:3.0:1787826319.292611:0:161484:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f24a from it ffffb97500da3bf8 00010000:00010000:3.0:1787826319.292615:0:161484:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 30 lru_type: 1 00010000:00010000:3.0:1787826319.292629:0:161484:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 31 lru_type: 1 00000080:00200000:3.0:1787826319.292642:0:161484:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826319.292648:0:161484:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80a.replay-single, flags=25 00000080:00200000:3.0:1787826319.292651:0:161484:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x2:0x0](ffff9181c83f3248), inode mode 41ed mask 1 00000080:00200000:3.0:1787826319.292654:0:161484:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=remote_dir, flags=25 00000080:00200000:3.0:1787826319.292656:0:161484:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 1 00000080:00200000:3.0:1787826319.292660:0:161484:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f-1, flags=265 00000080:00200000:3.0:1787826319.292666:0:161484:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x3:0x0](ffff9181d0803a88), inode mode 81a4 mask 44 00000080:00200008:3.0:1787826319.292672:0:161484:0:(file.c:1090:ll_file_open()) START file f-1:[0x240001f59:0x3:0x0](ffff9181d0803a88), flags 100400 00000002:00010000:3.0:1787826319.292684:0:161484:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001f59:0x3:0x0]) in obj [0x240001f59:0x1:0x0], intent: open flags 0240100401 00010000:00010000:3.0:1787826319.292726:0:161484:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480c00/0xbef42c0221d6f2f9 lrc: 3/1,0 mode: --/CR res: [0x240001f59:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 161484 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.292734:0:161484:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480c00/0xbef42c0221d6f2f9 lrc: 3/1,0 mode: --/CR res: [0x240001f59:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 161484 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:3.0:1787826319.292784:0:161484:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:3.0:1787826319.292792:0:161484:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c5634e00 x1874663043016960/t0(0) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/131856 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:3.0:1787826319.292802:0:161484:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480c00/0xbef42c0221d6f2f9 lrc: 3/1,0 mode: --/CR res: [0x240001f59:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 161484 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826319.292821:0:161484:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161484:1874663043016960:192.168.203.147@tcp:101:lfs.0 00000100:00100000:3.0:1787826319.292978:0:161484:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c32f6d80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826319.296099:0:161484:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161484:1874663043016960:192.168.203.147@tcp:101:lfs.0 00010000:00010000:3.0:1787826319.296129:0:161484:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480c00/0xbef42c0221d6f2f9 lrc: 4/1,0 mode: --/CR res: [0x240001f59:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee1ba expref: -99 pid: 161484 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.296155:0:161484:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480c00/0xbef42c0221d6f2f9 lrc: 4/1,0 mode: --/PR res: [0x240001f59:0x3:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee1ba expref: -99 pid: 161484 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.296162:0:161484:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480c00/0xbef42c0221d6f2f9 lrc: 4/1,0 mode: --/PR res: [0x240001f59:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee1ba expref: -99 pid: 161484 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.296173:0:161484:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480c00/0xbef42c0221d6f2f9 lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee1ba expref: -99 pid: 161484 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.296184:0:161484:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480c00/0xbef42c0221d6f2f9 lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee1ba expref: -99 pid: 161484 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.296198:0:161484:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480c00/0xbef42c0221d6f2f9 lrc: 5/2,0 mode: PR/PR res: [0x240001f59:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee1ba expref: -99 pid: 161484 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.296206:0:161484:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480c00/0xbef42c0221d6f2f9 lrc: 5/2,0 mode: PR/PR res: [0x240001f59:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee1ba expref: -99 pid: 161484 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.296214:0:161484:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480c00/0xbef42c0221d6f2f9 lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee1ba expref: -99 pid: 161484 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826319.296225:0:161484:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x1 disposition=0x2b, status=0 req@ffff9181c5634e00 x1874663043016960/t17179870029(17179870029) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00100000:3.0:1787826319.296236:0:161484:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c5634e00 x1874663043016960/t17179870029(17179870029) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826319.296248:0:161484:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480c00/0xbef42c0221d6f2f9 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee1ba expref: -99 pid: 161484 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:3.0:1787826319.296260:0:161484:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480c00/0xbef42c0221d6f2f9 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee1ba expref: -99 pid: 161484 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.296272:0:161484:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6c00/0xbef42c0221d6f26d lrc: 2/1,0 mode: PR/PR res: [0x240001f59:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00cedf21 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826319.296284:0:161484:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6c00/0xbef42c0221d6f26d lrc: 2/1,0 mode: PR/PR res: [0x240001f59:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00cedf21 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826319.296292:0:161484:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480c00/0xbef42c0221d6f2f9 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee1ba expref: -99 pid: 161484 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.296299:0:161484:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480c00/0xbef42c0221d6f2f9 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000400000000 nid: local remote: 0x878a5eea00cee1ba expref: -99 pid: 161484 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.296307:0:161484:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480c00/0xbef42c0221d6f2f9 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000400000000 nid: local remote: 0x878a5eea00cee1ba expref: -99 pid: 161484 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.296368:0:161484:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000080:00200000:3.0:1787826319.296375:0:161484:0:(namei.c:952:ll_intent_lock()) intent lock 1 on i1 [0x240001f59:0x1:0x0] suppgids 0 0: rc 0 00010000:00010000:0.0:1787826319.296388:0:159732:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000080:00200000:3.0:1787826319.296427:0:161484:0:(file.c:6843:ll_layout_conf()) [0x240001f59:0x3:0x0]: layout version change: 0 -> 0 00000080:00010000:3.0:1787826319.296433:0:161484:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001f59:0x3:0x0](ffff9181d0803a88) for lock 0xbef42c0221d6f26d 00000080:00010000:3.0:1787826319.296439:0:161484:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f26d from it ffffb97500da3bf0 00010000:00010000:3.0:1787826319.296442:0:161484:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6c00/0xbef42c0221d6f26d lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00cedf21 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826319.296451:0:161484:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6c00/0xbef42c0221d6f26d lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00cedf21 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 1 00000002:00100000:3.0:1787826319.296476:0:161484:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c5634e00 x1874663043016960/t17179870029(17179870029) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00010000:00010000:0.0:1787826319.296489:0:159732:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480c00/0xbef42c0221d6f2f9 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x400000000 nid: local remote: 0x878a5eea00cee1ba expref: -99 pid: 161484 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.296497:0:159732:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3480c00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787826319.296501:0:159732:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480c00/0xbef42c0221d6f2f9 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x3:0x0].0x0 bits 0x1b/0x1b rrc: 3 type: IBT gid 0 flags: 0x400000000 nid: local remote: 0x878a5eea00cee1ba expref: -99 pid: 161484 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.296509:0:159732:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480c00/0xbef42c0221d6f2f9 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x3:0x0].0x0 bits 0x1b/0x1b rrc: 3 type: IBT gid 0 flags: 0x400000000 nid: local remote: 0x878a5eea00cee1ba expref: -99 pid: 161484 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.296517:0:159732:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480c00/0xbef42c0221d6f2f9 lrc: 3/0,0 mode: PR/PR res: [0x240001f59:0x3:0x0].0x0 bits 0x1b/0x1b rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00cee1ba expref: -99 pid: 161484 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.296532:0:159732:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6c00/0xbef42c0221d6f26d lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00cedf21 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 1 00000080:00200080:3.0:1787826319.296552:0:161484:0:(file.c:4646:ll_file_ioctl()) VFS Op:inode=[0x240001f59:0x3:0x0](ffff9181d0803a88) cmd=800466af arg=7fff2e75d6ec 00010000:00010000:0.0:1787826319.296564:0:159732:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480c00/0xbef42c0221d6f2f9 lrc: 2/0,0 mode: --/PR res: [0x240001f59:0x3:0x0].0x0 bits 0x1b/0x1b rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cee1ba expref: -99 pid: 161484 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.296574:0:159732:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000080:00200008:3.0:1787826319.296574:0:161484:0:(file.c:479:ll_file_release()) START file f-1:[0x240001f59:0x3:0x0](ffff9181d0803a88), flags 100000 00000002:00100000:3.0:1787826319.296605:0:161484:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181c5634e00 x1874663043016960/t17179870029(17179870029) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:0 00010000:00010000:0.0:1787826319.296613:0:159732:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480c00/0xbef42c0221d6f2f9 lrc: 1/0,0 mode: --/PR res: [0x240001f59:0x3:0x0].0x0 bits 0x1b/0x1b rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cee1ba expref: -99 pid: 161484 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00000020:00100000:3.0:1787826319.296614:0:161484:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:3.0:1787826319.296619:0:161484:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c5637480 x1874663043017344/t0(0) o35->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:0.0:1787826319.296621:0:159732:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480c00/0xbef42c0221d6f2f9 lrc: 0/0,0 mode: --/PR res: [0x240001f59:0x3:0x0].0x0 bits 0x1b/0x1b rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cee1ba expref: -99 pid: 161484 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.296635:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826319.296637:0:161484:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161484:1874663043017344:192.168.203.147@tcp:35:lfs.0 00010000:00010000:0.0:1787826319.296638:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826319.296640:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826319.296679:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043017088:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1787826319.296964:0:161484:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c32f6840 going to sleep for 11 seconds 00000100:00100000:3.0:1787826319.308128:0:161484:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161484:1874663043017344:192.168.203.147@tcp:35:lfs.0 00000100:00100000:1.0:1787826319.308133:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043017088:192.168.203.147@tcp:103:ldlm_bl.0 00000002:00100000:3.0:1787826319.308178:0:161484:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181c5634e00 x1874663043016960/t17179870029(17179870029) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826335 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200008:3.0:1787826319.308344:0:161484:0:(file.c:479:ll_file_release()) START file remote_dir:[0x240001f59:0x1:0x0](ffff9181d0801988), flags 304000 00000002:00100000:3.0:1787826319.308409:0:161484:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181c5634700 x1874663043016704/t17179870028(17179870028) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 0 dl 1787826335 ref 1 fl Complete:RPQU/604/ffffffff rc 301/-1 job:'lfs.0' uid:0 gid:0 projid:0 00000020:00100000:3.0:1787826319.308419:0:161484:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:3.0:1787826319.309233:0:161484:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c5637480 x1874663043017600/t0(0) o35->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:3.0:1787826319.309268:0:161484:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161484:1874663043017600:192.168.203.147@tcp:35:lfs.0 00000100:00100000:3.0:1787826319.309348:0:161484:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c32f6840 going to sleep for 11 seconds 00000100:00100000:3.0:1787826319.319566:0:161484:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161484:1874663043017600:192.168.203.147@tcp:35:lfs.0 00000002:00100000:0.0:1787826319.319696:0:161484:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181c5634700 x1874663043016704/t17179870028(17179870028) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 0 dl 1787826335 ref 1 fl Complete:RQU/604/ffffffff rc 301/-1 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787826319.330066:0:161485:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:0.0:1787826319.330096:0:161485:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826319.330115:0:161485:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 31 lru_type: 1 00010000:00010000:0.0:1787826319.330131:0:161485:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 31 lru_type: 1 00000080:00200000:0.0:1787826319.330139:0:161485:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826319.330147:0:161485:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:0.0:1787826319.330150:0:161485:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f24a 00000080:00010000:0.0:1787826319.330158:0:161485:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f24a from it ffffb97501c3bb58 00010000:00010000:0.0:1787826319.330809:0:161485:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 31 lru_type: 1 00010000:00010000:0.0:1787826319.330826:0:161485:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 32 lru_type: 1 00000080:00200000:0.0:1787826319.330840:0:161485:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826319.330848:0:161485:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80a.replay-single, flags=8 00000080:00200008:0.0:1787826319.330857:0:161485:0:(file.c:6296:ll_getattr_dentry()) START file d80a.replay-single:[0x200027cd1:0x2:0x0](ffff9181c83f3248), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1787826319.330861:0:161485:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200027cd1:0x2:0x0](ffff9181c83f3248),name=d80a.replay-single 00000002:00010000:0.0:1787826319.330875:0:161485:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200027cd1:0x2:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826319.330884:0:161485:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4e00/0xbef42c0221d6f251 lrc: 2/1,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedeaa expref: -99 pid: 161472 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826319.330892:0:161485:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4e00/0xbef42c0221d6f251 lrc: 2/1,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedeaa expref: -99 pid: 161472 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826319.330899:0:161485:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826319.330904:0:161485:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200027cd1:0x2:0x0](ffff9181c83f3248) 00000080:00010000:0.0:1787826319.330906:0:161485:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200027cd1:0x2:0x0](ffff9181c83f3248) for lock 0xbef42c0221d6f251 00000080:00010000:0.0:1787826319.330911:0:161485:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f251 from it ffffb97501c3bcb8 00010000:00010000:0.0:1787826319.330913:0:161485:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4e00/0xbef42c0221d6f251 lrc: 3/1,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cedeaa expref: -99 pid: 161472 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826319.330920:0:161485:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4e00/0xbef42c0221d6f251 lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedeaa expref: -99 pid: 161472 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787826319.330971:0:161485:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:0.0:1787826319.330979:0:161485:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826319.330985:0:161485:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 32 lru_type: 1 00010000:00010000:0.0:1787826319.330993:0:161485:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 32 lru_type: 1 00000080:00200000:0.0:1787826319.330999:0:161485:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826319.331003:0:161485:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:0.0:1787826319.331006:0:161485:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f24a 00000080:00010000:0.0:1787826319.331009:0:161485:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f24a from it ffffb97501c3bbf8 00010000:00010000:0.0:1787826319.331012:0:161485:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 32 lru_type: 1 00010000:00010000:0.0:1787826319.331018:0:161485:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 33 lru_type: 1 00000080:00200000:0.0:1787826319.331026:0:161485:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826319.331030:0:161485:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80a.replay-single, flags=266 00000080:00200000:0.0:1787826319.331034:0:161485:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x2:0x0](ffff9181c83f3248), inode mode 41ed mask 44 00000080:00200008:0.0:1787826319.331039:0:161485:0:(file.c:1090:ll_file_open()) START file d80a.replay-single:[0x200027cd1:0x2:0x0](ffff9181c83f3248), flags 704400 00000002:00010000:0.0:1787826319.331053:0:161485:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200027cd1:0x2:0x0]) in obj [0x200000007:0x1:0x0], intent: open flags 0100240704401 00010000:00010000:0.0:1787826319.331099:0:161485:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513400/0xbef42c0221d6f300 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 161485 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.331106:0:161485:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513400/0xbef42c0221d6f300 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 161485 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:0.0:1787826319.331118:0:161485:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:0.0:1787826319.331131:0:161485:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c55a4380 x1874663043017856/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/131856 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:0.0:1787826319.331141:0:161485:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513400/0xbef42c0221d6f300 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 161485 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826319.331524:0:161485:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043017856:192.168.203.147@tcp:101:rm.0 00000100:00100000:0.0:1787826319.331714:0:161485:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5256600 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.340970:0:161485:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043017856:192.168.203.147@tcp:101:rm.0 00010000:00010000:2.0:1787826319.340994:0:161485:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513400/0xbef42c0221d6f300 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 161485 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.341008:0:161485:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513400/0xbef42c0221d6f300 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 161485 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.341016:0:161485:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513400/0xbef42c0221d6f300 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 161485 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.341024:0:161485:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513400/0xbef42c0221d6f300 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 161485 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.341032:0:161485:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513400/0xbef42c0221d6f300 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 161485 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.341038:0:161485:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0513400 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:2.0:1787826319.341043:0:161485:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513400/0xbef42c0221d6f300 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x206400000000 pid: 161485 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.341050:0:161485:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513400/0xbef42c0221d6f300 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x206400000000 pid: 161485 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.341057:0:161485:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513400/0xbef42c0221d6f300 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x20e400000000 pid: 161485 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.341063:0:161485:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787826319.341069:0:161485:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513400/0xbef42c0221d6f300 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 161485 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.341075:0:161485:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513400/0xbef42c0221d6f300 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 161485 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787826319.341096:0:161485:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x1 disposition=0x2b, status=0 req@ffff9181c55a4380 x1874663043017856/t352187318276(352187318276) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'rm.0' uid:0 gid:0 projid:0 00000002:00100000:2.0:1787826319.341109:0:161485:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c55a4380 x1874663043017856/t352187318276(352187318276) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'rm.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1787826319.341124:0:161485:0:(namei.c:952:ll_intent_lock()) intent lock 1 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:2.0:1787826319.341152:0:161485:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200027cd1:0x2:0x0], updating i_size 4096 00000002:00100000:2.0:1787826319.341186:0:161485:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c55a4380 x1874663043017856/t352187318276(352187318276) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'rm.0' uid:0 gid:0 projid:0 00000080:00200008:2.0:1787826319.341252:0:161485:0:(file.c:6296:ll_getattr_dentry()) START file d80a.replay-single:[0x200027cd1:0x2:0x0](ffff9181c83f3248), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1787826319.341259:0:161485:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200027cd1:0x2:0x0](ffff9181c83f3248),name=d80a.replay-single 00000002:00010000:2.0:1787826319.341280:0:161485:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200027cd1:0x2:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826319.341295:0:161485:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4e00/0xbef42c0221d6f251 lrc: 2/1,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedeaa expref: -99 pid: 161472 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787826319.341306:0:161485:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4e00/0xbef42c0221d6f251 lrc: 2/1,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedeaa expref: -99 pid: 161472 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1787826319.341314:0:161485:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826319.341320:0:161485:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200027cd1:0x2:0x0](ffff9181c83f3248) 00000080:00010000:2.0:1787826319.341324:0:161485:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200027cd1:0x2:0x0](ffff9181c83f3248) for lock 0xbef42c0221d6f251 00000080:00010000:2.0:1787826319.341329:0:161485:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f251 from it ffffb97501c3bcd8 00010000:00010000:2.0:1787826319.341333:0:161485:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4e00/0xbef42c0221d6f251 lrc: 3/1,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cedeaa expref: -99 pid: 161472 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787826319.341341:0:161485:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4e00/0xbef42c0221d6f251 lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedeaa expref: -99 pid: 161472 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:2.0:1787826319.341594:0:161485:0:(dir.c:332:ll_iterate()) VFS Op:inode=[0x200027cd1:0x2:0x0](ffff9181c83f3248) pos/size0/4096 32bit_api 0 00000002:00010000:2.0:1787826319.341609:0:161485:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200027cd1:0x2:0x0]) in obj [0x200027cd1:0x2:0x0], intent: readdir flags 0 00010000:00010000:2.0:1787826319.341626:0:161485:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4e00/0xbef42c0221d6f251 lrc: 2/1,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedeaa expref: -99 pid: 161472 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1787826319.341635:0:161485:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4e00/0xbef42c0221d6f251 lrc: 2/1,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedeaa expref: -99 pid: 161472 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000100:00100000:3.0:1787826319.343265:0:161485:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043017984:192.168.203.147@tcp:37:rm.0 00000100:00100000:3.0:1787826319.343356:0:161485:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c32f6840 going to sleep for 11 seconds 00000100:00100000:3.0:1787826319.364732:0:161485:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043017984:192.168.203.147@tcp:37:rm.0 00010000:00010000:3.0:1787826319.365618:0:161485:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4e00/0xbef42c0221d6f251 lrc: 3/1,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cedeaa expref: -99 pid: 161472 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:3.0:1787826319.365636:0:161485:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4e00/0xbef42c0221d6f251 lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedeaa expref: -99 pid: 161472 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00000080:00200008:3.0:1787826319.368729:0:161485:0:(file.c:479:ll_file_release()) START file d80a.replay-single:[0x200027cd1:0x2:0x0](ffff9181c83f3248), flags 704000 00000002:00100000:3.0:1787826319.368977:0:161485:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181c55a4380 x1874663043017856/t352187318276(352187318276) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 0 dl 1787826335 ref 1 fl Complete:RPQU/604/ffffffff rc 301/-1 job:'rm.0' uid:0 gid:0 projid:0 00000020:00100000:3.0:1787826319.368992:0:161485:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:3.0:1787826319.368998:0:161485:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c5634e00 x1874663043018240/t0(0) o35->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:3.0:1787826319.369147:0:161485:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043018240:192.168.203.147@tcp:35:rm.0 00000100:00100000:3.0:1787826319.369299:0:161485:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c32f6840 going to sleep for 11 seconds 00000100:00100000:3.0:1787826319.382872:0:161485:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043018240:192.168.203.147@tcp:35:rm.0 00000002:00100000:3.0:1787826319.382917:0:161485:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181c55a4380 x1874663043017856/t352187318276(352187318276) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 0 dl 1787826335 ref 1 fl Complete:RQU/604/ffffffff rc 301/-1 job:'rm.0' uid:0 gid:0 projid:0 00000080:00200000:3.0:1787826319.383139:0:161485:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:3.0:1787826319.383159:0:161485:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826319.383177:0:161485:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 33 lru_type: 1 00010000:00010000:3.0:1787826319.383190:0:161485:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 33 lru_type: 1 00000080:00200000:3.0:1787826319.383197:0:161485:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826319.383202:0:161485:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:3.0:1787826319.383206:0:161485:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f24a 00000080:00010000:3.0:1787826319.383212:0:161485:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f24a from it ffffb97501c3bbf8 00010000:00010000:3.0:1787826319.383216:0:161485:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 33 lru_type: 1 00010000:00010000:3.0:1787826319.383224:0:161485:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 34 lru_type: 1 00000080:00200000:3.0:1787826319.383238:0:161485:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826319.383245:0:161485:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80a.replay-single, flags=266 00000080:00200000:3.0:1787826319.383251:0:161485:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x2:0x0](ffff9181c83f3248), inode mode 41ed mask 44 00000080:00200008:3.0:1787826319.383258:0:161485:0:(file.c:1090:ll_file_open()) START file d80a.replay-single:[0x200027cd1:0x2:0x0](ffff9181c83f3248), flags 704400 00000002:00010000:3.0:1787826319.383271:0:161485:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200027cd1:0x2:0x0]) in obj [0x200000007:0x1:0x0], intent: open flags 0104240704401 00010000:00010000:3.0:1787826319.383312:0:161485:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481800/0xbef42c0221d6f307 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 161485 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.383320:0:161485:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481800/0xbef42c0221d6f307 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 161485 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:3.0:1787826319.383332:0:161485:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:3.0:1787826319.383339:0:161485:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c5634e00 x1874663043018496/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/131856 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:3.0:1787826319.383348:0:161485:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481800/0xbef42c0221d6f307 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 161485 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826319.383367:0:161485:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043018496:192.168.203.147@tcp:101:rm.0 00000100:00100000:3.0:1787826319.383545:0:161485:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c32f6840 going to sleep for 11 seconds 00000100:00100000:3.0:1787826319.387865:0:161485:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043018496:192.168.203.147@tcp:101:rm.0 00010000:00010000:3.0:1787826319.387900:0:161485:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481800/0xbef42c0221d6f307 lrc: 4/1,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee1dd expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.387911:0:161485:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481800/0xbef42c0221d6f307 lrc: 4/1,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee1dd expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.387922:0:161485:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481800/0xbef42c0221d6f307 lrc: 4/1,0 mode: CR/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee1dd expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.387932:0:161485:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481800/0xbef42c0221d6f307 lrc: 4/1,0 mode: CR/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee1dd expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826319.387948:0:161485:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x1 disposition=0x200002b, status=0 req@ffff9181c5634e00 x1874663043018496/t352187318278(352187318278) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00100000:3.0:1787826319.387971:0:161485:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c5634e00 x1874663043018496/t352187318278(352187318278) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826319.387991:0:161485:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481800/0xbef42c0221d6f307 lrc: 3/1,0 mode: CR/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee1dd expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.388002:0:161485:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d1a77c00 type 13 mode 16 res 8590097617/2 (0 0) 00000080:00200000:3.0:1787826319.388121:0:161485:0:(namei.c:952:ll_intent_lock()) intent lock 1 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:3.0:1787826319.388150:0:161485:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200027cd1:0x2:0x0], updating i_size 4096 00000080:00010000:3.0:1787826319.388156:0:161485:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200027cd1:0x2:0x0](ffff9181c83f3248) for lock 0xbef42c0221d6f307 00000080:00010000:3.0:1787826319.388163:0:161485:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f307 from it ffffb97501c3bbd0 00010000:00010000:3.0:1787826319.388167:0:161485:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481800/0xbef42c0221d6f307 lrc: 3/1,0 mode: CR/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee1dd expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826319.388178:0:161485:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481800/0xbef42c0221d6f307 lrc: 2/0,0 mode: CR/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee1dd expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000002:00100000:3.0:1787826319.388204:0:161485:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c5634e00 x1874663043018496/t352187318278(352187318278) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000080:00200008:3.0:1787826319.388265:0:161485:0:(file.c:6296:ll_getattr_dentry()) START file d80a.replay-single:[0x200027cd1:0x2:0x0](ffff9181c83f3248), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1787826319.388272:0:161485:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200027cd1:0x2:0x0](ffff9181c83f3248),name=d80a.replay-single 00000002:00010000:3.0:1787826319.388292:0:161485:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200027cd1:0x2:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826319.388302:0:161485:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4e00/0xbef42c0221d6f251 lrc: 2/1,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedeaa expref: -99 pid: 161472 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:3.0:1787826319.388313:0:161485:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4e00/0xbef42c0221d6f251 lrc: 2/1,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedeaa expref: -99 pid: 161472 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00000080:00200000:3.0:1787826319.388319:0:161485:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826319.388325:0:161485:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200027cd1:0x2:0x0](ffff9181c83f3248) 00000080:00010000:3.0:1787826319.388329:0:161485:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200027cd1:0x2:0x0](ffff9181c83f3248) for lock 0xbef42c0221d6f251 00000080:00010000:3.0:1787826319.388332:0:161485:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f251 from it ffffb97501c3bcd8 00010000:00010000:3.0:1787826319.388335:0:161485:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4e00/0xbef42c0221d6f251 lrc: 3/1,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cedeaa expref: -99 pid: 161472 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:3.0:1787826319.388342:0:161485:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4e00/0xbef42c0221d6f251 lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedeaa expref: -99 pid: 161472 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00000080:00200000:3.0:1787826319.388427:0:161485:0:(dir.c:332:ll_iterate()) VFS Op:inode=[0x200027cd1:0x2:0x0](ffff9181c83f3248) pos/size0/4096 32bit_api 0 00000080:00200000:3.0:1787826319.388454:0:161485:0:(dir.c:332:ll_iterate()) VFS Op:inode=[0x200027cd1:0x2:0x0](ffff9181c83f3248) pos/size18446744073709551614/4096 32bit_api 0 00000080:00200000:3.0:1787826319.388472:0:161485:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x2:0x0](ffff9181c83f3248), inode mode 41ed mask 1 00000080:00200000:3.0:1787826319.388479:0:161485:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=remote_dir, flags=0 00000080:00200008:3.0:1787826319.388486:0:161485:0:(file.c:6296:ll_getattr_dentry()) START file remote_dir:[0x240001f59:0x1:0x0](ffff9181d0801988), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1787826319.396778:0:161485:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988),name=remote_dir 00000002:00010000:2.0:1787826319.396823:0:161485:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001f59:0x1:0x0]) in obj [0x200027cd1:0x2:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826319.396883:0:161485:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843e00/0xbef42c0221d6f30e lrc: 3/1,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 161485 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.396897:0:161485:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843e00/0xbef42c0221d6f30e lrc: 3/1,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 161485 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.396907:0:161485:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843e00/0xbef42c0221d6f30e lrc: 3/1,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 161485 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826319.396933:0:161485:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c49e6d80 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043018624:192.168.203.147@tcp:101:rm.0 00000100:00100000:2.0:1787826319.397034:0:161485:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5903780 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.405783:0:161485:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c49e6d80 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043018624:192.168.203.147@tcp:101:rm.0 00010000:00010000:2.0:1787826319.405812:0:161485:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843e00/0xbef42c0221d6f30e lrc: 4/1,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee1f2 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.405840:0:161485:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843e00/0xbef42c0221d6f30e lrc: 4/1,0 mode: --/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee1f2 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.405854:0:161485:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843e00/0xbef42c0221d6f30e lrc: 4/1,0 mode: --/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee1f2 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.405864:0:161485:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843e00/0xbef42c0221d6f30e lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee1f2 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.405875:0:161485:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843e00/0xbef42c0221d6f30e lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee1f2 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.405887:0:161485:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843e00/0xbef42c0221d6f30e lrc: 5/2,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee1f2 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.405895:0:161485:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843e00/0xbef42c0221d6f30e lrc: 5/2,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee1f2 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.405903:0:161485:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843e00/0xbef42c0221d6f30e lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee1f2 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787826319.405914:0:161485:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c49e6d80 x1874663043018624/t0(0) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826335 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787826319.405938:0:161485:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843e00/0xbef42c0221d6f30e lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee1f2 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.405948:0:161485:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c77f3000 type 13 mode 4 res 9663684441/1 (0 0) 00000080:00200000:2.0:1787826319.405953:0:161485:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200027cd1:0x2:0x0] suppgids 0 0: rc 0 00000080:00200000:2.0:1787826319.405988:0:161485:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x240001f59:0x1:0x0], updating i_size 4096 00000080:00010000:2.0:1787826319.405994:0:161485:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001f59:0x1:0x0](ffff9181d0801988) for lock 0xbef42c0221d6f30e 00000080:00010000:2.0:1787826319.405998:0:161485:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001f59:0x1:0x0](ffff9181d0801988) 00000080:00010000:2.0:1787826319.406001:0:161485:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f30e from it ffffb97501c3bcb8 00010000:00010000:2.0:1787826319.406005:0:161485:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843e00/0xbef42c0221d6f30e lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee1f2 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.406013:0:161485:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843e00/0xbef42c0221d6f30e lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee1f2 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826319.406073:0:161485:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x2:0x0](ffff9181c83f3248), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.406082:0:161485:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=remote_dir, flags=258 00000080:00200000:2.0:1787826319.406097:0:161485:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 44 00000080:00200008:2.0:1787826319.406106:0:161485:0:(file.c:1090:ll_file_open()) START file remote_dir:[0x240001f59:0x1:0x0](ffff9181d0801988), flags 704400 00000002:00010000:2.0:1787826319.406130:0:161485:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001f59:0x1:0x0]) in obj [0x200027cd1:0x2:0x0], intent: open flags 0104240704401 00010000:00010000:2.0:1787826319.406187:0:161485:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843a00/0xbef42c0221d6f315 lrc: 3/1,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 161485 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.406195:0:161485:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843a00/0xbef42c0221d6f315 lrc: 3/1,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 161485 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1787826319.406206:0:161485:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1787826319.406213:0:161485:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c49e6d80 x1874663043018880/t0(0) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/131856 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1787826319.406222:0:161485:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843a00/0xbef42c0221d6f315 lrc: 3/1,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 161485 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826319.406241:0:161485:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c49e6d80 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043018880:192.168.203.147@tcp:101:rm.0 00000100:00100000:2.0:1787826319.406682:0:161485:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5903780 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.410341:0:161485:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c49e6d80 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043018880:192.168.203.147@tcp:101:rm.0 00010000:00010000:2.0:1787826319.410395:0:161485:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843a00/0xbef42c0221d6f315 lrc: 4/1,0 mode: --/CR res: [0x240001f59:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee200 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.410407:0:161485:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843a00/0xbef42c0221d6f315 lrc: 4/1,0 mode: --/CR res: [0x240001f59:0x1:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee200 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.410420:0:161485:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843a00/0xbef42c0221d6f315 lrc: 4/1,0 mode: CR/CR res: [0x240001f59:0x1:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee200 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.410432:0:161485:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843a00/0xbef42c0221d6f315 lrc: 4/1,0 mode: CR/CR res: [0x240001f59:0x1:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee200 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787826319.410447:0:161485:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x1 disposition=0x200002b, status=0 req@ffff9181c49e6d80 x1874663043018880/t17179870032(17179870032) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00100000:2.0:1787826319.410459:0:161485:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c49e6d80 x1874663043018880/t17179870032(17179870032) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787826319.410475:0:161485:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843a00/0xbef42c0221d6f315 lrc: 3/1,0 mode: CR/CR res: [0x240001f59:0x1:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee200 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.410486:0:161485:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c77f3000 type 13 mode 16 res 9663684441/1 (0 0) 00000080:00200000:2.0:1787826319.410492:0:161485:0:(namei.c:952:ll_intent_lock()) intent lock 1 on i1 [0x200027cd1:0x2:0x0] suppgids 0 0: rc 0 00000080:00200000:2.0:1787826319.410517:0:161485:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x240001f59:0x1:0x0], updating i_size 4096 00000080:00010000:2.0:1787826319.410523:0:161485:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001f59:0x1:0x0](ffff9181d0801988) for lock 0xbef42c0221d6f315 00000080:00010000:2.0:1787826319.410530:0:161485:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f315 from it ffffb97501c3bbd0 00010000:00010000:2.0:1787826319.410534:0:161485:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843a00/0xbef42c0221d6f315 lrc: 3/1,0 mode: CR/CR res: [0x240001f59:0x1:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee200 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.410549:0:161485:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843a00/0xbef42c0221d6f315 lrc: 2/0,0 mode: CR/CR res: [0x240001f59:0x1:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee200 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000002:00100000:2.0:1787826319.410576:0:161485:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c49e6d80 x1874663043018880/t17179870032(17179870032) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 0 dl 1787826335 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000080:00200008:2.0:1787826319.410650:0:161485:0:(file.c:6296:ll_getattr_dentry()) START file remote_dir:[0x240001f59:0x1:0x0](ffff9181d0801988), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1787826319.410658:0:161485:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988),name=remote_dir 00000002:00010000:2.0:1787826319.410678:0:161485:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001f59:0x1:0x0]) in obj [0x200027cd1:0x2:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826319.410689:0:161485:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843e00/0xbef42c0221d6f30e lrc: 2/1,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee1f2 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826319.410700:0:161485:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843e00/0xbef42c0221d6f30e lrc: 2/1,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee1f2 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826319.410708:0:161485:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200027cd1:0x2:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826319.410714:0:161485:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001f59:0x1:0x0](ffff9181d0801988) 00000080:00010000:2.0:1787826319.410717:0:161485:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001f59:0x1:0x0](ffff9181d0801988) for lock 0xbef42c0221d6f30e 00000080:00010000:2.0:1787826319.410722:0:161485:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f30e from it ffffb97501c3bcd8 00010000:00010000:2.0:1787826319.410725:0:161485:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843e00/0xbef42c0221d6f30e lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee1f2 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826319.410732:0:161485:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843e00/0xbef42c0221d6f30e lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee1f2 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1787826319.410784:0:161485:0:(dir.c:332:ll_iterate()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988) pos/size0/4096 32bit_api 0 00000002:00010000:2.0:1787826319.410798:0:161485:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001f59:0x1:0x0]) in obj [0x240001f59:0x1:0x0], intent: readdir flags 0 00010000:00010000:2.0:1787826319.410805:0:161485:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843e00/0xbef42c0221d6f30e lrc: 2/1,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee1f2 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787826319.410813:0:161485:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843e00/0xbef42c0221d6f30e lrc: 2/1,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee1f2 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000100:00100000:2.0:1787826319.411478:0:161485:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c49e5f80 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043019008:192.168.203.147@tcp:37:rm.0 00000100:00100000:2.0:1787826319.411570:0:161485:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5903780 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.425566:0:161485:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c49e5f80 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043019008:192.168.203.147@tcp:37:rm.0 00010000:00010000:2.0:1787826319.426554:0:161485:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843e00/0xbef42c0221d6f30e lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee1f2 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787826319.426575:0:161485:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843e00/0xbef42c0221d6f30e lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee1f2 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00000080:00200008:2.0:1787826319.426908:0:161485:0:(file.c:479:ll_file_release()) START file remote_dir:[0x240001f59:0x1:0x0](ffff9181d0801988), flags 704000 00010000:00010000:2.0:1787826319.426937:0:161485:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843a00/0xbef42c0221d6f315 lrc: 2/0,0 mode: CR/CR res: [0x240001f59:0x1:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee200 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1787826319.426988:0:161485:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x2:0x0](ffff9181c83f3248), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.426995:0:161485:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=remote_dir, flags=258 00000080:00200000:2.0:1787826319.427002:0:161485:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 44 00000080:00200008:2.0:1787826319.427008:0:161485:0:(file.c:1090:ll_file_open()) START file remote_dir:[0x240001f59:0x1:0x0](ffff9181d0801988), flags 704400 00000080:00200008:2.0:1787826319.427026:0:161485:0:(file.c:6296:ll_getattr_dentry()) START file remote_dir:[0x240001f59:0x1:0x0](ffff9181d0801988), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1787826319.427031:0:161485:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988),name=remote_dir 00000002:00010000:2.0:1787826319.427050:0:161485:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001f59:0x1:0x0]) in obj [0x200027cd1:0x2:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826319.427060:0:161485:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843e00/0xbef42c0221d6f30e lrc: 2/1,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee1f2 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00010000:00010000:2.0:1787826319.427070:0:161485:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843e00/0xbef42c0221d6f30e lrc: 2/1,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee1f2 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00000080:00200000:2.0:1787826319.427077:0:161485:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200027cd1:0x2:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826319.427083:0:161485:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001f59:0x1:0x0](ffff9181d0801988) 00000080:00010000:2.0:1787826319.427086:0:161485:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001f59:0x1:0x0](ffff9181d0801988) for lock 0xbef42c0221d6f30e 00000080:00010000:2.0:1787826319.427092:0:161485:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f30e from it ffffb97501c3bcd8 00010000:00010000:2.0:1787826319.427095:0:161485:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843e00/0xbef42c0221d6f30e lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee1f2 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00010000:00010000:2.0:1787826319.427103:0:161485:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843e00/0xbef42c0221d6f30e lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee1f2 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00000080:00200000:2.0:1787826319.427128:0:161485:0:(llite_lib.c:2903:ll_statfs()) VFS Op:sb=lustre (ffff9181c5620000) remote_dir proj=0 00000080:00000004:2.0:1787826319.427133:0:161485:0:(obd_class.h:1090:obd_statfs()) lustre-clilmv-ffff9181c5620000: age 8306, max_age 8390 00800000:00000004:2.0:1787826319.427143:0:161485:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9181c5620000: age 8390, max_age 8390 00800000:00000004:2.0:1787826319.427146:0:161485:0:(obd_class.h:1130:obd_statfs()) lustre-MDT0000-mdc-ffff9181c5620000: use ffff9181c354d178 cache blocks 317223/353529 objects 1023643/1024000 00800000:00000004:2.0:1787826319.427150:0:161485:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9181c5620000: age 8390, max_age 8390 00800000:00000004:2.0:1787826319.427151:0:161485:0:(obd_class.h:1130:obd_statfs()) lustre-MDT0001-mdc-ffff9181c5620000: use ffff9181c3548450 cache blocks 317323/353529 objects 1023702/1024000 00000080:00000004:2.0:1787826319.427155:0:161485:0:(obd_class.h:1116:obd_statfs()) lustre-clilmv-ffff9181c5620000: update ffff9181d1a19e08 cache blocks 634546/707058 objects 2047345/2048000 00000080:00000004:2.0:1787826319.427158:0:161485:0:(llite_lib.c:2755:ll_statfs_internal()) MDC blocks 634546/707058 objects 2047345/2048000 00000080:00000004:2.0:1787826319.427161:0:161485:0:(obd_class.h:1090:obd_statfs()) lustre-clilov-ffff9181c5620000: age 7306, max_age 8390 00020000:00000004:2.0:1787826319.427174:0:161485:0:(obd_class.h:1056:obd_statfs_async()) lustre-OST0000-osc-ffff9181c5620000: age 8390, max_age 8390 00000008:00000004:2.0:1787826319.427178:0:161485:0:(osc_request.c:3417:osc_statfs_async()) lustre-OST0000-osc-ffff9181c5620000: use ffff9181c3549e08 cache blocks 901356/958279 objects 261710/262144 00020000:00000004:2.0:1787826319.427183:0:161485:0:(obd_class.h:1056:obd_statfs_async()) lustre-OST0001-osc-ffff9181c5620000: age 8390, max_age 8390 00000008:00000004:2.0:1787826319.427185:0:161485:0:(osc_request.c:3417:osc_statfs_async()) lustre-OST0001-osc-ffff9181c5620000: use ffff9181c354b7c0 cache blocks 901360/958279 objects 261727/262144 00000080:00000004:2.0:1787826319.427193:0:161485:0:(obd_class.h:1116:obd_statfs()) lustre-clilov-ffff9181c5620000: update ffff9181d1a1b7c0 cache blocks 1802716/1916558 objects 523437/524288 00000080:00000004:2.0:1787826319.427196:0:161485:0:(llite_lib.c:2766:ll_statfs_internal()) OSC blocks 1802716/1916558 objects 523437/524288 00000080:00200000:2.0:1787826319.427209:0:161485:0:(dir.c:332:ll_iterate()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988) pos/size0/4096 32bit_api 0 00000080:00200000:2.0:1787826319.427268:0:161485:0:(dir.c:332:ll_iterate()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988) pos/size18446744073709551614/4096 32bit_api 0 00000080:00200000:2.0:1787826319.427280:0:161485:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.427287:0:161485:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f-5, flags=0 00000080:00200000:2.0:1787826319.427292:0:161485:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 3 00000080:00200000:2.0:1787826319.427297:0:161485:0:(namei.c:2418:ll_unlink()) VFS Op:name=f-5, dir=[0x240001f59:0x1:0x0](ffff9181d0801988) 00010000:00010000:2.0:1787826319.427324:0:161485:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843e00/0xbef42c0221d6f30e lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00cee1f2 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00010000:00010000:2.0:1787826319.427403:0:161485:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481200/0xbef42c0221d6f289 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x878a5eea00cedfad expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826319.427415:0:161485:0:(vvp_object.c:117:vvp_conf_set()) [0x240001f59:0x7:0x0]: losing layout lock 00010000:00010000:2.0:1787826319.427448:0:161485:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481200/0xbef42c0221d6f289 lrc: 1/0,0 mode: --/PR res: [0x240001f59:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cedfad expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826319.427456:0:161485:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843e00/0xbef42c0221d6f30e lrc: 1/0,0 mode: --/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cee1f2 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00010000:00010000:2.0:1787826319.427462:0:161485:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:2.0:1787826319.427465:0:161485:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481200/0xbef42c0221d6f289 lrc: 0/0,0 mode: --/PR res: [0x240001f59:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cedfad expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826319.427481:0:161485:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843e00/0xbef42c0221d6f30e lrc: 0/0,0 mode: --/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cee1f2 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00000020:00100000:2.0:1787826319.427497:0:161485:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787826319.427510:0:161485:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c49e5f80 x1874663043019264/t0(0) o36->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826319.427531:0:161485:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c49e5f80 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043019264:192.168.203.147@tcp:36:rm.0 00000100:00100000:2.0:1787826319.427624:0:161485:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5903780 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.440142:0:161485:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c49e5f80 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043019264:192.168.203.147@tcp:36:rm.0 00000080:00200000:2.0:1787826319.440216:0:161485:0:(vvp_io.c:1826:vvp_io_init()) [0x240001f59:0x7:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826319.440230:0:161485:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826319.440257:0:161485:0:(vvp_io.c:304:vvp_io_fini()) [0x240001f59:0x7:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826319.440265:0:161485:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f59:0x7:0x0](ffff9181d080ec08) 00000080:00200000:2.0:1787826319.440351:0:161485:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.440362:0:161485:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f-0, flags=0 00000080:00200000:2.0:1787826319.440368:0:161485:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 3 00000080:00200000:2.0:1787826319.440373:0:161485:0:(namei.c:2418:ll_unlink()) VFS Op:name=f-0, dir=[0x240001f59:0x1:0x0](ffff9181d0801988) 00010000:00010000:2.0:1787826319.440413:0:161485:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7000/0xbef42c0221d6f266 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x878a5eea00cedefe expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826319.440433:0:161485:0:(vvp_object.c:117:vvp_conf_set()) [0x240001f59:0x2:0x0]: losing layout lock 00010000:00010000:2.0:1787826319.440476:0:161485:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7000/0xbef42c0221d6f266 lrc: 1/0,0 mode: --/PR res: [0x240001f59:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cedefe expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826319.440484:0:161485:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826319.440487:0:161485:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7000/0xbef42c0221d6f266 lrc: 0/0,0 mode: --/PR res: [0x240001f59:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cedefe expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000020:00100000:2.0:1787826319.440510:0:161485:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787826319.440518:0:161485:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c49e7b80 x1874663043019520/t0(0) o36->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826319.440544:0:161485:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c49e7b80 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043019520:192.168.203.147@tcp:36:rm.0 00000100:00100000:2.0:1787826319.440623:0:161485:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5903780 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.448294:0:161485:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c49e7b80 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043019520:192.168.203.147@tcp:36:rm.0 00000080:00200000:2.0:1787826319.448355:0:161485:0:(vvp_io.c:1826:vvp_io_init()) [0x240001f59:0x2:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826319.448369:0:161485:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826319.448396:0:161485:0:(vvp_io.c:304:vvp_io_fini()) [0x240001f59:0x2:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826319.448404:0:161485:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f59:0x2:0x0](ffff9181d0805b88) 00000080:00200000:2.0:1787826319.448487:0:161485:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.448499:0:161485:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f-6, flags=0 00000080:00200000:2.0:1787826319.448504:0:161485:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 3 00000080:00200000:2.0:1787826319.448509:0:161485:0:(namei.c:2418:ll_unlink()) VFS Op:name=f-6, dir=[0x240001f59:0x1:0x0](ffff9181d0801988) 00010000:00010000:2.0:1787826319.448552:0:161485:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f290 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x878a5eea00cedfd0 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826319.448574:0:161485:0:(vvp_object.c:117:vvp_conf_set()) [0x240001f59:0x8:0x0]: losing layout lock 00010000:00010000:2.0:1787826319.448620:0:161485:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f290 lrc: 1/0,0 mode: --/PR res: [0x240001f59:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cedfd0 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826319.448641:0:161485:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826319.448644:0:161485:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f290 lrc: 0/0,0 mode: --/PR res: [0x240001f59:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cedfd0 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000020:00100000:2.0:1787826319.448678:0:161485:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787826319.448686:0:161485:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c49e4700 x1874663043019776/t0(0) o36->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826319.448709:0:161485:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c49e4700 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043019776:192.168.203.147@tcp:36:rm.0 00000100:00100000:2.0:1787826319.448950:0:161485:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5903780 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.456085:0:161485:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c49e4700 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043019776:192.168.203.147@tcp:36:rm.0 00000080:00200000:2.0:1787826319.456152:0:161485:0:(vvp_io.c:1826:vvp_io_init()) [0x240001f59:0x8:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826319.456166:0:161485:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826319.456194:0:161485:0:(vvp_io.c:304:vvp_io_fini()) [0x240001f59:0x8:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826319.456204:0:161485:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f59:0x8:0x0](ffff9181d08080c8) 00000080:00200000:2.0:1787826319.456283:0:161485:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.456294:0:161485:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f-14, flags=0 00000080:00200000:2.0:1787826319.456299:0:161485:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 3 00000080:00200000:2.0:1787826319.456304:0:161485:0:(namei.c:2418:ll_unlink()) VFS Op:name=f-14, dir=[0x240001f59:0x1:0x0](ffff9181d0801988) 00010000:00010000:2.0:1787826319.456343:0:161485:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6000/0xbef42c0221d6f2c8 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x878a5eea00cee0e8 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826319.456363:0:161485:0:(vvp_object.c:117:vvp_conf_set()) [0x240001f59:0x10:0x0]: losing layout lock 00010000:00010000:2.0:1787826319.456405:0:161485:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6000/0xbef42c0221d6f2c8 lrc: 1/0,0 mode: --/PR res: [0x240001f59:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cee0e8 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826319.456414:0:161485:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826319.456417:0:161485:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6000/0xbef42c0221d6f2c8 lrc: 0/0,0 mode: --/PR res: [0x240001f59:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cee0e8 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000020:00100000:2.0:1787826319.456440:0:161485:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787826319.456448:0:161485:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c49e5180 x1874663043020032/t0(0) o36->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826319.456471:0:161485:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c49e5180 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043020032:192.168.203.147@tcp:36:rm.0 00000100:00100000:2.0:1787826319.456550:0:161485:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5903780 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.471149:0:161485:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c49e5180 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043020032:192.168.203.147@tcp:36:rm.0 00000080:00200000:2.0:1787826319.471209:0:161485:0:(vvp_io.c:1826:vvp_io_init()) [0x240001f59:0x10:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826319.471221:0:161485:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826319.471247:0:161485:0:(vvp_io.c:304:vvp_io_fini()) [0x240001f59:0x10:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826319.471254:0:161485:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f59:0x10:0x0](ffff9181d0807448) 00000080:00200000:2.0:1787826319.471334:0:161485:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.471344:0:161485:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f-8, flags=0 00000080:00200000:2.0:1787826319.471349:0:161485:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 3 00000080:00200000:2.0:1787826319.471354:0:161485:0:(namei.c:2418:ll_unlink()) VFS Op:name=f-8, dir=[0x240001f59:0x1:0x0](ffff9181d0801988) 00010000:00010000:2.0:1787826319.471390:0:161485:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6800/0xbef42c0221d6f29e lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x878a5eea00cee016 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826319.471407:0:161485:0:(vvp_object.c:117:vvp_conf_set()) [0x240001f59:0xa:0x0]: losing layout lock 00010000:00010000:2.0:1787826319.471452:0:161485:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6800/0xbef42c0221d6f29e lrc: 1/0,0 mode: --/PR res: [0x240001f59:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cee016 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826319.471465:0:161485:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826319.471468:0:161485:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6800/0xbef42c0221d6f29e lrc: 0/0,0 mode: --/PR res: [0x240001f59:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cee016 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000020:00100000:2.0:1787826319.471493:0:161485:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787826319.471500:0:161485:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181d0fc3480 x1874663043020288/t0(0) o36->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826319.471522:0:161485:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc3480 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043020288:192.168.203.147@tcp:36:rm.0 00000100:00100000:2.0:1787826319.471624:0:161485:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5903300 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.478971:0:161485:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc3480 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043020288:192.168.203.147@tcp:36:rm.0 00000080:00200000:2.0:1787826319.479103:0:161485:0:(vvp_io.c:1826:vvp_io_init()) [0x240001f59:0xa:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826319.479116:0:161485:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826319.479143:0:161485:0:(vvp_io.c:304:vvp_io_fini()) [0x240001f59:0xa:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826319.479150:0:161485:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f59:0xa:0x0](ffff9181d0804b08) 00000080:00200000:2.0:1787826319.479234:0:161485:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.479246:0:161485:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f-4, flags=0 00000080:00200000:2.0:1787826319.479251:0:161485:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 3 00000080:00200000:2.0:1787826319.479256:0:161485:0:(namei.c:2418:ll_unlink()) VFS Op:name=f-4, dir=[0x240001f59:0x1:0x0](ffff9181d0801988) 00010000:00010000:2.0:1787826319.479298:0:161485:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f282 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x878a5eea00cedf8a expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826319.479318:0:161485:0:(vvp_object.c:117:vvp_conf_set()) [0x240001f59:0x6:0x0]: losing layout lock 00010000:00010000:2.0:1787826319.479366:0:161485:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f282 lrc: 1/0,0 mode: --/PR res: [0x240001f59:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cedf8a expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826319.479374:0:161485:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826319.479377:0:161485:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f282 lrc: 0/0,0 mode: --/PR res: [0x240001f59:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cedf8a expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000020:00100000:2.0:1787826319.479400:0:161485:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787826319.479408:0:161485:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181d0fc2d80 x1874663043020544/t0(0) o36->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826319.479430:0:161485:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc2d80 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043020544:192.168.203.147@tcp:36:rm.0 00000100:00100000:2.0:1787826319.479557:0:161485:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5903300 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.482780:0:161485:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc2d80 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043020544:192.168.203.147@tcp:36:rm.0 00000080:00200000:2.0:1787826319.482840:0:161485:0:(vvp_io.c:1826:vvp_io_init()) [0x240001f59:0x6:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826319.482852:0:161485:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826319.482878:0:161485:0:(vvp_io.c:304:vvp_io_fini()) [0x240001f59:0x6:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826319.482886:0:161485:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f59:0x6:0x0](ffff9181d0803248) 00000080:00200000:2.0:1787826319.482968:0:161485:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.482981:0:161485:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f-13, flags=0 00000080:00200000:2.0:1787826319.482987:0:161485:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 3 00000080:00200000:2.0:1787826319.482992:0:161485:0:(namei.c:2418:ll_unlink()) VFS Op:name=f-13, dir=[0x240001f59:0x1:0x0](ffff9181d0801988) 00010000:00010000:2.0:1787826319.483042:0:161485:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7c00/0xbef42c0221d6f2c1 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x878a5eea00cee0c5 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826319.483062:0:161485:0:(vvp_object.c:117:vvp_conf_set()) [0x240001f59:0xf:0x0]: losing layout lock 00010000:00010000:2.0:1787826319.483105:0:161485:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7c00/0xbef42c0221d6f2c1 lrc: 1/0,0 mode: --/PR res: [0x240001f59:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cee0c5 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826319.483109:0:161485:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826319.483111:0:161485:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7c00/0xbef42c0221d6f2c1 lrc: 0/0,0 mode: --/PR res: [0x240001f59:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cee0c5 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000020:00100000:2.0:1787826319.483129:0:161485:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787826319.483135:0:161485:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181d0fc3100 x1874663043020800/t0(0) o36->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826319.483151:0:161485:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc3100 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043020800:192.168.203.147@tcp:36:rm.0 00000100:00100000:2.0:1787826319.483212:0:161485:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5903300 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.491390:0:161485:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc3100 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043020800:192.168.203.147@tcp:36:rm.0 00000080:00200000:2.0:1787826319.491624:0:161485:0:(vvp_io.c:1826:vvp_io_init()) [0x240001f59:0xf:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826319.491643:0:161485:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826319.491966:0:161485:0:(vvp_io.c:304:vvp_io_fini()) [0x240001f59:0xf:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826319.491975:0:161485:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f59:0xf:0x0](ffff9181d0805348) 00000080:00200000:2.0:1787826319.492056:0:161485:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.492067:0:161485:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f-15, flags=0 00000080:00200000:2.0:1787826319.492072:0:161485:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 3 00000080:00200000:2.0:1787826319.492077:0:161485:0:(namei.c:2418:ll_unlink()) VFS Op:name=f-15, dir=[0x240001f59:0x1:0x0](ffff9181d0801988) 00010000:00010000:2.0:1787826319.492119:0:161485:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7e00/0xbef42c0221d6f2cf lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x878a5eea00cee10b expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826319.492141:0:161485:0:(vvp_object.c:117:vvp_conf_set()) [0x240001f59:0x11:0x0]: losing layout lock 00010000:00010000:2.0:1787826319.492191:0:161485:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7e00/0xbef42c0221d6f2cf lrc: 1/0,0 mode: --/PR res: [0x240001f59:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cee10b expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826319.492198:0:161485:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826319.492202:0:161485:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7e00/0xbef42c0221d6f2cf lrc: 0/0,0 mode: --/PR res: [0x240001f59:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cee10b expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000020:00100000:2.0:1787826319.492225:0:161485:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787826319.492234:0:161485:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181d0fc0000 x1874663043021056/t0(0) o36->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826319.492256:0:161485:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc0000 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043021056:192.168.203.147@tcp:36:rm.0 00000100:00100000:2.0:1787826319.492397:0:161485:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c59033c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.497484:0:161485:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc0000 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043021056:192.168.203.147@tcp:36:rm.0 00000080:00200000:2.0:1787826319.497544:0:161485:0:(vvp_io.c:1826:vvp_io_init()) [0x240001f59:0x11:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826319.497557:0:161485:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826319.498097:0:161485:0:(vvp_io.c:304:vvp_io_fini()) [0x240001f59:0x11:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826319.498108:0:161485:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f59:0x11:0x0](ffff9181d08021c8) 00000080:00200000:2.0:1787826319.498209:0:161485:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.498222:0:161485:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f-11, flags=0 00000080:00200000:2.0:1787826319.498227:0:161485:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 3 00000080:00200000:2.0:1787826319.498232:0:161485:0:(namei.c:2418:ll_unlink()) VFS Op:name=f-11, dir=[0x240001f59:0x1:0x0](ffff9181d0801988) 00010000:00010000:2.0:1787826319.498281:0:161485:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7a00/0xbef42c0221d6f2b3 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x878a5eea00cee07f expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826319.498309:0:161485:0:(vvp_object.c:117:vvp_conf_set()) [0x240001f59:0xd:0x0]: losing layout lock 00010000:00010000:2.0:1787826319.498371:0:161485:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7a00/0xbef42c0221d6f2b3 lrc: 1/0,0 mode: --/PR res: [0x240001f59:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cee07f expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826319.498378:0:161485:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826319.498381:0:161485:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7a00/0xbef42c0221d6f2b3 lrc: 0/0,0 mode: --/PR res: [0x240001f59:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cee07f expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000020:00100000:2.0:1787826319.498403:0:161485:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787826319.498415:0:161485:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181d0fc2a00 x1874663043021312/t0(0) o36->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826319.498437:0:161485:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc2a00 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043021312:192.168.203.147@tcp:36:rm.0 00000100:00100000:2.0:1787826319.498647:0:161485:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5903180 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.505684:0:161485:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc2a00 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043021312:192.168.203.147@tcp:36:rm.0 00000080:00200000:2.0:1787826319.505943:0:161485:0:(vvp_io.c:1826:vvp_io_init()) [0x240001f59:0xd:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826319.505964:0:161485:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826319.505991:0:161485:0:(vvp_io.c:304:vvp_io_fini()) [0x240001f59:0xd:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826319.506083:0:161485:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f59:0xd:0x0](ffff9181d08000c8) 00000080:00200000:2.0:1787826319.506164:0:161485:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.506174:0:161485:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f-9, flags=0 00000080:00200000:2.0:1787826319.506179:0:161485:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 3 00000080:00200000:2.0:1787826319.506183:0:161485:0:(namei.c:2418:ll_unlink()) VFS Op:name=f-9, dir=[0x240001f59:0x1:0x0](ffff9181d0801988) 00010000:00010000:2.0:1787826319.506229:0:161485:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f2a5 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x878a5eea00cee039 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826319.506247:0:161485:0:(vvp_object.c:117:vvp_conf_set()) [0x240001f59:0xb:0x0]: losing layout lock 00010000:00010000:2.0:1787826319.506291:0:161485:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f2a5 lrc: 1/0,0 mode: --/PR res: [0x240001f59:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cee039 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826319.506299:0:161485:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826319.506302:0:161485:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7600/0xbef42c0221d6f2a5 lrc: 0/0,0 mode: --/PR res: [0x240001f59:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cee039 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000020:00100000:2.0:1787826319.506328:0:161485:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787826319.506341:0:161485:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181d0fc1180 x1874663043021568/t0(0) o36->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826319.506360:0:161485:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc1180 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043021568:192.168.203.147@tcp:36:rm.0 00000100:00100000:2.0:1787826319.506468:0:161485:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5903180 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.511400:0:161485:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc1180 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043021568:192.168.203.147@tcp:36:rm.0 00000080:00200000:2.0:1787826319.511493:0:161485:0:(vvp_io.c:1826:vvp_io_init()) [0x240001f59:0xb:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826319.511507:0:161485:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826319.511547:0:161485:0:(vvp_io.c:304:vvp_io_fini()) [0x240001f59:0xb:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826319.511555:0:161485:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f59:0xb:0x0](ffff9181d0800908) 00000080:00200000:2.0:1787826319.511669:0:161485:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.511680:0:161485:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f-3, flags=0 00000080:00200000:2.0:1787826319.511685:0:161485:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 3 00000080:00200000:2.0:1787826319.511712:0:161485:0:(namei.c:2418:ll_unlink()) VFS Op:name=f-3, dir=[0x240001f59:0x1:0x0](ffff9181d0801988) 00010000:00010000:2.0:1787826319.511804:0:161485:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f27b lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x878a5eea00cedf67 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826319.511841:0:161485:0:(vvp_object.c:117:vvp_conf_set()) [0x240001f59:0x5:0x0]: losing layout lock 00010000:00010000:2.0:1787826319.511885:0:161485:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f27b lrc: 1/0,0 mode: --/PR res: [0x240001f59:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cedf67 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826319.511897:0:161485:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826319.511900:0:161485:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f27b lrc: 0/0,0 mode: --/PR res: [0x240001f59:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cedf67 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000020:00100000:2.0:1787826319.511929:0:161485:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787826319.511944:0:161485:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181d0fc1880 x1874663043021824/t0(0) o36->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826319.511981:0:161485:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc1880 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043021824:192.168.203.147@tcp:36:rm.0 00000100:00100000:2.0:1787826319.512067:0:161485:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5903180 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.515820:0:161485:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc1880 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043021824:192.168.203.147@tcp:36:rm.0 00000080:00200000:2.0:1787826319.515908:0:161485:0:(vvp_io.c:1826:vvp_io_init()) [0x240001f59:0x5:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826319.515929:0:161485:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826319.515976:0:161485:0:(vvp_io.c:304:vvp_io_fini()) [0x240001f59:0x5:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826319.515986:0:161485:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f59:0x5:0x0](ffff9181d080d348) 00000080:00200000:2.0:1787826319.516163:0:161485:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.516186:0:161485:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f-19, flags=0 00000080:00200000:2.0:1787826319.516191:0:161485:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 3 00000080:00200000:2.0:1787826319.516204:0:161485:0:(namei.c:2418:ll_unlink()) VFS Op:name=f-19, dir=[0x240001f59:0x1:0x0](ffff9181d0801988) 00010000:00010000:2.0:1787826319.516250:0:161485:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842600/0xbef42c0221d6f2eb lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x15:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x878a5eea00cee197 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826319.516277:0:161485:0:(vvp_object.c:117:vvp_conf_set()) [0x240001f59:0x15:0x0]: losing layout lock 00010000:00010000:2.0:1787826319.516349:0:161485:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842600/0xbef42c0221d6f2eb lrc: 1/0,0 mode: --/PR res: [0x240001f59:0x15:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cee197 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826319.516357:0:161485:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826319.516361:0:161485:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842600/0xbef42c0221d6f2eb lrc: 0/0,0 mode: --/PR res: [0x240001f59:0x15:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cee197 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000020:00100000:2.0:1787826319.516383:0:161485:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787826319.516392:0:161485:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181d0fc1500 x1874663043022080/t0(0) o36->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826319.516435:0:161485:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc1500 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043022080:192.168.203.147@tcp:36:rm.0 00000100:00100000:2.0:1787826319.516587:0:161485:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5903180 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.521455:0:161485:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc1500 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043022080:192.168.203.147@tcp:36:rm.0 00000080:00200000:2.0:1787826319.521535:0:161485:0:(vvp_io.c:1826:vvp_io_init()) [0x240001f59:0x15:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826319.521548:0:161485:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826319.521573:0:161485:0:(vvp_io.c:304:vvp_io_fini()) [0x240001f59:0x15:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826319.521589:0:161485:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f59:0x15:0x0](ffff9181c830ec08) 00000080:00200000:2.0:1787826319.521811:0:161485:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.521829:0:161485:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f-18, flags=0 00000080:00200000:2.0:1787826319.521836:0:161485:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 3 00000080:00200000:2.0:1787826319.521840:0:161485:0:(namei.c:2418:ll_unlink()) VFS Op:name=f-18, dir=[0x240001f59:0x1:0x0](ffff9181d0801988) 00010000:00010000:2.0:1787826319.521890:0:161485:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f2e4 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x14:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x878a5eea00cee174 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826319.521909:0:161485:0:(vvp_object.c:117:vvp_conf_set()) [0x240001f59:0x14:0x0]: losing layout lock 00010000:00010000:2.0:1787826319.521985:0:161485:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f2e4 lrc: 1/0,0 mode: --/PR res: [0x240001f59:0x14:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cee174 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826319.521993:0:161485:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826319.521996:0:161485:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f2e4 lrc: 0/0,0 mode: --/PR res: [0x240001f59:0x14:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cee174 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000020:00100000:2.0:1787826319.522034:0:161485:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787826319.522042:0:161485:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181d0fc3800 x1874663043022336/t0(0) o36->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826319.522063:0:161485:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc3800 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043022336:192.168.203.147@tcp:36:rm.0 00000100:00100000:2.0:1787826319.522136:0:161485:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5903600 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.527476:0:161485:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc3800 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043022336:192.168.203.147@tcp:36:rm.0 00000080:00200000:2.0:1787826319.527549:0:161485:0:(vvp_io.c:1826:vvp_io_init()) [0x240001f59:0x14:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826319.527562:0:161485:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826319.527594:0:161485:0:(vvp_io.c:304:vvp_io_fini()) [0x240001f59:0x14:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826319.527602:0:161485:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f59:0x14:0x0](ffff9181c830aa08) 00000080:00200000:2.0:1787826319.527711:0:161485:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.527723:0:161485:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f-2, flags=0 00000080:00200000:2.0:1787826319.527729:0:161485:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 3 00000080:00200000:2.0:1787826319.527782:0:161485:0:(namei.c:2418:ll_unlink()) VFS Op:name=f-2, dir=[0x240001f59:0x1:0x0](ffff9181d0801988) 00010000:00010000:2.0:1787826319.527833:0:161485:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481600/0xbef42c0221d6f274 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x878a5eea00cedf44 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826319.527861:0:161485:0:(vvp_object.c:117:vvp_conf_set()) [0x240001f59:0x4:0x0]: losing layout lock 00010000:00010000:2.0:1787826319.527920:0:161485:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481600/0xbef42c0221d6f274 lrc: 1/0,0 mode: --/PR res: [0x240001f59:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cedf44 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826319.527929:0:161485:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826319.527932:0:161485:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481600/0xbef42c0221d6f274 lrc: 0/0,0 mode: --/PR res: [0x240001f59:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cedf44 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000020:00100000:2.0:1787826319.527973:0:161485:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787826319.527983:0:161485:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181d0fc2680 x1874663043022592/t0(0) o36->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826319.528019:0:161485:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc2680 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043022592:192.168.203.147@tcp:36:rm.0 00000100:00100000:2.0:1787826319.528128:0:161485:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5903c00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.533275:0:161485:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc2680 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043022592:192.168.203.147@tcp:36:rm.0 00000080:00200000:2.0:1787826319.533332:0:161485:0:(vvp_io.c:1826:vvp_io_init()) [0x240001f59:0x4:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826319.533345:0:161485:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826319.533369:0:161485:0:(vvp_io.c:304:vvp_io_fini()) [0x240001f59:0x4:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826319.533376:0:161485:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f59:0x4:0x0](ffff9181d080c2c8) 00000080:00200000:2.0:1787826319.533449:0:161485:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.533461:0:161485:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f-1, flags=0 00000080:00200000:2.0:1787826319.533467:0:161485:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 3 00000080:00200000:2.0:1787826319.533471:0:161485:0:(namei.c:2418:ll_unlink()) VFS Op:name=f-1, dir=[0x240001f59:0x1:0x0](ffff9181d0801988) 00010000:00010000:2.0:1787826319.533520:0:161485:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6c00/0xbef42c0221d6f26d lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x878a5eea00cedf21 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 1 00000080:00200000:2.0:1787826319.533539:0:161485:0:(vvp_object.c:117:vvp_conf_set()) [0x240001f59:0x3:0x0]: losing layout lock 00010000:00010000:2.0:1787826319.533588:0:161485:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6c00/0xbef42c0221d6f26d lrc: 1/0,0 mode: --/PR res: [0x240001f59:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cedf21 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 1 00010000:00010000:2.0:1787826319.533594:0:161485:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826319.533598:0:161485:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6c00/0xbef42c0221d6f26d lrc: 0/0,0 mode: --/PR res: [0x240001f59:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cedf21 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 1 00000020:00100000:2.0:1787826319.533623:0:161485:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787826319.533632:0:161485:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181d0fc1c00 x1874663043022848/t0(0) o36->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826319.533661:0:161485:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc1c00 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043022848:192.168.203.147@tcp:36:rm.0 00000100:00100000:2.0:1787826319.533830:0:161485:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5903c00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.540195:0:161485:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc1c00 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043022848:192.168.203.147@tcp:36:rm.0 00000080:00200000:2.0:1787826319.540261:0:161485:0:(vvp_io.c:1826:vvp_io_init()) [0x240001f59:0x3:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826319.540275:0:161485:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826319.540305:0:161485:0:(vvp_io.c:304:vvp_io_fini()) [0x240001f59:0x3:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826319.540313:0:161485:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f59:0x3:0x0](ffff9181d0803a88) 00000080:00200000:2.0:1787826319.540394:0:161485:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.540416:0:161485:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f-16, flags=0 00000080:00200000:2.0:1787826319.540421:0:161485:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 3 00000080:00200000:2.0:1787826319.540427:0:161485:0:(namei.c:2418:ll_unlink()) VFS Op:name=f-16, dir=[0x240001f59:0x1:0x0](ffff9181d0801988) 00010000:00010000:2.0:1787826319.540468:0:161485:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7800/0xbef42c0221d6f2d6 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x878a5eea00cee12e expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826319.540489:0:161485:0:(vvp_object.c:117:vvp_conf_set()) [0x240001f59:0x12:0x0]: losing layout lock 00010000:00010000:2.0:1787826319.540557:0:161485:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7800/0xbef42c0221d6f2d6 lrc: 1/0,0 mode: --/PR res: [0x240001f59:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cee12e expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826319.540565:0:161485:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826319.540568:0:161485:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7800/0xbef42c0221d6f2d6 lrc: 0/0,0 mode: --/PR res: [0x240001f59:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cee12e expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000020:00100000:2.0:1787826319.540635:0:161485:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787826319.540644:0:161485:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181d0fc0380 x1874663043023104/t0(0) o36->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826319.540665:0:161485:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc0380 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043023104:192.168.203.147@tcp:36:rm.0 00000100:00100000:2.0:1787826319.540794:0:161485:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5903c00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.546526:0:161485:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc0380 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043023104:192.168.203.147@tcp:36:rm.0 00000080:00200000:2.0:1787826319.546614:0:161485:0:(vvp_io.c:1826:vvp_io_init()) [0x240001f59:0x12:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826319.546626:0:161485:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826319.546659:0:161485:0:(vvp_io.c:304:vvp_io_fini()) [0x240001f59:0x12:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826319.546667:0:161485:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f59:0x12:0x0](ffff9181d08063c8) 00000080:00200000:2.0:1787826319.547019:0:161485:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.547033:0:161485:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f-7, flags=0 00000080:00200000:2.0:1787826319.547038:0:161485:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 3 00000080:00200000:2.0:1787826319.547042:0:161485:0:(namei.c:2418:ll_unlink()) VFS Op:name=f-7, dir=[0x240001f59:0x1:0x0](ffff9181d0801988) 00010000:00010000:2.0:1787826319.547081:0:161485:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7200/0xbef42c0221d6f297 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x878a5eea00cedff3 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826319.547099:0:161485:0:(vvp_object.c:117:vvp_conf_set()) [0x240001f59:0x9:0x0]: losing layout lock 00010000:00010000:2.0:1787826319.547140:0:161485:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7200/0xbef42c0221d6f297 lrc: 1/0,0 mode: --/PR res: [0x240001f59:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cedff3 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826319.547148:0:161485:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826319.547151:0:161485:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f7200/0xbef42c0221d6f297 lrc: 0/0,0 mode: --/PR res: [0x240001f59:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cedff3 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000020:00100000:2.0:1787826319.547173:0:161485:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787826319.547183:0:161485:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181d0fc2300 x1874663043023360/t0(0) o36->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826319.547205:0:161485:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc2300 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043023360:192.168.203.147@tcp:36:rm.0 00000100:00100000:2.0:1787826319.547280:0:161485:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5903000 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.552367:0:161485:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc2300 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043023360:192.168.203.147@tcp:36:rm.0 00000080:00200000:2.0:1787826319.552436:0:161485:0:(vvp_io.c:1826:vvp_io_init()) [0x240001f59:0x9:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826319.552450:0:161485:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826319.552489:0:161485:0:(vvp_io.c:304:vvp_io_fini()) [0x240001f59:0x9:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826319.552497:0:161485:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f59:0x9:0x0](ffff9181d0802a08) 00000080:00200000:2.0:1787826319.552692:0:161485:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.552704:0:161485:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f-12, flags=0 00000080:00200000:2.0:1787826319.552709:0:161485:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 3 00000080:00200000:2.0:1787826319.552720:0:161485:0:(namei.c:2418:ll_unlink()) VFS Op:name=f-12, dir=[0x240001f59:0x1:0x0](ffff9181d0801988) 00010000:00010000:2.0:1787826319.552798:0:161485:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6a00/0xbef42c0221d6f2ba lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x878a5eea00cee0a2 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826319.552827:0:161485:0:(vvp_object.c:117:vvp_conf_set()) [0x240001f59:0xe:0x0]: losing layout lock 00010000:00010000:2.0:1787826319.552883:0:161485:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6a00/0xbef42c0221d6f2ba lrc: 1/0,0 mode: --/PR res: [0x240001f59:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cee0a2 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826319.552892:0:161485:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826319.552895:0:161485:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6a00/0xbef42c0221d6f2ba lrc: 0/0,0 mode: --/PR res: [0x240001f59:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cee0a2 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000020:00100000:2.0:1787826319.552925:0:161485:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787826319.552941:0:161485:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181d0fc0700 x1874663043023616/t0(0) o36->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826319.552975:0:161485:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc0700 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043023616:192.168.203.147@tcp:36:rm.0 00000100:00100000:2.0:1787826319.553053:0:161485:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5903600 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.557784:0:161485:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc0700 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043023616:192.168.203.147@tcp:36:rm.0 00000080:00200000:2.0:1787826319.557844:0:161485:0:(vvp_io.c:1826:vvp_io_init()) [0x240001f59:0xe:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826319.557857:0:161485:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826319.557894:0:161485:0:(vvp_io.c:304:vvp_io_fini()) [0x240001f59:0xe:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826319.557911:0:161485:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f59:0xe:0x0](ffff9181d0806c08) 00000080:00200000:2.0:1787826319.558021:0:161485:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.558033:0:161485:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f-10, flags=0 00000080:00200000:2.0:1787826319.558038:0:161485:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 3 00000080:00200000:2.0:1787826319.558043:0:161485:0:(namei.c:2418:ll_unlink()) VFS Op:name=f-10, dir=[0x240001f59:0x1:0x0](ffff9181d0801988) 00010000:00010000:2.0:1787826319.558081:0:161485:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6e00/0xbef42c0221d6f2ac lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x878a5eea00cee05c expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826319.558109:0:161485:0:(vvp_object.c:117:vvp_conf_set()) [0x240001f59:0xc:0x0]: losing layout lock 00010000:00010000:2.0:1787826319.558164:0:161485:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6e00/0xbef42c0221d6f2ac lrc: 1/0,0 mode: --/PR res: [0x240001f59:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cee05c expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826319.558172:0:161485:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826319.558175:0:161485:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6e00/0xbef42c0221d6f2ac lrc: 0/0,0 mode: --/PR res: [0x240001f59:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cee05c expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000020:00100000:2.0:1787826319.558205:0:161485:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787826319.558214:0:161485:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181d0fc0e00 x1874663043023872/t0(0) o36->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826319.558243:0:161485:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc0e00 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043023872:192.168.203.147@tcp:36:rm.0 00000100:00100000:2.0:1787826319.558341:0:161485:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5903600 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.563412:0:161485:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc0e00 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043023872:192.168.203.147@tcp:36:rm.0 00000080:00200000:2.0:1787826319.563479:0:161485:0:(vvp_io.c:1826:vvp_io_init()) [0x240001f59:0xc:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826319.563496:0:161485:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826319.563521:0:161485:0:(vvp_io.c:304:vvp_io_fini()) [0x240001f59:0xc:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826319.563534:0:161485:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f59:0xc:0x0](ffff9181d0801148) 00000080:00200000:2.0:1787826319.563630:0:161485:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.563642:0:161485:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f-17, flags=0 00000080:00200000:2.0:1787826319.563655:0:161485:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988), inode mode 41ed mask 3 00000080:00200000:2.0:1787826319.563660:0:161485:0:(namei.c:2418:ll_unlink()) VFS Op:name=f-17, dir=[0x240001f59:0x1:0x0](ffff9181d0801988) 00010000:00010000:2.0:1787826319.563718:0:161485:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6200/0xbef42c0221d6f2dd lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x878a5eea00cee151 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826319.563820:0:161485:0:(vvp_object.c:117:vvp_conf_set()) [0x240001f59:0x13:0x0]: losing layout lock 00010000:00010000:2.0:1787826319.563873:0:161485:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6200/0xbef42c0221d6f2dd lrc: 1/0,0 mode: --/PR res: [0x240001f59:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cee151 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826319.563888:0:161485:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826319.563891:0:161485:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e06f6200/0xbef42c0221d6f2dd lrc: 0/0,0 mode: --/PR res: [0x240001f59:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cee151 expref: -99 pid: 161476 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000020:00100000:2.0:1787826319.563924:0:161485:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787826319.563932:0:161485:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181d0fc3b80 x1874663043024128/t0(0) o36->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826319.563967:0:161485:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc3b80 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043024128:192.168.203.147@tcp:36:rm.0 00000100:00100000:2.0:1787826319.564250:0:161485:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5903600 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.569065:0:161485:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc3b80 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043024128:192.168.203.147@tcp:36:rm.0 00000080:00200000:2.0:1787826319.569136:0:161485:0:(vvp_io.c:1826:vvp_io_init()) [0x240001f59:0x13:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826319.569149:0:161485:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826319.569188:0:161485:0:(vvp_io.c:304:vvp_io_fini()) [0x240001f59:0x13:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826319.569196:0:161485:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f59:0x13:0x0](ffff9181d08042c8) 00000080:00200008:2.0:1787826319.569290:0:161485:0:(file.c:479:ll_file_release()) START file remote_dir:[0x240001f59:0x1:0x0](ffff9181d0801988), flags 704000 00010000:00010000:2.0:1787826319.569317:0:161485:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843a00/0xbef42c0221d6f315 lrc: 2/0,0 mode: CR/CR res: [0x240001f59:0x1:0x0].0x0 bits 0x5/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee200 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:2.0:1787826319.569355:0:161485:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x2:0x0](ffff9181c83f3248), inode mode 41ed mask 1 00000080:00200000:2.0:1787826319.569371:0:161485:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=remote_dir, dir=[0x200027cd1:0x2:0x0](ffff9181c83f3248), flags=0 00000080:00200000:2.0:1787826319.569376:0:161485:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=remote_dir, dir=[0x200027cd1:0x2:0x0](ffff9181c83f3248), intent=getattr 00000002:00010000:2.0:1787826319.569404:0:161485:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: remote_dir,[0x0:0x0:0x0]) in obj [0x200027cd1:0x2:0x0], intent: getattr flags 01 00010000:00010000:2.0:1787826319.569449:0:161485:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f31c lrc: 3/1,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 161485 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.569457:0:161485:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f31c lrc: 3/1,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 161485 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.569465:0:161485:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f31c lrc: 3/1,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 161485 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826319.569483:0:161485:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc0a80 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043024256:192.168.203.147@tcp:101:rm.0 00000100:00100000:2.0:1787826319.569553:0:161485:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5903540 going to sleep for 11 seconds 00000100:00100000:2.0:1787826319.573533:0:161485:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc0a80 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043024256:192.168.203.147@tcp:101:rm.0 00010000:00010000:2.0:1787826319.573562:0:161485:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f31c lrc: 4/1,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee453 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.573590:0:161485:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f31c lrc: 4/1,0 mode: --/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee453 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.573599:0:161485:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f31c lrc: 4/1,0 mode: --/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee453 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.573611:0:161485:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f31c lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee453 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.573627:0:161485:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f31c lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee453 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.573639:0:161485:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f31c lrc: 5/2,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee453 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.573648:0:161485:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f31c lrc: 5/2,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee453 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.573656:0:161485:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f31c lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee453 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787826319.573668:0:161485:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181d0fc0a80 x1874663043024256/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 584/584 e 0 to 0 dl 1787826335 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787826319.573680:0:161485:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f31c lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee453 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.573692:0:161485:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7400/0xbef42c0221d6f258 lrc: 2/1,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedec6 expref: -99 pid: 161472 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826319.573704:0:161485:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7400/0xbef42c0221d6f258 lrc: 2/1,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedec6 expref: -99 pid: 161472 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826319.573710:0:161485:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f31c lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee453 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.573716:0:161485:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f31c lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000400000000 nid: local remote: 0x878a5eea00cee453 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.573722:0:161485:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f31c lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000400000000 nid: local remote: 0x878a5eea00cee453 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.573765:0:161485:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000002:00010000:2.0:1787826319.573783:0:161485:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x0:0x0:0x0]) in obj [0x240001f59:0x1:0x0], intent: getattr flags 01 00010000:00010000:2.0:1787826319.573823:0:161485:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842600/0xbef42c0221d6f323 lrc: 3/1,0 mode: --/CR res: [0x240001f59:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 161485 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.573831:0:161485:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842600/0xbef42c0221d6f323 lrc: 3/1,0 mode: --/CR res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 161485 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.573839:0:161485:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842600/0xbef42c0221d6f323 lrc: 3/1,0 mode: --/CR res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 161485 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826319.573858:0:161485:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc1f80 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043024384:192.168.203.147@tcp:101:rm.0 00000100:00100000:2.0:1787826319.573928:0:161485:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5903600 going to sleep for 11 seconds 00010000:00010000:1.0:1787826319.574044:0:159733:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826319.574052:0:159733:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f31c lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x400000000 nid: local remote: 0x878a5eea00cee453 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826319.574061:0:159733:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181d1842a00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:1.0:1787826319.574065:0:159733:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f31c lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT gid 0 flags: 0x400000000 nid: local remote: 0x878a5eea00cee453 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826319.574073:0:159733:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f31c lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT gid 0 flags: 0x400000000 nid: local remote: 0x878a5eea00cee453 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826319.574081:0:159733:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f31c lrc: 3/0,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00cee453 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826319.574100:0:159733:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7400/0xbef42c0221d6f258 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedec6 expref: -99 pid: 161472 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826319.574130:0:159733:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f31c lrc: 2/0,0 mode: --/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cee453 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826319.574137:0:159733:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826319.574158:0:159733:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f31c lrc: 1/0,0 mode: --/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cee453 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826319.574165:0:159733:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f31c lrc: 0/0,0 mode: --/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cee453 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826319.574177:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826319.574179:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826319.574180:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826319.574192:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663043024512:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1787826319.578447:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663043024512:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:2.0:1787826319.582365:0:161485:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc1f80 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043024384:192.168.203.147@tcp:101:rm.0 00010000:00010000:2.0:1787826319.582390:0:161485:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842600/0xbef42c0221d6f323 lrc: 4/1,0 mode: --/CR res: [0x240001f59:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee461 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.582401:0:161485:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842600/0xbef42c0221d6f323 lrc: 4/1,0 mode: --/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee461 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.582412:0:161485:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842600/0xbef42c0221d6f323 lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee461 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.582423:0:161485:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842600/0xbef42c0221d6f323 lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee461 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.582435:0:161485:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842600/0xbef42c0221d6f323 lrc: 5/2,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee461 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.582443:0:161485:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842600/0xbef42c0221d6f323 lrc: 5/2,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee461 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.582449:0:161485:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842600/0xbef42c0221d6f323 lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee461 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787826319.582459:0:161485:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181d0fc1f80 x1874663043024384/t0(0) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826335 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787826319.582471:0:161485:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842600/0xbef42c0221d6f323 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee461 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.582483:0:161485:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c77f3000 type 13 mode 4 res 9663684441/1 (0 0) 00000080:00200000:2.0:1787826319.582504:0:161485:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200027cd1:0x2:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1787826319.582531:0:161485:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x240001f59:0x1:0x0], updating i_size 4096 00000080:00200000:2.0:1787826319.582534:0:161485:0:(namei.c:147:ll_iget()) got inode: [0x240001f59:0x1:0x0](ffff9181d0801988): rc = 0 00000080:00010000:2.0:1787826319.582541:0:161485:0:(llite_internal.h:1981:ll_set_lock_data()) setting l_data to inode [0x240001f59:0x1:0x0](ffff9181d0801988) for remote lock 0xbef42c0221d6f323 00000080:00010000:2.0:1787826319.582546:0:161485:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001f59:0x1:0x0](ffff9181d0801988) for lock 0xbef42c0221d6f258 00000080:00010000:2.0:1787826319.582557:0:161485:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001f59:0x1:0x0](ffff9181d0801988) 00000080:00010000:2.0:1787826319.582561:0:161485:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f258 from it ffffb97501c3be00 00010000:00010000:2.0:1787826319.582565:0:161485:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7400/0xbef42c0221d6f258 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cedec6 expref: -99 pid: 161472 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826319.582574:0:161485:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7400/0xbef42c0221d6f258 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cedec6 expref: -99 pid: 161472 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00010000:2.0:1787826319.582598:0:161485:0:(dcache.c:164:ll_intent_drop_lock()) releasing remote lock with cookie 0xbef42c0221d6f323 from it ffffb97501c3be00 00010000:00010000:2.0:1787826319.582602:0:161485:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842600/0xbef42c0221d6f323 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee461 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826319.582608:0:161485:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842600/0xbef42c0221d6f323 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee461 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826319.582626:0:161485:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x2:0x0](ffff9181c83f3248), inode mode 41ed mask 3 00000080:00200000:2.0:1787826319.582634:0:161485:0:(namei.c:2328:ll_rmdir()) VFS Op:name=remote_dir, dir=[0x200027cd1:0x2:0x0](ffff9181c83f3248) 00010000:00010000:2.0:1787826319.582659:0:161485:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7400/0xbef42c0221d6f258 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00cedec6 expref: -99 pid: 161472 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787826319.582710:0:161485:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7400/0xbef42c0221d6f258 lrc: 1/0,0 mode: --/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cedec6 expref: -99 pid: 161472 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787826319.582717:0:161485:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826319.582778:0:161485:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f7400/0xbef42c0221d6f258 lrc: 0/0,0 mode: --/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cedec6 expref: -99 pid: 161472 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787826319.582803:0:161485:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842600/0xbef42c0221d6f323 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00cee461 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826319.582819:0:161485:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843a00/0xbef42c0221d6f315 lrc: 2/0,0 mode: CR/CR res: [0x240001f59:0x1:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00cee200 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000002:00100000:2.0:1787826319.582850:0:161485:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181c49e6d80 x1874663043018880/t17179870032(17179870032) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 0 dl 1787826335 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'rm.0' uid:0 gid:0 projid:0 00000020:00100000:2.0:1787826319.582866:0:161485:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1787826319.582873:0:161485:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181d0fc0a80 x1874663043024896/t0(0) o35->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:2.0:1787826319.582891:0:161485:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc0a80 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043024896:192.168.203.147@tcp:35:rm.0 00000100:00100000:2.0:1787826319.582966:0:161485:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5903600 going to sleep for 11 seconds 00000100:00100000:3.0:1787826319.583032:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663043024640:192.168.203.147@tcp:103:rm.0 00000100:00100000:2.0:1787826319.598207:0:161485:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc0a80 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043024896:192.168.203.147@tcp:35:rm.0 00000002:00100000:2.0:1787826319.598246:0:161485:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181c49e6d80 x1874663043018880/t17179870032(17179870032) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 0 dl 1787826335 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'rm.0' uid:0 gid:0 projid:0 00010000:00010000:2.0:1787826319.598319:0:161485:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842600/0xbef42c0221d6f323 lrc: 1/0,0 mode: --/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cee461 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826319.598329:0:161485:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843a00/0xbef42c0221d6f315 lrc: 1/0,0 mode: --/CR res: [0x240001f59:0x1:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cee200 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1787826319.598335:0:161485:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:2.0:1787826319.598338:0:161485:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842600/0xbef42c0221d6f323 lrc: 0/0,0 mode: --/PR res: [0x240001f59:0x1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cee461 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826319.598350:0:161485:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843a00/0xbef42c0221d6f315 lrc: 0/0,0 mode: --/CR res: [0x240001f59:0x1:0x0].0x0 bits 0x5/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cee200 expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000020:00100000:2.0:1787826319.598370:0:161485:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787826319.598377:0:161485:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181d0fc0a80 x1874663043025152/t0(0) o36->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 496/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826319.598398:0:161485:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc0a80 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043025152:192.168.203.147@tcp:36:rm.0 00000100:00100000:2.0:1787826319.598475:0:161485:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5903600 going to sleep for 11 seconds 00000100:00100000:3.0:1787826319.604501:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663043024640:192.168.203.147@tcp:103:rm.0 00000100:00100000:1.0:1787826319.605834:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826319.605998:0:159723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049389184 00000100:00100000:0.0:1787826319.606008:0:159723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049389184 00000100:00100000:0.0:1787826319.606072:0:159723:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 3 00000100:00100000:0.0:1787826319.606083:0:159723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c35b3800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:c0d7758a-4501-4a89-8ec1-407fef4f0c7f+5:177390:x1874663049389184:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826319.606101:0:159723:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4e00/0xbef42c0221d6f251 lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cedeaa expref: -99 pid: 161472 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00010000:00010000:0.0:1787826319.606252:0:159723:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826319.606260:0:159723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c35b3800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:c0d7758a-4501-4a89-8ec1-407fef4f0c7f+5:177390:x1874663049389184:12345-192.168.203.147@tcp:104: Request processed in 179us (433us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826319.606267:0:159723:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 3 00010000:00010000:3.0:1787826319.606334:0:159732:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826319.606341:0:159732:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4e00/0xbef42c0221d6f251 lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cedeaa expref: -99 pid: 161472 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00010000:00010000:3.0:1787826319.606350:0:159732:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4e00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:3.0:1787826319.606354:0:159732:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4e00/0xbef42c0221d6f251 lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x13 rrc: 3 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00cedeaa expref: -99 pid: 161472 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00010000:00010000:3.0:1787826319.606363:0:159732:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4e00/0xbef42c0221d6f251 lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x13 rrc: 3 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00cedeaa expref: -99 pid: 161472 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00010000:00010000:3.0:1787826319.606375:0:159732:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4e00/0xbef42c0221d6f251 lrc: 3/0,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x13 rrc: 3 type: IBT gid 0 flags: 0x408400000000 nid: local remote: 0x878a5eea00cedeaa expref: -99 pid: 161472 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00010000:00010000:3.0:1787826319.606399:0:159732:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481800/0xbef42c0221d6f307 lrc: 2/0,0 mode: CR/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee1dd expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826319.606443:0:159732:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4e00/0xbef42c0221d6f251 lrc: 2/0,0 mode: --/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x13 rrc: 3 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cedeaa expref: -99 pid: 161472 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00010000:00010000:3.0:1787826319.606450:0:159732:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:2.0:1787826319.606503:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663043025280:192.168.203.147@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826319.609164:0:159732:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4e00/0xbef42c0221d6f251 lrc: 1/0,0 mode: --/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x13 rrc: 3 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cedeaa expref: -99 pid: 161472 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00010000:00010000:3.0:1787826319.609185:0:159732:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4e00/0xbef42c0221d6f251 lrc: 0/0,0 mode: --/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x13 rrc: 3 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cedeaa expref: -99 pid: 161472 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00010000:00010000:3.0:1787826319.609205:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826319.609207:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826319.609210:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826319.612507:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663043025280:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:2.0:1787826319.627496:0:161485:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc0a80 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043025152:192.168.203.147@tcp:36:rm.0 00000080:00200000:2.0:1787826319.629371:0:161485:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f59:0x1:0x0](ffff9181d0801988) 00000080:00200008:0.0:1787826319.630362:0:161485:0:(file.c:479:ll_file_release()) START file d80a.replay-single:[0x200027cd1:0x2:0x0](ffff9181c83f3248), flags 704000 00010000:00010000:0.0:1787826319.630413:0:161485:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481800/0xbef42c0221d6f307 lrc: 2/0,0 mode: CR/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x5/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee1dd expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1787826319.630454:0:161485:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:0.0:1787826319.630473:0:161485:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826319.630483:0:161485:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 34 lru_type: 1 00010000:00010000:0.0:1787826319.630499:0:161485:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 34 lru_type: 1 00000080:00200000:0.0:1787826319.630506:0:161485:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826319.630513:0:161485:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:0.0:1787826319.630517:0:161485:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f24a 00000080:00010000:0.0:1787826319.630525:0:161485:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f24a from it ffffb97501c3bc10 00010000:00010000:0.0:1787826319.630530:0:161485:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 34 lru_type: 1 00010000:00010000:0.0:1787826319.630551:0:161485:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 35 lru_type: 1 00000080:00200000:0.0:1787826319.630567:0:161485:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826319.630584:0:161485:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=d80a.replay-single, dir=[0x200000007:0x1:0x0](ffff9181d080cb08), flags=0 00000080:00200000:0.0:1787826319.630589:0:161485:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=d80a.replay-single, dir=[0x200000007:0x1:0x0](ffff9181d080cb08), intent=getattr 00000002:00010000:0.0:1787826319.630601:0:161485:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: d80a.replay-single,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 01 00010000:00010000:0.0:1787826319.630649:0:161485:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513800/0xbef42c0221d6f32a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 161485 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.630656:0:161485:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513800/0xbef42c0221d6f32a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 161485 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.630665:0:161485:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513800/0xbef42c0221d6f32a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 161485 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826319.630690:0:161485:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043025408:192.168.203.147@tcp:101:rm.0 00000100:00100000:0.0:1787826319.632088:0:161485:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c52560c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826319.637919:0:161485:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043025408:192.168.203.147@tcp:101:rm.0 00010000:00010000:0.0:1787826319.637945:0:161485:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513800/0xbef42c0221d6f32a lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee4ae expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.637974:0:161485:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513800/0xbef42c0221d6f32a lrc: 4/1,0 mode: --/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee4ae expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.637983:0:161485:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513800/0xbef42c0221d6f32a lrc: 4/1,0 mode: --/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee4ae expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.637994:0:161485:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513800/0xbef42c0221d6f32a lrc: 4/1,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee4ae expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.638006:0:161485:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513800/0xbef42c0221d6f32a lrc: 4/1,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee4ae expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.638018:0:161485:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513800/0xbef42c0221d6f32a lrc: 5/2,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee4ae expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.638027:0:161485:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513800/0xbef42c0221d6f32a lrc: 5/2,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee4ae expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.638034:0:161485:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513800/0xbef42c0221d6f32a lrc: 4/1,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee4ae expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826319.638046:0:161485:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c55a5f80 x1874663043025408/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 592/688 e 0 to 0 dl 1787826335 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787826319.638060:0:161485:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513800/0xbef42c0221d6f32a lrc: 3/1,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee4ae expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826319.638070:0:161485:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d1a77c00 type 13 mode 4 res 8590097617/2 (0 0) 00000080:00200000:0.0:1787826319.638076:0:161485:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1787826319.640062:0:161485:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200027cd1:0x2:0x0], updating i_size 4096 00000080:00200000:1.0:1787826319.640074:0:161485:0:(namei.c:147:ll_iget()) got inode: [0x200027cd1:0x2:0x0](ffff9181c83f3248): rc = 0 00000080:00010000:1.0:1787826319.640087:0:161485:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200027cd1:0x2:0x0](ffff9181c83f3248) for lock 0xbef42c0221d6f32a 00000080:00010000:1.0:1787826319.640099:0:161485:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200027cd1:0x2:0x0](ffff9181c83f3248) 00000080:00010000:1.0:1787826319.640102:0:161485:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f32a from it ffffb97501c3be00 00010000:00010000:1.0:1787826319.640108:0:161485:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513800/0xbef42c0221d6f32a lrc: 3/1,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee4ae expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826319.640202:0:161485:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513800/0xbef42c0221d6f32a lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee4ae expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826319.640243:0:161485:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:1.0:1787826319.640261:0:161485:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826319.640274:0:161485:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 35 lru_type: 1 00010000:00010000:1.0:1787826319.640285:0:161485:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 35 lru_type: 1 00000080:00200000:1.0:1787826319.640362:0:161485:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826319.640369:0:161485:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:1.0:1787826319.640423:0:161485:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f24a 00000080:00010000:1.0:1787826319.640428:0:161485:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f24a from it ffffb97501c3bd88 00010000:00010000:1.0:1787826319.640431:0:161485:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 35 lru_type: 1 00010000:00010000:1.0:1787826319.640440:0:161485:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 36 lru_type: 1 00000080:00200000:1.0:1787826319.640449:0:161485:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 3 00000080:00200000:1.0:1787826319.640456:0:161485:0:(namei.c:2328:ll_rmdir()) VFS Op:name=d80a.replay-single, dir=[0x200000007:0x1:0x0](ffff9181d080cb08) 00010000:00010000:1.0:1787826319.640477:0:161485:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 36 lru_type: 1 00010000:00010000:1.0:1787826319.640502:0:161485:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513800/0xbef42c0221d6f32a lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00cee4ae expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826319.640995:0:161485:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481800/0xbef42c0221d6f307 lrc: 2/0,0 mode: CR/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00cee1dd expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000002:00100000:1.0:1787826319.641231:0:161485:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181c5634e00 x1874663043018496/t352187318278(352187318278) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 0 dl 1787826335 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'rm.0' uid:0 gid:0 projid:0 00000020:00100000:1.0:1787826319.641242:0:161485:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:1.0:1787826319.641248:0:161485:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c35b1500 x1874663043025664/t0(0) o35->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:1.0:1787826319.641274:0:161485:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043025664:192.168.203.147@tcp:35:rm.0 00000100:00100000:1.0:1787826319.641479:0:161485:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d225f180 going to sleep for 11 seconds 00000100:00100000:0.0:1787826319.649160:0:161485:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043025664:192.168.203.147@tcp:35:rm.0 00000002:00100000:0.0:1787826319.649195:0:161485:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181c5634e00 x1874663043018496/t352187318278(352187318278) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 0 dl 1787826335 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'rm.0' uid:0 gid:0 projid:0 00010000:00010000:0.0:1787826319.649267:0:161485:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513800/0xbef42c0221d6f32a lrc: 1/0,0 mode: --/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cee4ae expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826319.649276:0:161485:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481800/0xbef42c0221d6f307 lrc: 1/0,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cee1dd expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787826319.649284:0:161485:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 36 lru_type: 1 00010000:00010000:0.0:1787826319.649290:0:161485:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 3 locks packed 00010000:00010000:0.0:1787826319.649293:0:161485:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513800/0xbef42c0221d6f32a lrc: 0/0,0 mode: --/PR res: [0x200027cd1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cee4ae expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826319.649304:0:161485:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481800/0xbef42c0221d6f307 lrc: 0/0,0 mode: --/CR res: [0x200027cd1:0x2:0x0].0x0 bits 0x5/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cee1dd expref: -99 pid: 161485 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787826319.649316:0:161485:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e06f6400/0xbef42c0221d6f24a lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cede80 expref: -99 pid: 160988 timeout: 0 lvb_type: 0 lru_score: 36 lru_type: 1 00000020:00100000:0.0:1787826319.649339:0:161485:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787826319.649344:0:161485:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c55a6680 x1874663043025920/t0(0) o36->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 512/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826319.649363:0:161485:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043025920:192.168.203.147@tcp:36:rm.0 00000100:00100000:0.0:1787826319.656909:0:161485:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c52560c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826319.663552:0:161485:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:161485:1874663043025920:192.168.203.147@tcp:36:rm.0 00000080:00200000:0.0:1787826319.663600:0:161485:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200027cd1:0x2:0x0](ffff9181c83f3248) 00010000:00010000:2.0:1787826319.970015:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826320.993346:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826322.018497:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826323.044577:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826323.491061:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:f40147eb-9b53-47d4-b77c-99ceedd9664c:2390:1874663043026048:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826323.491158:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043026304:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826323.491185:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043026560:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826323.491213:0:2390:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1787826323.491218:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043026176:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826323.491239:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043026432:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826323.517317:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043026176:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826323.517337:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043026432:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826323.517348:0:2390:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff9181c5620000: committing for last_committed 17179870054 gen 1 00000100:00100000:1.0:1787826323.517503:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043026304:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826323.517520:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043026560:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826323.517627:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:f40147eb-9b53-47d4-b77c-99ceedd9664c:2390:1874663043026048:192.168.203.147@tcp:400:kworker.0 00010000:00010000:2.0:1787826324.065284:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826325.089647:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826326.118615:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:1.0:1787826326.343485:0:162053:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == replay-single test 80b: DNE: create remote dir, drop update rep from MDT0, fail MDT1 ========================================================== 06:25:24 (1787826324) 00000080:00200000:2.0:1787826326.518510:0:162066:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:2.0:1787826326.518541:0:162066:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826326.518600:0:162066:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f331 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 162066 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826326.518611:0:162066:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f331 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 162066 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826326.518621:0:162066:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f331 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 162066 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826326.518645:0:162066:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc0a80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162066:1874663043026688:192.168.203.147@tcp:101:lfs.0 00000100:00100000:2.0:1787826326.518732:0:162066:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c32f6480 going to sleep for 11 seconds 00000100:00100000:3.0:1787826326.529371:0:162066:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9181c5620000: committing for last_committed 352187318281 gen 1 00000100:00100000:3.0:1787826326.529399:0:162066:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc0a80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162066:1874663043026688:192.168.203.147@tcp:101:lfs.0 00010000:00010000:3.0:1787826326.529422:0:162066:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f331 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee51e expref: -99 pid: 162066 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.529434:0:162066:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f331 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee51e expref: -99 pid: 162066 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.529451:0:162066:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f331 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee51e expref: -99 pid: 162066 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.529463:0:162066:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f331 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee51e expref: -99 pid: 162066 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.529475:0:162066:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f331 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee51e expref: -99 pid: 162066 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.529484:0:162066:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f331 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee51e expref: -99 pid: 162066 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.529491:0:162066:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f331 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee51e expref: -99 pid: 162066 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826326.529502:0:162066:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181d0fc0a80 x1874663043026688/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826342 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826326.529515:0:162066:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f331 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee51e expref: -99 pid: 162066 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.529528:0:162066:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d1a77c00 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:3.0:1787826326.529534:0:162066:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:3.0:1787826326.529561:0:162066:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 69632 00000080:00010000:3.0:1787826326.529567:0:162066:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f331 00000080:00010000:3.0:1787826326.529572:0:162066:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:3.0:1787826326.529575:0:162066:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f331 from it ffffb97501d8bb58 00010000:00010000:3.0:1787826326.529579:0:162066:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f331 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee51e expref: -99 pid: 162066 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.529588:0:162066:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f331 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee51e expref: -99 pid: 162066 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826326.529609:0:162066:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826326.529635:0:162066:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=d80b.replay-single, dir=[0x200000007:0x1:0x0](ffff9181d080cb08), flags=13 00000080:00200000:3.0:1787826326.529640:0:162066:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=d80b.replay-single, dir=[0x200000007:0x1:0x0](ffff9181d080cb08), intent=getattr 00000002:00010000:3.0:1787826326.529661:0:162066:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: d80b.replay-single,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 01 00010000:00010000:3.0:1787826326.529699:0:162066:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f338 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 162066 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.529706:0:162066:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f338 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 162066 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.529715:0:162066:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f338 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 162066 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826326.529733:0:162066:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636680 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162066:1874663043026816:192.168.203.147@tcp:101:lfs.0 00000100:00100000:3.0:1787826326.531169:0:162066:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d22a2000 going to sleep for 11 seconds 00000100:00100000:3.0:1787826326.537348:0:162066:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636680 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162066:1874663043026816:192.168.203.147@tcp:101:lfs.0 00010000:00010000:3.0:1787826326.537373:0:162066:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f338 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 162066 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.537387:0:162066:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f338 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 162066 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.537395:0:162066:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f338 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 162066 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.537403:0:162066:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f338 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 162066 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.537412:0:162066:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f338 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 162066 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.537419:0:162066:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3480200 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:3.0:1787826326.537423:0:162066:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f338 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 162066 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.537430:0:162066:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f338 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 162066 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.537438:0:162066:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f338 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x20e400000000 pid: 162066 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.537445:0:162066:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826326.537452:0:162066:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f338 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 162066 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826326.537458:0:162066:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f338 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 162066 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826326.537475:0:162066:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c5636680 x1874663043026816/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 592/584 e 0 to 0 dl 1787826342 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:3.0:1787826326.537489:0:162066:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00010000:00010000:3.0:1787826326.537518:0:162066:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f331 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee51e expref: -99 pid: 162066 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826326.537530:0:162066:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f331 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee51e expref: -99 pid: 162066 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00010000:3.0:1787826326.537539:0:162066:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f331 from it ffffb97501d8b930 00010000:00010000:3.0:1787826326.537543:0:162066:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f331 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee51e expref: -99 pid: 162066 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826326.537550:0:162066:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f331 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee51e expref: -99 pid: 162066 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:3.0:1787826326.537617:0:162066:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:3.0:1787826326.537631:0:162066:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826326.537638:0:162066:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f331 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee51e expref: -99 pid: 162066 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826326.537646:0:162066:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f331 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee51e expref: -99 pid: 162066 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:3.0:1787826326.537652:0:162066:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826326.537657:0:162066:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:3.0:1787826326.537660:0:162066:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f331 00000080:00010000:3.0:1787826326.537665:0:162066:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f331 from it ffffb97501d8bbe8 00010000:00010000:3.0:1787826326.537667:0:162066:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f331 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee51e expref: -99 pid: 162066 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826326.537675:0:162066:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f331 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee51e expref: -99 pid: 162066 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:3.0:1787826326.537684:0:162066:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826326.537690:0:162066:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80b.replay-single, flags=16392 00000080:00200008:3.0:1787826326.537715:0:162066:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181d080cb08), request_mask 0, flags 0, foreign 0 00000080:00200000:3.0:1787826326.537720:0:162066:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:3.0:1787826326.537726:0:162066:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826326.537732:0:162066:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f331 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee51e expref: -99 pid: 162066 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:3.0:1787826326.537739:0:162066:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f331 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee51e expref: -99 pid: 162066 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:3.0:1787826326.537778:0:162066:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826326.537783:0:162066:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:3.0:1787826326.537785:0:162066:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f331 00000080:00010000:3.0:1787826326.537789:0:162066:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f331 from it ffffb97501d8bcb8 00010000:00010000:3.0:1787826326.537793:0:162066:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f331 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee51e expref: -99 pid: 162066 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:3.0:1787826326.537800:0:162066:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f331 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee51e expref: -99 pid: 162066 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00000080:00200008:3.0:1787826326.538315:0:162066:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181d080cb08), request_mask 0, flags 0, foreign 0 00000080:00200000:3.0:1787826326.538320:0:162066:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:3.0:1787826326.538327:0:162066:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826326.538335:0:162066:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f331 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee51e expref: -99 pid: 162066 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:3.0:1787826326.538343:0:162066:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f331 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee51e expref: -99 pid: 162066 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00000080:00200000:3.0:1787826326.538349:0:162066:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826326.538353:0:162066:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:3.0:1787826326.538356:0:162066:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f331 00000080:00010000:3.0:1787826326.538359:0:162066:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f331 from it ffffb97501d8bcb8 00010000:00010000:3.0:1787826326.538362:0:162066:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f331 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee51e expref: -99 pid: 162066 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:3.0:1787826326.538369:0:162066:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f331 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee51e expref: -99 pid: 162066 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00000080:00200000:3.0:1787826326.538469:0:162066:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:3.0:1787826326.538477:0:162066:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826326.538483:0:162066:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f331 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee51e expref: -99 pid: 162066 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00010000:00010000:3.0:1787826326.538491:0:162066:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f331 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee51e expref: -99 pid: 162066 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00000080:00200000:3.0:1787826326.538498:0:162066:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826326.538501:0:162066:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:3.0:1787826326.538504:0:162066:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f331 00000080:00010000:3.0:1787826326.538507:0:162066:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f331 from it ffffb97501d8bbb0 00010000:00010000:3.0:1787826326.538510:0:162066:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f331 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee51e expref: -99 pid: 162066 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00010000:00010000:3.0:1787826326.538517:0:162066:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f331 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee51e expref: -99 pid: 162066 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00000080:00200000:3.0:1787826326.538525:0:162066:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 44 00000080:00200008:3.0:1787826326.538531:0:162066:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9181d080cb08), flags 300000 00000080:00200080:3.0:1787826326.538545:0:162066:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c03066f0 arg=7fffb87ee8d0 00000080:00200000:3.0:1787826326.538558:0:162066:0:(dir.c:451:ll_dir_setdirstripe()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) name=d80b.replay-single stripe_offset=0 stripe_count=1, hash_type=0 00010000:00010000:3.0:1787826326.538580:0:162066:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f331 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00cee51e expref: -99 pid: 162066 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00010000:00010000:3.0:1787826326.538628:0:162066:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f331 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cee51e expref: -99 pid: 162066 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00010000:00010000:3.0:1787826326.538635:0:162066:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826326.538637:0:162066:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f331 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cee51e expref: -99 pid: 162066 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00000020:00100000:3.0:1787826326.538659:0:162066:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:3.0:1787826326.538667:0:162066:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c5636680 x1874663043027072/t0(0) o36->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 568/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826326.538688:0:162066:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636680 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162066:1874663043027072:192.168.203.147@tcp:36:lfs.0 00000100:00100000:3.0:1787826326.538785:0:162066:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d22a2000 going to sleep for 11 seconds 00000100:00100000:3.0:1787826326.545188:0:162066:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636680 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162066:1874663043027072:192.168.203.147@tcp:36:lfs.0 00000080:00200000:3.0:1787826326.545240:0:162066:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200027cd1:0x4:0x0](ffff9181d0809148) 00000080:00200000:3.0:1787826326.545247:0:162066:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200027cd1:0x4:0x0], updating i_size 4096 00000080:00200008:3.0:1787826326.545281:0:162066:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9181d080cb08), flags 300000 00010000:00010000:0.0:1787826327.137342:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200000:3.0:1787826327.599806:0:162075:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:3.0:1787826327.599839:0:162075:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826327.599909:0:162075:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 162075 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.599926:0:162075:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 162075 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.599937:0:162075:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 162075 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826327.599962:0:162075:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162075:1874663043027200:192.168.203.147@tcp:101:lfs.0 00000100:00100000:3.0:1787826327.600042:0:162075:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d22a2000 going to sleep for 11 seconds 00000100:00100000:3.0:1787826327.611192:0:162075:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162075:1874663043027200:192.168.203.147@tcp:101:lfs.0 00010000:00010000:3.0:1787826327.611221:0:162075:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee556 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.611233:0:162075:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee556 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.611244:0:162075:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee556 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.611254:0:162075:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee556 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.611267:0:162075:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee556 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.611275:0:162075:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee556 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.611287:0:162075:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee556 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826327.611298:0:162075:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c5634000 x1874663043027200/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826343 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826327.611312:0:162075:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee556 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.611325:0:162075:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d1a77c00 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:3.0:1787826327.611331:0:162075:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:3.0:1787826327.611358:0:162075:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 69632 00000080:00010000:3.0:1787826327.611364:0:162075:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f33f 00000080:00010000:3.0:1787826327.611370:0:162075:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:3.0:1787826327.611373:0:162075:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f33f from it ffffb97501603b58 00010000:00010000:3.0:1787826327.611376:0:162075:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee556 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.611384:0:162075:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee556 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826327.611408:0:162075:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826327.611426:0:162075:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=d80b.replay-single, dir=[0x200000007:0x1:0x0](ffff9181d080cb08), flags=29 00000080:00200000:3.0:1787826327.611431:0:162075:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=d80b.replay-single, dir=[0x200000007:0x1:0x0](ffff9181d080cb08), intent=0 00000002:00010000:3.0:1787826327.611450:0:162075:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: d80b.replay-single,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: lookup flags 0 00010000:00010000:3.0:1787826327.611489:0:162075:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f346 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 162075 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.611499:0:162075:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f346 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 162075 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.611507:0:162075:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f346 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 162075 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826327.611525:0:162075:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162075:1874663043027328:192.168.203.147@tcp:101:lfs.0 00000100:00100000:3.0:1787826327.611608:0:162075:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d22a2000 going to sleep for 11 seconds 00000100:00100000:3.0:1787826327.625105:0:162075:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162075:1874663043027328:192.168.203.147@tcp:101:lfs.0 00010000:00010000:3.0:1787826327.625133:0:162075:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f346 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee572 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.625154:0:162075:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f346 lrc: 4/1,0 mode: --/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee572 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.625163:0:162075:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f346 lrc: 4/1,0 mode: --/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee572 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.625174:0:162075:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f346 lrc: 4/1,0 mode: PR/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee572 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.625185:0:162075:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f346 lrc: 4/1,0 mode: PR/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee572 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.625197:0:162075:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f346 lrc: 5/2,0 mode: PR/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee572 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.625209:0:162075:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f346 lrc: 5/2,0 mode: PR/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee572 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.625216:0:162075:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f346 lrc: 4/1,0 mode: PR/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee572 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826327.625227:0:162075:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x10 disposition=0xb, status=0 req@ffff9181c5634000 x1874663043027328/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 592/688 e 0 to 0 dl 1787826343 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826327.625240:0:162075:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f346 lrc: 3/1,0 mode: PR/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee572 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.625249:0:162075:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d1a77c00 type 13 mode 4 res 8590097617/4 (0 0) 00000080:00200000:3.0:1787826327.625255:0:162075:0:(namei.c:952:ll_intent_lock()) intent lock 16 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:3.0:1787826327.625281:0:162075:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200027cd1:0x4:0x0], updating i_size 4096 00000080:00200000:3.0:1787826327.625284:0:162075:0:(namei.c:147:ll_iget()) got inode: [0x200027cd1:0x4:0x0](ffff9181d0809148): rc = 0 00000080:00010000:3.0:1787826327.625288:0:162075:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200027cd1:0x4:0x0](ffff9181d0809148) for lock 0xbef42c0221d6f346 00000080:00010000:3.0:1787826327.625300:0:162075:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200027cd1:0x4:0x0](ffff9181d0809148) 00000080:00010000:3.0:1787826327.625303:0:162075:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f346 from it ffffb97501603a28 00010000:00010000:3.0:1787826327.625306:0:162075:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f346 lrc: 3/1,0 mode: PR/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee572 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.625314:0:162075:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f346 lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee572 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826327.625343:0:162075:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x4:0x0](ffff9181d0809148), inode mode 41ed mask 1 00000080:00200000:3.0:1787826327.625357:0:162075:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=remote_dir, dir=[0x200027cd1:0x4:0x0](ffff9181d0809148), flags=13 00000080:00200000:3.0:1787826327.625362:0:162075:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=remote_dir, dir=[0x200027cd1:0x4:0x0](ffff9181d0809148), intent=getattr 00000002:00010000:3.0:1787826327.625382:0:162075:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: remote_dir,[0x0:0x0:0x0]) in obj [0x200027cd1:0x4:0x0], intent: getattr flags 01 00010000:00010000:3.0:1787826327.625419:0:162075:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f34d lrc: 3/1,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 162075 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.625425:0:162075:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f34d lrc: 3/1,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 162075 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.625434:0:162075:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f34d lrc: 3/1,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 162075 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826327.625452:0:162075:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162075:1874663043027456:192.168.203.147@tcp:101:lfs.0 00000100:00100000:3.0:1787826327.625541:0:162075:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d22a2000 going to sleep for 11 seconds 00000100:00100000:3.0:1787826327.637359:0:162075:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162075:1874663043027456:192.168.203.147@tcp:101:lfs.0 00010000:00010000:3.0:1787826327.637395:0:162075:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f34d lrc: 4/1,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 162075 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.637410:0:162075:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f34d lrc: 4/1,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 162075 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.637418:0:162075:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f34d lrc: 4/1,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 162075 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.637424:0:162075:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f34d lrc: 3/0,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 162075 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.637433:0:162075:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f34d lrc: 4/0,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 162075 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.637439:0:162075:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3480400 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:3.0:1787826327.637444:0:162075:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f34d lrc: 4/0,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 162075 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.637451:0:162075:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f34d lrc: 4/0,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 162075 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.637458:0:162075:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f34d lrc: 5/0,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x20e400000000 pid: 162075 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.637465:0:162075:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826327.637472:0:162075:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f34d lrc: 3/0,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 162075 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.637479:0:162075:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f34d lrc: 0/0,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 162075 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826327.637496:0:162075:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c5634000 x1874663043027456/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 584/584 e 0 to 0 dl 1787826343 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:3.0:1787826327.637510:0:162075:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200027cd1:0x4:0x0] suppgids 0 -1: rc 0 00010000:00010000:3.0:1787826327.637545:0:162075:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d1a77c00 type 13 mode 30 res 8590097617/4 (0 0) 00000080:00200000:3.0:1787826327.638398:0:162075:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:3.0:1787826327.638414:0:162075:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826327.638424:0:162075:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee556 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826327.638436:0:162075:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee556 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826327.638443:0:162075:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826327.638447:0:162075:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:3.0:1787826327.638450:0:162075:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f33f 00000080:00010000:3.0:1787826327.638455:0:162075:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f33f from it ffffb97501603be8 00010000:00010000:3.0:1787826327.638459:0:162075:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee556 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826327.638466:0:162075:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee556 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:3.0:1787826327.638480:0:162075:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826327.638486:0:162075:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80b.replay-single, flags=16408 00000080:00200000:3.0:1787826327.638491:0:162075:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x4:0x0](ffff9181d0809148), inode mode 41ed mask 1 00000080:00200000:3.0:1787826327.638502:0:162075:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=remote_dir, dir=[0x200027cd1:0x4:0x0](ffff9181d0809148), flags=16392 00000080:00200000:3.0:1787826327.638507:0:162075:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=remote_dir, dir=[0x200027cd1:0x4:0x0](ffff9181d0809148), intent=getattr 00000002:00010000:3.0:1787826327.638520:0:162075:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: remote_dir,[0x0:0x0:0x0]) in obj [0x200027cd1:0x4:0x0], intent: getattr flags 01 00010000:00010000:3.0:1787826327.638796:0:162075:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481200/0xbef42c0221d6f354 lrc: 3/1,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 162075 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.638805:0:162075:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481200/0xbef42c0221d6f354 lrc: 3/1,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 162075 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.638814:0:162075:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481200/0xbef42c0221d6f354 lrc: 3/1,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 162075 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826327.638836:0:162075:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162075:1874663043027584:192.168.203.147@tcp:101:lfs.0 00000100:00100000:3.0:1787826327.638925:0:162075:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d22a2000 going to sleep for 11 seconds 00000100:00100000:3.0:1787826327.646981:0:162075:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162075:1874663043027584:192.168.203.147@tcp:101:lfs.0 00010000:00010000:3.0:1787826327.647008:0:162075:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481200/0xbef42c0221d6f354 lrc: 4/1,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 162075 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.647028:0:162075:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481200/0xbef42c0221d6f354 lrc: 4/1,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 162075 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.647036:0:162075:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481200/0xbef42c0221d6f354 lrc: 4/1,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 162075 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.647044:0:162075:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481200/0xbef42c0221d6f354 lrc: 3/0,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 162075 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.647053:0:162075:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481200/0xbef42c0221d6f354 lrc: 4/0,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 162075 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.647060:0:162075:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3481200 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:3.0:1787826327.647065:0:162075:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481200/0xbef42c0221d6f354 lrc: 4/0,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 162075 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.647073:0:162075:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481200/0xbef42c0221d6f354 lrc: 4/0,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 162075 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.647080:0:162075:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481200/0xbef42c0221d6f354 lrc: 5/0,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x20e400000000 pid: 162075 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.647087:0:162075:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826327.647093:0:162075:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481200/0xbef42c0221d6f354 lrc: 3/0,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 162075 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.647100:0:162075:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481200/0xbef42c0221d6f354 lrc: 0/0,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 162075 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826327.647117:0:162075:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c5634000 x1874663043027584/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 584/584 e 0 to 0 dl 1787826343 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:3.0:1787826327.647137:0:162075:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200027cd1:0x4:0x0] suppgids 0 -1: rc 0 00010000:00010000:3.0:1787826327.647162:0:162075:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d1a77c00 type 13 mode 30 res 8590097617/4 (0 0) 00000080:00200000:3.0:1787826327.647246:0:162075:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:3.0:1787826327.647260:0:162075:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826327.647269:0:162075:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee556 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826327.647281:0:162075:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee556 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:3.0:1787826327.647288:0:162075:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826327.647294:0:162075:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:3.0:1787826327.647297:0:162075:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f33f 00000080:00010000:3.0:1787826327.647302:0:162075:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f33f from it ffffb97501603b58 00010000:00010000:3.0:1787826327.647306:0:162075:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee556 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826327.647314:0:162075:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee556 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:3.0:1787826327.647328:0:162075:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826327.647335:0:162075:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80b.replay-single, flags=13 00000080:00200008:3.0:1787826327.647344:0:162075:0:(file.c:6296:ll_getattr_dentry()) START file d80b.replay-single:[0x200027cd1:0x4:0x0](ffff9181d0809148), request_mask 0, flags 0, foreign 0 00000080:00200000:3.0:1787826327.647349:0:162075:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200027cd1:0x4:0x0](ffff9181d0809148),name=d80b.replay-single 00000002:00010000:3.0:1787826327.647361:0:162075:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200027cd1:0x4:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826327.647445:0:162075:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d1a77c00 type 13 mode 30 res 8590097617/4 (0 0) 00010000:00010000:3.0:1787826327.647483:0:162075:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480c00/0xbef42c0221d6f35b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 162075 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.647492:0:162075:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480c00/0xbef42c0221d6f35b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 162075 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.647501:0:162075:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480c00/0xbef42c0221d6f35b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 162075 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826327.647519:0:162075:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162075:1874663043027712:192.168.203.147@tcp:101:lfs.0 00000100:00100000:3.0:1787826327.647590:0:162075:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d22a2000 going to sleep for 11 seconds 00000100:00100000:3.0:1787826327.653291:0:162075:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162075:1874663043027712:192.168.203.147@tcp:101:lfs.0 00010000:00010000:3.0:1787826327.653319:0:162075:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480c00/0xbef42c0221d6f35b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee5aa expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.653337:0:162075:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480c00/0xbef42c0221d6f35b lrc: 4/1,0 mode: --/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee5aa expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.653345:0:162075:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480c00/0xbef42c0221d6f35b lrc: 4/1,0 mode: --/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee5aa expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.653357:0:162075:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480c00/0xbef42c0221d6f35b lrc: 4/1,0 mode: PR/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee5aa expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.653367:0:162075:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480c00/0xbef42c0221d6f35b lrc: 4/1,0 mode: PR/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee5aa expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.653379:0:162075:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480c00/0xbef42c0221d6f35b lrc: 5/2,0 mode: PR/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee5aa expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.653393:0:162075:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480c00/0xbef42c0221d6f35b lrc: 5/2,0 mode: PR/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee5aa expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.653401:0:162075:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480c00/0xbef42c0221d6f35b lrc: 4/1,0 mode: PR/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee5aa expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826327.653413:0:162075:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c5634000 x1874663043027712/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826343 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826327.653426:0:162075:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480c00/0xbef42c0221d6f35b lrc: 3/1,0 mode: PR/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee5aa expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.653435:0:162075:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d1a77c00 type 13 mode 4 res 8590097617/4 (0 0) 00000080:00200000:3.0:1787826327.653440:0:162075:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:3.0:1787826327.653467:0:162075:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200027cd1:0x4:0x0], updating i_size 4096 00000080:00010000:3.0:1787826327.653473:0:162075:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200027cd1:0x4:0x0](ffff9181d0809148) for lock 0xbef42c0221d6f35b 00000080:00010000:3.0:1787826327.653479:0:162075:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200027cd1:0x4:0x0](ffff9181d0809148) 00000080:00010000:3.0:1787826327.653482:0:162075:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f35b from it ffffb97501603cb8 00010000:00010000:3.0:1787826327.653486:0:162075:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480c00/0xbef42c0221d6f35b lrc: 3/1,0 mode: PR/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee5aa expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.653495:0:162075:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480c00/0xbef42c0221d6f35b lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee5aa expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:3.0:1787826327.655858:0:162075:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181d080cb08), request_mask 0, flags 0, foreign 0 00000080:00200000:3.0:1787826327.655868:0:162075:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:3.0:1787826327.655890:0:162075:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826327.655918:0:162075:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee556 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:3.0:1787826327.655932:0:162075:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee556 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:3.0:1787826327.655940:0:162075:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826327.655947:0:162075:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:3.0:1787826327.655951:0:162075:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f33f 00000080:00010000:3.0:1787826327.655958:0:162075:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f33f from it ffffb97501603cb8 00010000:00010000:3.0:1787826327.655961:0:162075:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee556 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:3.0:1787826327.655970:0:162075:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee556 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00000080:00200000:3.0:1787826327.656216:0:162075:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:3.0:1787826327.656226:0:162075:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826327.656234:0:162075:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee556 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:3.0:1787826327.656254:0:162075:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee556 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00000080:00200000:3.0:1787826327.656261:0:162075:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826327.656265:0:162075:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:3.0:1787826327.656268:0:162075:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f33f 00000080:00010000:3.0:1787826327.656272:0:162075:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f33f from it ffffb97501603bf8 00010000:00010000:3.0:1787826327.656276:0:162075:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee556 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:3.0:1787826327.656284:0:162075:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee556 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00000080:00200000:3.0:1787826327.656303:0:162075:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826327.656310:0:162075:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80b.replay-single, flags=267 00000080:00200000:3.0:1787826327.656316:0:162075:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x4:0x0](ffff9181d0809148), inode mode 41ed mask 44 00000080:00200008:3.0:1787826327.656323:0:162075:0:(file.c:1090:ll_file_open()) START file d80b.replay-single:[0x200027cd1:0x4:0x0](ffff9181d0809148), flags 300000 00000002:00010000:3.0:1787826327.656336:0:162075:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200027cd1:0x4:0x0]) in obj [0x200000007:0x1:0x0], intent: open flags 0240300001 00010000:00010000:3.0:1787826327.656713:0:162075:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f362 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 162075 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.656723:0:162075:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f362 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 162075 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:3.0:1787826327.656735:0:162075:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:3.0:1787826327.656765:0:162075:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c5634000 x1874663043027968/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/131856 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:3.0:1787826327.656946:0:162075:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f362 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 162075 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826327.656972:0:162075:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162075:1874663043027968:192.168.203.147@tcp:101:lfs.0 00000100:00100000:3.0:1787826327.657503:0:162075:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d22a2000 going to sleep for 11 seconds 00000100:00100000:3.0:1787826327.672914:0:162075:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162075:1874663043027968:192.168.203.147@tcp:101:lfs.0 00010000:00010000:3.0:1787826327.672941:0:162075:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f362 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 162075 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.673039:0:162075:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f362 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 162075 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.673047:0:162075:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f362 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 162075 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.673054:0:162075:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f362 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 162075 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.673063:0:162075:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f362 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 162075 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.673070:0:162075:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3480400 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:3.0:1787826327.673473:0:162075:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f362 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x206400000000 pid: 162075 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.673483:0:162075:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f362 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x206400000000 pid: 162075 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.673491:0:162075:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f362 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x20e400000000 pid: 162075 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.673497:0:162075:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826327.673505:0:162075:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f362 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 162075 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826327.673511:0:162075:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f362 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 162075 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826327.673532:0:162075:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x1 disposition=0x2b, status=0 req@ffff9181c5634000 x1874663043027968/t352187318284(352187318284) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826343 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00100000:3.0:1787826327.673551:0:162075:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c5634000 x1874663043027968/t352187318284(352187318284) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826343 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:3.0:1787826327.673566:0:162075:0:(namei.c:952:ll_intent_lock()) intent lock 1 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:3.0:1787826327.673595:0:162075:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200027cd1:0x4:0x0], updating i_size 4096 00000002:00100000:3.0:1787826327.673615:0:162075:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c5634000 x1874663043027968/t352187318284(352187318284) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 0 dl 1787826343 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200080:3.0:1787826327.673679:0:162075:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200027cd1:0x4:0x0](ffff9181d0809148) cmd=c03066f0 arg=7ffdcb9af2a0 00000080:00200000:3.0:1787826327.673695:0:162075:0:(dir.c:451:ll_dir_setdirstripe()) VFS Op:inode=[0x200027cd1:0x4:0x0](ffff9181d0809148) name=remote_dir stripe_offset=1 stripe_count=0, hash_type=0 00000020:00100000:3.0:1787826327.675054:0:162075:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:3.0:1787826327.675069:0:162075:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c5635c00 x1874663043028224/t0(0) o36->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 560/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826327.675166:0:162075:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162075:1874663043028224:192.168.203.147@tcp:36:lfs.0 00000100:00100000:3.0:1787826327.675324:0:162075:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d22a2000 going to sleep for 11 seconds 00000100:00100000:0.0:1787826327.704886:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826327.704996:0:159723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049397248 00000100:00100000:1.0:1787826327.705007:0:159723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049397248 00000100:00100000:1.0:1787826327.705061:0:159723:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 4 00000100:00100000:1.0:1787826327.705085:0:159723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a7b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:c0d7758a-4501-4a89-8ec1-407fef4f0c7f+5:177390:x1874663049397248:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826327.705130:0:159723:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480c00/0xbef42c0221d6f35b lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cee5aa expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826327.705211:0:159723:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826327.705218:0:159723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a7b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:c0d7758a-4501-4a89-8ec1-407fef4f0c7f+5:177390:x1874663049397248:12345-192.168.203.147@tcp:104: Request processed in 149us (347us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826327.705226:0:159723:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 4 00010000:00010000:2.0:1787826327.705450:0:159733:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826327.705457:0:159733:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480c00/0xbef42c0221d6f35b lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cee5aa expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826327.705466:0:159733:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3480c00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:2.0:1787826327.705471:0:159733:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480c00/0xbef42c0221d6f35b lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x13/0x13 rrc: 3 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00cee5aa expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826327.705479:0:159733:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480c00/0xbef42c0221d6f35b lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x13/0x13 rrc: 3 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00cee5aa expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826327.705486:0:159733:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480c00/0xbef42c0221d6f35b lrc: 3/0,0 mode: PR/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x13/0x13 rrc: 3 type: IBT gid 0 flags: 0x408400000000 nid: local remote: 0x878a5eea00cee5aa expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826327.705510:0:159733:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f346 lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee572 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787826327.705582:0:159733:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480c00/0xbef42c0221d6f35b lrc: 2/0,0 mode: --/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x13/0x13 rrc: 3 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cee5aa expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826327.705590:0:159733:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826327.705615:0:159733:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480c00/0xbef42c0221d6f35b lrc: 1/0,0 mode: --/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x13/0x13 rrc: 3 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cee5aa expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826327.705623:0:159733:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480c00/0xbef42c0221d6f35b lrc: 0/0,0 mode: --/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x13/0x13 rrc: 3 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cee5aa expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826327.705637:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826327.705638:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826327.705640:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826327.705648:0:2391:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787826327.705660:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663043028352:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:2.0:1787826327.734677:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663043028352:192.168.203.147@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826328.160923:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200008:1.0:1787826328.606354:0:162085:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181d080cb08), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1787826328.606372:0:162085:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:1.0:1787826328.606394:0:162085:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826328.606412:0:162085:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee556 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00010000:00010000:1.0:1787826328.606427:0:162085:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee556 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00000080:00200000:1.0:1787826328.606435:0:162085:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826328.606443:0:162085:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:1.0:1787826328.606447:0:162085:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f33f 00000080:00010000:1.0:1787826328.606454:0:162085:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f33f from it ffffb97500dabcb8 00010000:00010000:1.0:1787826328.606457:0:162085:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee556 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00010000:00010000:1.0:1787826328.606464:0:162085:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee556 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00000080:00200008:1.0:1787826328.607109:0:162085:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181d080cb08), request_mask 0, flags 0, foreign 0 00000080:00200000:1.0:1787826328.607114:0:162085:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:1.0:1787826328.607123:0:162085:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826328.607132:0:162085:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee556 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00010000:00010000:1.0:1787826328.607142:0:162085:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee556 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00000080:00200000:1.0:1787826328.607150:0:162085:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826328.607154:0:162085:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:1.0:1787826328.607156:0:162085:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f33f 00000080:00010000:1.0:1787826328.607161:0:162085:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f33f from it ffffb97500dabcb8 00010000:00010000:1.0:1787826328.607164:0:162085:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee556 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00010000:00010000:1.0:1787826328.607171:0:162085:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee556 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00000080:00200000:1.0:1787826328.607258:0:162085:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:1.0:1787826328.607267:0:162085:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826328.607274:0:162085:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee556 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00010000:00010000:1.0:1787826328.607283:0:162085:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee556 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00000080:00200000:1.0:1787826328.607295:0:162085:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826328.607299:0:162085:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:1.0:1787826328.607302:0:162085:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f33f 00000080:00010000:1.0:1787826328.607306:0:162085:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f33f from it ffffb97500dabbb0 00010000:00010000:1.0:1787826328.607309:0:162085:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee556 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00010000:00010000:1.0:1787826328.607317:0:162085:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee556 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00000080:00200000:1.0:1787826328.607327:0:162085:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 44 00000080:00200008:1.0:1787826328.607334:0:162085:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9181d080cb08), flags 100000 00000080:00200080:1.0:1787826328.607420:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:1.0:1787826328.607435:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053ec00 uarg=0000000000000000 00800000:00000004:1.0:1787826328.607442:0:162085:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9181c5620000: age 8390, max_age 8399 00000100:00100000:1.0:1787826328.607486:0:162085:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162085:1874663043028480:192.168.203.147@tcp:41:lfs.0 00000100:00100000:1.0:1787826328.607565:0:162085:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c3d40300 going to sleep for 11 seconds 00000100:00100000:2.0:1787826328.619970:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:f40147eb-9b53-47d4-b77c-99ceedd9664c:2391:1874663043028608:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826328.620169:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663043028864:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826328.620385:0:162085:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9181c5620000: committing for last_committed 352187318284 gen 1 00000100:00100000:1.0:1787826328.620404:0:162085:0:(client.c:3174:ptlrpc_free_committed()) @@@ keeping (FL_REPLAY) req@ffff9181c5634000 x1874663043027968/t352187318284(352187318284) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 0 dl 1787826343 ref 1 fl Complete:RPQU/604/ffffffff rc 301/-1 job:'lfs.0' uid:0 gid:0 projid:0 00000100:00100000:1.0:1787826328.620417:0:162085:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162085:1874663043028480:192.168.203.147@tcp:41:lfs.0 00800000:00000004:1.0:1787826328.620434:0:162085:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0000-mdc-ffff9181c5620000: update ffff9181c354d178 cache blocks 317223/353529 objects 1023643/1024000 00000080:00200080:1.0:1787826328.620570:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:1.0:1787826328.620586:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053ec00 uarg=0000000000000000 00800000:00000004:1.0:1787826328.620594:0:162085:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9181c5620000: age 8390, max_age 8399 00000100:00100000:1.0:1787826328.620614:0:162085:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162085:1874663043029120:192.168.203.147@tcp:41:lfs.0 00000100:00100000:1.0:1787826328.620643:0:162085:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c3d40300 going to sleep for 11 seconds 00000100:00100000:3.0:1787826328.620939:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663043028736:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826328.620981:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663043028992:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826328.623027:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:f40147eb-9b53-47d4-b77c-99ceedd9664c:2391:1874663043028608:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826328.625699:0:2392:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff9181c5620000: committing for last_committed 17179870055 gen 1 00000100:00100000:3.0:1787826328.625719:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663043028736:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826328.625734:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663043028992:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826328.625881:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663043028864:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826328.626001:0:162085:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162085:1874663043029120:192.168.203.147@tcp:41:lfs.0 00800000:00000004:0.0:1787826328.626019:0:162085:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0001-mdc-ffff9181c5620000: update ffff9181c3548450 cache blocks 317323/353529 objects 1023702/1024000 00000080:00200080:0.0:1787826328.626085:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626102:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626111:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626115:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626120:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626123:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626127:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626132:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626136:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626139:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626144:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626147:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626152:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626155:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626159:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626162:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626165:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626168:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626172:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626176:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626180:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626183:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626187:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626191:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626195:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626198:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626202:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626205:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626209:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626212:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626217:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626220:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626224:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626228:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626232:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626236:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626240:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626243:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626247:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626251:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626254:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626258:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626261:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626265:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626268:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626272:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626276:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626279:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626284:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626287:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626290:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626294:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626297:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626300:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626304:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626307:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626310:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626313:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626317:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626320:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626324:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626327:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626330:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626333:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626337:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626340:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626344:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626348:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626352:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626355:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626359:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626363:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626367:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626370:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626374:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626377:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626381:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626385:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626389:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626392:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626395:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626399:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626403:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626406:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626409:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626413:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626417:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626420:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626423:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626427:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626430:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626433:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626437:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626440:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626444:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626448:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626451:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626455:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626458:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626461:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626465:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626469:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626472:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626475:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626479:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626482:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626486:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626489:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626493:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626496:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626500:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626503:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626506:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626509:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626524:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626529:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626534:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626537:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626853:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626859:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626863:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626866:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626870:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626873:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626876:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00800000:00000080:0.0:1787826328.626879:0:162085:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.626883:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.626889:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00020000:00000004:0.0:1787826328.626906:0:162085:0:(obd_class.h:1090:obd_statfs()) lustre-OST0000-osc-ffff9181c5620000: age 8390, max_age 8399 00000100:00100000:0.0:1787826328.627064:0:162085:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6300 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162085:1874663043029248:192.168.203.147@tcp:13:lfs.0 00000100:00100000:0.0:1787826328.627185:0:162085:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d225f6c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826328.645669:0:162085:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6300 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162085:1874663043029248:192.168.203.147@tcp:13:lfs.0 00020000:00000004:0.0:1787826328.645700:0:162085:0:(obd_class.h:1116:obd_statfs()) lustre-OST0000-osc-ffff9181c5620000: update ffff9181c3549e08 cache blocks 901356/958279 objects 261720/262144 00000080:00200080:0.0:1787826328.645813:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.645831:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00020000:00000004:0.0:1787826328.645839:0:162085:0:(obd_class.h:1090:obd_statfs()) lustre-OST0001-osc-ffff9181c5620000: age 8390, max_age 8399 00000100:00100000:0.0:1787826328.645872:0:162085:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6300 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162085:1874663043029376:192.168.203.147@tcp:13:lfs.0 00000100:00100000:0.0:1787826328.645945:0:162085:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d225f6c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826328.650497:0:162085:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6300 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162085:1874663043029376:192.168.203.147@tcp:13:lfs.0 00020000:00000004:0.0:1787826328.650526:0:162085:0:(obd_class.h:1116:obd_statfs()) lustre-OST0001-osc-ffff9181c5620000: update ffff9181c354b7c0 cache blocks 901360/958279 objects 261737/262144 00000080:00200080:0.0:1787826328.650592:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.650608:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.650615:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.650618:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.650622:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.650626:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.650629:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.650632:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.650636:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.650639:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.650642:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.650645:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.650648:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.650651:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.650655:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.650658:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.650661:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.650664:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.650670:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.650673:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.650676:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.650679:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.650682:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.650685:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.650689:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.650691:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.650695:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.650698:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.650701:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.650704:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.650707:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.650710:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.650714:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.650717:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.650720:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.650723:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.650726:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.650729:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.650733:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.651016:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.651031:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.651035:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.651039:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.651042:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.651046:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.651049:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.651053:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.651056:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.651059:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.651062:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.651066:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.651068:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.651072:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.651075:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.651078:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.651080:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.651084:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.651088:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.651092:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.651095:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.651098:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.651101:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.651105:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.651109:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.651112:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.651115:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.651119:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.651123:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.651128:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.651131:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.651135:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.651139:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.651143:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.651147:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.651151:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.651154:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.651158:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.651162:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.651166:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.651170:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.651173:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.652123:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.652138:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.652142:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.652146:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.652150:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.652154:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.652158:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.652162:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.652166:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.652175:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.652179:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.652183:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.652188:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.652191:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.652194:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.652198:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.652201:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.652205:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.652208:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.652212:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.652215:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.652219:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.652222:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.652226:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.652230:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.652234:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.652241:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.652245:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.652249:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.652253:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.652256:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.652259:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.652262:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.652266:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.652269:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.652272:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.652275:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.652279:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.652282:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.652286:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.652289:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.652292:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.652295:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200080:0.0:1787826328.652298:0:162085:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc91cbf230 00020000:00000080:0.0:1787826328.652302:0:162085:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d0169c00 uarg=0000000000000000 00000080:00200008:0.0:1787826328.652316:0:162085:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9181d080cb08), flags 100000 00000080:00200000:0.0:1787826328.708486:0:162092:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:0.0:1787826328.708593:0:162092:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826328.708614:0:162092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee556 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00010000:00010000:0.0:1787826328.708631:0:162092:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee556 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00000080:00200000:0.0:1787826328.708640:0:162092:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826328.708647:0:162092:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:0.0:1787826328.708651:0:162092:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f33f 00000080:00010000:0.0:1787826328.708658:0:162092:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f33f from it ffffb975019cbbd0 00010000:00010000:0.0:1787826328.708662:0:162092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee556 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00010000:00010000:0.0:1787826328.708671:0:162092:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee556 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00000080:00200000:0.0:1787826328.708692:0:162092:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826328.708708:0:162092:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=fsa-oleg347-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9181d080cb08), flags=1536 00000080:00200000:0.0:1787826328.708712:0:162092:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:0.0:1787826328.708717:0:162092:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826328.708723:0:162092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee556 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00010000:00010000:0.0:1787826328.708731:0:162092:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee556 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00000080:00200000:0.0:1787826328.708736:0:162092:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826328.708740:0:162092:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:0.0:1787826328.708907:0:162092:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f33f 00000080:00010000:0.0:1787826328.708912:0:162092:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f33f from it ffffb975019cbcd0 00010000:00010000:0.0:1787826328.708915:0:162092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee556 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00010000:00010000:0.0:1787826328.708923:0:162092:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee556 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00000080:00200000:0.0:1787826328.708932:0:162092:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 3 00000080:00200000:0.0:1787826328.708939:0:162092:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:0.0:1787826328.708946:0:162092:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826328.708952:0:162092:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee556 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00010000:00010000:0.0:1787826328.708965:0:162092:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee556 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00000080:00200000:0.0:1787826328.708971:0:162092:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826328.708975:0:162092:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:0.0:1787826328.708978:0:162092:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f33f 00000080:00010000:0.0:1787826328.708981:0:162092:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f33f from it ffffb975019cbdb0 00010000:00010000:0.0:1787826328.708984:0:162092:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee556 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00010000:00010000:0.0:1787826328.708992:0:162092:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee556 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 11 lru_type: 1 00000080:00200000:0.0:1787826328.708999:0:162092:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 3 00000080:00200000:0.0:1787826328.709004:0:162092:0:(namei.c:2129:ll_create_nd()) VFS Op:name=fsa-oleg347-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9181d080cb08), flags=33188, excl=1 00000080:00200000:0.0:1787826328.709008:0:162092:0:(namei.c:2084:ll_mknod()) VFS Op:name=fsa-oleg347-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9181d080cb08) mode 100644 dev 0 00010000:00010000:0.0:1787826328.709031:0:162092:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00cee556 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 11 lru_type: 1 00010000:00010000:0.0:1787826328.709337:0:162092:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cee556 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 11 lru_type: 1 00010000:00010000:0.0:1787826328.709346:0:162092:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826328.709348:0:162092:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f33f lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cee556 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 11 lru_type: 1 00000020:00100000:0.0:1787826328.709372:0:162092:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787826328.709384:0:162092:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c55a6300 x1874663043029632/t0(0) o36->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 528/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826328.709406:0:162092:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6300 pname:cluuid:pid:xid:nid:opc:job mcreate:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162092:1874663043029632:192.168.203.147@tcp:36:mcreate.0 00000100:00100000:0.0:1787826328.709607:0:162092:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d225f6c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826328.727122:0:162092:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6300 pname:cluuid:pid:xid:nid:opc:job mcreate:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162092:1874663043029632:192.168.203.147@tcp:36:mcreate.0 00000080:00200000:0.0:1787826328.727182:0:162092:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200027cd1:0x5:0x0](ffff9181c83f3248) 00000080:00200000:0.0:1787826328.727224:0:162092:0:(namei.c:2137:ll_create_nd()) VFS Op:name=fsa-oleg347-client.virtnet, unhashed 1 00000080:00200000:1.0:1787826328.745206:0:162095:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:1.0:1787826328.745242:0:162095:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826328.745310:0:162095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5200/0xbef42c0221d6f369 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 162095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826328.745320:0:162095:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5200/0xbef42c0221d6f369 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 162095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826328.745330:0:162095:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5200/0xbef42c0221d6f369 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 162095 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826328.745354:0:162095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162095:1874663043029760:192.168.203.147@tcp:101:rm.0 00000100:00100000:1.0:1787826328.745602:0:162095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c3d400c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826328.755269:0:162095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162095:1874663043029760:192.168.203.147@tcp:101:rm.0 00010000:00010000:3.0:1787826328.755304:0:162095:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5200/0xbef42c0221d6f369 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee62f expref: -99 pid: 162095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.755325:0:162095:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5200/0xbef42c0221d6f369 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee62f expref: -99 pid: 162095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.755340:0:162095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5200/0xbef42c0221d6f369 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee62f expref: -99 pid: 162095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.755354:0:162095:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5200/0xbef42c0221d6f369 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee62f expref: -99 pid: 162095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.755382:0:162095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5200/0xbef42c0221d6f369 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee62f expref: -99 pid: 162095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.755393:0:162095:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5200/0xbef42c0221d6f369 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee62f expref: -99 pid: 162095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.755402:0:162095:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5200/0xbef42c0221d6f369 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee62f expref: -99 pid: 162095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826328.755416:0:162095:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c35b3b80 x1874663043029760/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826344 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826328.755433:0:162095:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5200/0xbef42c0221d6f369 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee62f expref: -99 pid: 162095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.755450:0:162095:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d1a77c00 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:3.0:1787826328.755459:0:162095:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:3.0:1787826328.755497:0:162095:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 69632 00000080:00010000:3.0:1787826328.755504:0:162095:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f369 00000080:00010000:3.0:1787826328.755520:0:162095:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:3.0:1787826328.755524:0:162095:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f369 from it ffffb97501bd7b58 00010000:00010000:3.0:1787826328.755528:0:162095:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5200/0xbef42c0221d6f369 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee62f expref: -99 pid: 162095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.755539:0:162095:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5200/0xbef42c0221d6f369 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee62f expref: -99 pid: 162095 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826328.755569:0:162095:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826328.755591:0:162095:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=fsa-oleg347-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9181d080cb08), flags=8 00000080:00200000:3.0:1787826328.755596:0:162095:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=fsa-oleg347-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9181d080cb08), intent=getattr 00000002:00010000:3.0:1787826328.755623:0:162095:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: fsa-oleg347-client.virtnet,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 01 00010000:00010000:3.0:1787826328.755674:0:162095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f370 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 162095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.755682:0:162095:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f370 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 162095 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.755691:0:162095:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f370 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 162095 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826328.755728:0:162095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635180 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162095:1874663043029888:192.168.203.147@tcp:101:rm.0 00000100:00100000:3.0:1787826328.756240:0:162095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d22a2480 going to sleep for 11 seconds 00000100:00100000:3.0:1787826328.766222:0:162095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635180 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162095:1874663043029888:192.168.203.147@tcp:101:rm.0 00010000:00010000:3.0:1787826328.766259:0:162095:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f370 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee64b expref: -99 pid: 162095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.766295:0:162095:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f370 lrc: 4/1,0 mode: --/PR res: [0x200027cd1:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee64b expref: -99 pid: 162095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.766305:0:162095:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f370 lrc: 4/1,0 mode: --/PR res: [0x200027cd1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee64b expref: -99 pid: 162095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.766319:0:162095:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f370 lrc: 4/1,0 mode: PR/PR res: [0x200027cd1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee64b expref: -99 pid: 162095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.766332:0:162095:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f370 lrc: 4/1,0 mode: PR/PR res: [0x200027cd1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee64b expref: -99 pid: 162095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.766348:0:162095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f370 lrc: 5/2,0 mode: PR/PR res: [0x200027cd1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee64b expref: -99 pid: 162095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.766356:0:162095:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f370 lrc: 5/2,0 mode: PR/PR res: [0x200027cd1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee64b expref: -99 pid: 162095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.766364:0:162095:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f370 lrc: 4/1,0 mode: PR/PR res: [0x200027cd1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee64b expref: -99 pid: 162095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826328.766379:0:162095:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c5635180 x1874663043029888/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 600/584 e 0 to 0 dl 1787826344 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826328.766395:0:162095:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f370 lrc: 3/1,0 mode: PR/PR res: [0x200027cd1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee64b expref: -99 pid: 162095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.766406:0:162095:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d1a77c00 type 13 mode 4 res 8590097617/5 (0 0) 00000080:00200000:3.0:1787826328.766416:0:162095:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:3.0:1787826328.766446:0:162095:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200027cd1:0x5:0x0], updating i_size 0 00000080:00200000:3.0:1787826328.766451:0:162095:0:(namei.c:147:ll_iget()) got inode: [0x200027cd1:0x5:0x0](ffff9181c83f3248): rc = 0 00000080:00200000:3.0:1787826328.766473:0:162095:0:(file.c:6843:ll_layout_conf()) [0x200027cd1:0x5:0x0]: layout version change: 4294967294 -> 4294967295 00000080:00010000:3.0:1787826328.766480:0:162095:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200027cd1:0x5:0x0](ffff9181c83f3248) for lock 0xbef42c0221d6f370 00000080:00010000:3.0:1787826328.766495:0:162095:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200027cd1:0x5:0x0](ffff9181c83f3248) 00000080:00010000:3.0:1787826328.766499:0:162095:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f370 from it ffffb97501bd7b20 00010000:00010000:3.0:1787826328.766503:0:162095:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f370 lrc: 3/1,0 mode: PR/PR res: [0x200027cd1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00cee64b expref: -99 pid: 162095 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826328.766525:0:162095:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f370 lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00cee64b expref: -99 pid: 162095 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:3.0:1787826328.766575:0:162095:0:(file.c:6296:ll_getattr_dentry()) START file fsa-oleg347-client.virtnet:[0x200027cd1:0x5:0x0](ffff9181c83f3248), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1787826328.766583:0:162095:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200027cd1:0x5:0x0](ffff9181c83f3248),name=fsa-oleg347-client.virtnet 00000002:00010000:3.0:1787826328.766610:0:162095:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200027cd1:0x5:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826328.766621:0:162095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f370 lrc: 2/1,0 mode: PR/PR res: [0x200027cd1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00cee64b expref: -99 pid: 162095 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826328.766633:0:162095:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f370 lrc: 2/1,0 mode: PR/PR res: [0x200027cd1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00cee64b expref: -99 pid: 162095 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826328.766640:0:162095:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826328.766645:0:162095:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200027cd1:0x5:0x0](ffff9181c83f3248) 00000080:00010000:3.0:1787826328.766649:0:162095:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200027cd1:0x5:0x0](ffff9181c83f3248) for lock 0xbef42c0221d6f370 00000080:00010000:3.0:1787826328.766653:0:162095:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f370 from it ffffb97501bd7cb8 00010000:00010000:3.0:1787826328.766656:0:162095:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f370 lrc: 3/1,0 mode: PR/PR res: [0x200027cd1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00cee64b expref: -99 pid: 162095 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826328.766663:0:162095:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f370 lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00cee64b expref: -99 pid: 162095 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:3.0:1787826328.766908:0:162095:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:3.0:1787826328.766919:0:162095:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826328.766929:0:162095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5200/0xbef42c0221d6f369 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee62f expref: -99 pid: 162095 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826328.766937:0:162095:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5200/0xbef42c0221d6f369 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee62f expref: -99 pid: 162095 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826328.767086:0:162095:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826328.767092:0:162095:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:3.0:1787826328.767095:0:162095:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f369 00000080:00010000:3.0:1787826328.767100:0:162095:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f369 from it ffffb97501bd7c10 00010000:00010000:3.0:1787826328.767104:0:162095:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5200/0xbef42c0221d6f369 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee62f expref: -99 pid: 162095 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826328.767113:0:162095:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5200/0xbef42c0221d6f369 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee62f expref: -99 pid: 162095 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:3.0:1787826328.767124:0:162095:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:2.0:1787826328.768409:0:162095:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=fsa-oleg347-client.virtnet, flags=0 00000080:00200000:2.0:1787826328.768425:0:162095:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:2.0:1787826328.768455:0:162095:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826328.768473:0:162095:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5200/0xbef42c0221d6f369 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee62f expref: -99 pid: 162095 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787826328.768490:0:162095:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5200/0xbef42c0221d6f369 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee62f expref: -99 pid: 162095 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1787826328.768498:0:162095:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826328.768507:0:162095:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:2.0:1787826328.768511:0:162095:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f369 00000080:00010000:2.0:1787826328.768517:0:162095:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f369 from it ffffb97501bd7d68 00010000:00010000:2.0:1787826328.768521:0:162095:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5200/0xbef42c0221d6f369 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee62f expref: -99 pid: 162095 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787826328.768530:0:162095:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5200/0xbef42c0221d6f369 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee62f expref: -99 pid: 162095 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:2.0:1787826328.768546:0:162095:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 3 00000080:00200000:2.0:1787826328.768553:0:162095:0:(namei.c:2418:ll_unlink()) VFS Op:name=fsa-oleg347-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9181d080cb08) 00010000:00010000:2.0:1787826328.768583:0:162095:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5200/0xbef42c0221d6f369 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00cee62f expref: -99 pid: 162095 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1787826328.768610:0:162095:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f370 lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x878a5eea00cee64b expref: -99 pid: 162095 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1787826328.768620:0:162095:0:(vvp_object.c:117:vvp_conf_set()) [0x200027cd1:0x5:0x0]: losing layout lock 00010000:00010000:2.0:1787826328.768659:0:162095:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f370 lrc: 1/0,0 mode: --/PR res: [0x200027cd1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cee64b expref: -99 pid: 162095 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787826328.768666:0:162095:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5200/0xbef42c0221d6f369 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cee62f expref: -99 pid: 162095 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1787826328.768671:0:162095:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:2.0:1787826328.768674:0:162095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f370 lrc: 0/0,0 mode: --/PR res: [0x200027cd1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cee64b expref: -99 pid: 162095 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787826328.768692:0:162095:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5200/0xbef42c0221d6f369 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cee62f expref: -99 pid: 162095 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000020:00100000:2.0:1787826328.768712:0:162095:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787826328.768724:0:162095:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181d0fc0700 x1874663043030144/t0(0) o36->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 512/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826328.768810:0:162095:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc0700 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162095:1874663043030144:192.168.203.147@tcp:36:rm.0 00000100:00100000:2.0:1787826328.768888:0:162095:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c32f6240 going to sleep for 11 seconds 00000100:00100000:2.0:1787826328.777666:0:162095:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc0700 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162095:1874663043030144:192.168.203.147@tcp:36:rm.0 00000080:00200000:2.0:1787826328.777732:0:162095:0:(vvp_io.c:1826:vvp_io_init()) [0x200027cd1:0x5:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:2.0:1787826328.777777:0:162095:0:(vvp_io.c:304:vvp_io_fini()) [0x200027cd1:0x5:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826328.777788:0:162095:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200027cd1:0x5:0x0](ffff9181c83f3248) 00010000:00010000:2.0:1787826329.191932:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826330.209024:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826331.232919:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826332.256926:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826333.281220:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826333.733936:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:f40147eb-9b53-47d4-b77c-99ceedd9664c:2389:1874663043030272:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826333.734020:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663043030528:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826333.734046:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663043030784:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826333.734070:0:2389:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1787826333.734075:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663043030400:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826333.734095:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663043030656:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826333.743876:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663043030656:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826333.743898:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663043030528:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826333.743916:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663043030784:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826333.751214:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663043030400:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826333.751232:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:f40147eb-9b53-47d4-b77c-99ceedd9664c:2389:1874663043030272:192.168.203.147@tcp:400:kworker.0 00010000:00010000:0.0:1787826334.306577:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826335.329172:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:2.0:1787826335.602908:0:162167:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: local REPLAY BARRIER on lustre-MDT0000 00010000:00010000:0.0:1787826336.352980:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200008:1.0:1787826336.492420:0:162176:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181d080cb08), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1787826336.492437:0:162176:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:1.0:1787826336.492459:0:162176:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826336.492519:0:162176:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4400/0xbef42c0221d6f377 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 162176 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826336.492529:0:162176:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4400/0xbef42c0221d6f377 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 162176 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826336.492539:0:162176:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4400/0xbef42c0221d6f377 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 162176 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826336.492562:0:162176:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2300 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162176:1874663043030912:192.168.203.147@tcp:101:lfs.0 00000100:00100000:1.0:1787826336.492644:0:162176:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c3d403c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826336.501990:0:162176:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2300 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162176:1874663043030912:192.168.203.147@tcp:101:lfs.0 00010000:00010000:1.0:1787826336.502016:0:162176:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4400/0xbef42c0221d6f377 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee675 expref: -99 pid: 162176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826336.502029:0:162176:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4400/0xbef42c0221d6f377 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee675 expref: -99 pid: 162176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826336.502040:0:162176:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4400/0xbef42c0221d6f377 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee675 expref: -99 pid: 162176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826336.502050:0:162176:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4400/0xbef42c0221d6f377 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee675 expref: -99 pid: 162176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826336.502063:0:162176:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4400/0xbef42c0221d6f377 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee675 expref: -99 pid: 162176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826336.502072:0:162176:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4400/0xbef42c0221d6f377 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee675 expref: -99 pid: 162176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826336.502086:0:162176:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4400/0xbef42c0221d6f377 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee675 expref: -99 pid: 162176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826336.502095:0:162176:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c35b2300 x1874663043030912/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826352 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826336.502108:0:162176:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4400/0xbef42c0221d6f377 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee675 expref: -99 pid: 162176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826336.502119:0:162176:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d1a77c00 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:1.0:1787826336.502125:0:162176:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1787826336.502149:0:162176:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 69632 00000080:00010000:1.0:1787826336.502153:0:162176:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f377 00000080:00010000:1.0:1787826336.502158:0:162176:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:1.0:1787826336.502161:0:162176:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f377 from it ffffb97501d3bcb8 00010000:00010000:1.0:1787826336.502164:0:162176:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4400/0xbef42c0221d6f377 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee675 expref: -99 pid: 162176 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826336.502173:0:162176:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4400/0xbef42c0221d6f377 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee675 expref: -99 pid: 162176 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:1.0:1787826336.502808:0:162176:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181d080cb08), request_mask 0, flags 0, foreign 0 00000080:00200000:1.0:1787826336.502814:0:162176:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:1.0:1787826336.502830:0:162176:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826336.502852:0:162176:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4400/0xbef42c0221d6f377 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee675 expref: -99 pid: 162176 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826336.502870:0:162176:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4400/0xbef42c0221d6f377 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee675 expref: -99 pid: 162176 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826336.502878:0:162176:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826336.502883:0:162176:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:1.0:1787826336.502886:0:162176:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f377 00000080:00010000:1.0:1787826336.502890:0:162176:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f377 from it ffffb97501d3bcb8 00010000:00010000:1.0:1787826336.502894:0:162176:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4400/0xbef42c0221d6f377 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee675 expref: -99 pid: 162176 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1787826336.502902:0:162176:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4400/0xbef42c0221d6f377 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee675 expref: -99 pid: 162176 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826336.502992:0:162176:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:1.0:1787826336.503000:0:162176:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826336.503006:0:162176:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4400/0xbef42c0221d6f377 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee675 expref: -99 pid: 162176 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826336.503014:0:162176:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4400/0xbef42c0221d6f377 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee675 expref: -99 pid: 162176 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1787826336.503020:0:162176:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826336.503024:0:162176:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:1.0:1787826336.503026:0:162176:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f377 00000080:00010000:1.0:1787826336.503030:0:162176:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f377 from it ffffb97501d3bbb0 00010000:00010000:1.0:1787826336.503033:0:162176:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4400/0xbef42c0221d6f377 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee675 expref: -99 pid: 162176 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787826336.503041:0:162176:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4400/0xbef42c0221d6f377 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee675 expref: -99 pid: 162176 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787826336.503049:0:162176:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 44 00000080:00200008:1.0:1787826336.503055:0:162176:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9181d080cb08), flags 100000 00000080:00200080:1.0:1787826336.503138:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.503151:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00800000:00000004:1.0:1787826336.503158:0:162176:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9181c5620000: age 8400, max_age 8407 00000100:00100000:1.0:1787826336.503189:0:162176:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2300 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162176:1874663043031040:192.168.203.147@tcp:41:lfs.0 00000100:00100000:1.0:1787826336.503250:0:162176:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c3d403c0 going to sleep for 50 seconds 00000100:00100000:1.0:1787826336.508977:0:162176:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2300 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162176:1874663043031040:192.168.203.147@tcp:41:lfs.0 00800000:00000004:1.0:1787826336.509006:0:162176:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0000-mdc-ffff9181c5620000: update ffff9181c354d178 cache blocks 317223/353529 objects 1023643/1024000 00000080:00200080:1.0:1787826336.509303:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.509321:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00800000:00000004:1.0:1787826336.509328:0:162176:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9181c5620000: age 8400, max_age 8407 00000100:00100000:1.0:1787826336.509375:0:162176:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2300 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162176:1874663043031168:192.168.203.147@tcp:41:lfs.0 00000100:00100000:1.0:1787826336.509447:0:162176:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c3d403c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826336.516572:0:162176:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2300 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162176:1874663043031168:192.168.203.147@tcp:41:lfs.0 00800000:00000004:1.0:1787826336.516603:0:162176:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0001-mdc-ffff9181c5620000: update ffff9181c3548450 cache blocks 317323/353529 objects 1023702/1024000 00000080:00200080:1.0:1787826336.516678:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.516697:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.516705:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.516710:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.516715:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.516718:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.516723:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.516727:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.516731:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.516735:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.516739:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.516782:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.516789:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.516794:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.516798:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.516802:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.516808:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.516812:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.516817:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.516821:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.516825:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.516829:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.516834:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.516850:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.516855:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.516858:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.516862:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.516865:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.516869:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.516873:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.516877:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.516881:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.516885:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.516891:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.516895:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.516898:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.516901:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.516904:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.516908:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.516911:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.516914:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.516918:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.516921:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.516924:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.516927:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.516930:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.516934:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.516938:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.516942:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.516944:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.516948:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.516950:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.516954:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.516957:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.516961:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.516964:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.516968:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.516972:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.516975:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.516979:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.516983:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.516988:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.516991:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.516995:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.516998:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.517002:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.517005:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.517008:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.517011:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.517014:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.517018:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.517021:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.517025:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.517028:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.517032:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.517035:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.517039:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.517042:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.517046:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.517049:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.517053:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.517056:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.517060:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.517064:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.517068:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.517071:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.517075:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.517078:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.517081:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.517084:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.517088:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.517091:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.517095:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.517098:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.517102:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.517105:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.517109:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.517113:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.517117:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.517120:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.517123:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.517127:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.517130:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.517133:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.517137:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.517140:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.517144:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.517148:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.517152:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.517155:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.517158:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.517161:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.517164:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.517168:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.517171:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.517174:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.517178:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.517181:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.517185:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.517189:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.517192:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.517196:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.517199:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.517202:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.517206:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00800000:00000080:1.0:1787826336.517210:0:162176:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.517214:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.517220:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00020000:00000004:1.0:1787826336.517227:0:162176:0:(obd_class.h:1090:obd_statfs()) lustre-OST0000-osc-ffff9181c5620000: age 8400, max_age 8407 00000100:00100000:1.0:1787826336.517263:0:162176:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2300 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162176:1874663043031296:192.168.203.147@tcp:13:lfs.0 00000100:00100000:1.0:1787826336.517340:0:162176:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c3d403c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826336.527262:0:162176:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2300 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162176:1874663043031296:192.168.203.147@tcp:13:lfs.0 00020000:00000004:1.0:1787826336.527292:0:162176:0:(obd_class.h:1116:obd_statfs()) lustre-OST0000-osc-ffff9181c5620000: update ffff9181c3549e08 cache blocks 901356/958279 objects 261720/262144 00000080:00200080:1.0:1787826336.527371:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.527389:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00020000:00000004:1.0:1787826336.527397:0:162176:0:(obd_class.h:1090:obd_statfs()) lustre-OST0001-osc-ffff9181c5620000: age 8400, max_age 8407 00000100:00100000:1.0:1787826336.527430:0:162176:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2300 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162176:1874663043031424:192.168.203.147@tcp:13:lfs.0 00000100:00100000:1.0:1787826336.527515:0:162176:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c3d403c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826336.531733:0:162176:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2300 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162176:1874663043031424:192.168.203.147@tcp:13:lfs.0 00020000:00000004:1.0:1787826336.531791:0:162176:0:(obd_class.h:1116:obd_statfs()) lustre-OST0001-osc-ffff9181c5620000: update ffff9181c354b7c0 cache blocks 901360/958279 objects 261737/262144 00000080:00200080:1.0:1787826336.531859:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.531876:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.531883:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.531887:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.531892:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.531896:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.531901:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.531905:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.531909:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.531913:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.531918:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.531922:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.531926:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.531931:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.531935:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.531939:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.531944:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.531948:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.531952:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.531955:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.531959:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.531963:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.531966:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.531970:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.531974:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.531978:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.531982:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.531986:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.531990:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.531994:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.531998:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.532001:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.532005:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.532008:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.532012:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.532015:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.532018:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.532022:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.532026:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.532030:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.532034:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.532038:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.532042:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.532045:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.532050:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.532054:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.532058:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.532061:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.532065:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.532069:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.532073:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.532076:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.532080:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.532084:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.532088:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.532092:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.532096:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.532099:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.532103:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.532107:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.532111:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.532114:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.532119:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.532123:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.532126:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.532130:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.532134:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.532138:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.532142:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.532145:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.532149:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.532153:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.532156:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.532160:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.532164:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.532168:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.532172:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.532175:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.532179:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.532183:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.532187:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.532191:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.532195:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.532199:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.532203:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.532207:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.532211:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.532215:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.532220:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.532223:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.532227:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.532231:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.532235:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.532238:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.532242:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.532246:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.532249:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.532252:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.532256:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.532259:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.532263:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.532267:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.532271:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.532274:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.532278:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.532281:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.532286:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.532289:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.532293:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.532297:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.532301:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.532304:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.532308:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.532313:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.532316:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.532320:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.532324:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.532328:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.532333:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.532337:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.532340:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.532344:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.532348:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.532351:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200080:1.0:1787826336.532355:0:162176:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffc89542230 00020000:00000080:1.0:1787826336.532358:0:162176:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181e053f000 uarg=0000000000000000 00000080:00200008:1.0:1787826336.532370:0:162176:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9181d080cb08), flags 100000 00000080:00200000:1.0:1787826336.579153:0:162183:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:1.0:1787826336.579183:0:162183:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826336.579201:0:162183:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4400/0xbef42c0221d6f377 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee675 expref: -99 pid: 162176 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787826336.579218:0:162183:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4400/0xbef42c0221d6f377 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee675 expref: -99 pid: 162176 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787826336.579226:0:162183:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826336.579232:0:162183:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:1.0:1787826336.579236:0:162183:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f377 00000080:00010000:1.0:1787826336.579243:0:162183:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f377 from it ffffb97501cc3bd0 00010000:00010000:1.0:1787826336.579246:0:162183:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4400/0xbef42c0221d6f377 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee675 expref: -99 pid: 162176 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787826336.579256:0:162183:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4400/0xbef42c0221d6f377 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee675 expref: -99 pid: 162176 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00000080:00200000:1.0:1787826336.579270:0:162183:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826336.579283:0:162183:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=fsa-oleg347-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9181d080cb08), flags=1536 00000080:00200000:1.0:1787826336.579286:0:162183:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:1.0:1787826336.579292:0:162183:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826336.579298:0:162183:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4400/0xbef42c0221d6f377 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee675 expref: -99 pid: 162176 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:1.0:1787826336.579305:0:162183:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4400/0xbef42c0221d6f377 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee675 expref: -99 pid: 162176 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00000080:00200000:1.0:1787826336.579311:0:162183:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826336.579315:0:162183:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:1.0:1787826336.579317:0:162183:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f377 00000080:00010000:1.0:1787826336.579320:0:162183:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f377 from it ffffb97501cc3cd0 00010000:00010000:1.0:1787826336.579323:0:162183:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4400/0xbef42c0221d6f377 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee675 expref: -99 pid: 162176 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:1.0:1787826336.579330:0:162183:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4400/0xbef42c0221d6f377 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee675 expref: -99 pid: 162176 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00000080:00200000:1.0:1787826336.579338:0:162183:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 3 00000080:00200000:1.0:1787826336.579343:0:162183:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:1.0:1787826336.579348:0:162183:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826336.579354:0:162183:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4400/0xbef42c0221d6f377 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee675 expref: -99 pid: 162176 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00010000:00010000:1.0:1787826336.579360:0:162183:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4400/0xbef42c0221d6f377 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee675 expref: -99 pid: 162176 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00000080:00200000:1.0:1787826336.579366:0:162183:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826336.579369:0:162183:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:1.0:1787826336.579371:0:162183:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f377 00000080:00010000:1.0:1787826336.579374:0:162183:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f377 from it ffffb97501cc3db0 00010000:00010000:1.0:1787826336.579377:0:162183:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4400/0xbef42c0221d6f377 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee675 expref: -99 pid: 162176 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00010000:00010000:1.0:1787826336.579383:0:162183:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4400/0xbef42c0221d6f377 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee675 expref: -99 pid: 162176 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00000080:00200000:1.0:1787826336.579390:0:162183:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 3 00000080:00200000:1.0:1787826336.579395:0:162183:0:(namei.c:2129:ll_create_nd()) VFS Op:name=fsa-oleg347-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9181d080cb08), flags=33188, excl=1 00000080:00200000:1.0:1787826336.579398:0:162183:0:(namei.c:2084:ll_mknod()) VFS Op:name=fsa-oleg347-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9181d080cb08) mode 100644 dev 0 00010000:00010000:1.0:1787826336.579422:0:162183:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4400/0xbef42c0221d6f377 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00cee675 expref: -99 pid: 162176 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00010000:00010000:1.0:1787826336.579468:0:162183:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4400/0xbef42c0221d6f377 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cee675 expref: -99 pid: 162176 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00010000:00010000:1.0:1787826336.579474:0:162183:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826336.579476:0:162183:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4400/0xbef42c0221d6f377 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cee675 expref: -99 pid: 162176 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00000020:00100000:1.0:1787826336.579504:0:162183:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787826336.579516:0:162183:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c35b2300 x1874663043031680/t0(0) o36->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 528/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826336.579539:0:162183:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2300 pname:cluuid:pid:xid:nid:opc:job mcreate:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162183:1874663043031680:192.168.203.147@tcp:36:mcreate.0 00000100:00100000:1.0:1787826336.579612:0:162183:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c3d403c0 going to sleep for 50 seconds 00000100:00100000:3.0:1787826336.583413:0:162183:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2300 pname:cluuid:pid:xid:nid:opc:job mcreate:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162183:1874663043031680:192.168.203.147@tcp:36:mcreate.0 00000080:00200000:3.0:1787826336.583487:0:162183:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200027cd1:0x6:0x0](ffff9181d080e3c8) 00000080:00200000:3.0:1787826336.583538:0:162183:0:(namei.c:2137:ll_create_nd()) VFS Op:name=fsa-oleg347-client.virtnet, unhashed 1 00000080:00200000:3.0:1787826336.603868:0:162186:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:3.0:1787826336.604400:0:162186:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826336.604491:0:162186:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f37e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 162186 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826336.604504:0:162186:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f37e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 162186 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826336.604516:0:162186:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f37e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 162186 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826336.604546:0:162186:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635180 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162186:1874663043031808:192.168.203.147@tcp:101:rm.0 00000100:00100000:3.0:1787826336.604727:0:162186:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d22a2180 going to sleep for 50 seconds 00000100:00100000:3.0:1787826336.607739:0:162186:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635180 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162186:1874663043031808:192.168.203.147@tcp:101:rm.0 00010000:00010000:3.0:1787826336.607818:0:162186:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f37e lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee691 expref: -99 pid: 162186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826336.607832:0:162186:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f37e lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee691 expref: -99 pid: 162186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826336.607858:0:162186:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f37e lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee691 expref: -99 pid: 162186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826336.607872:0:162186:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f37e lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee691 expref: -99 pid: 162186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826336.607887:0:162186:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f37e lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee691 expref: -99 pid: 162186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826336.607907:0:162186:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f37e lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee691 expref: -99 pid: 162186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826336.607917:0:162186:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f37e lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee691 expref: -99 pid: 162186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826336.607931:0:162186:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c5635180 x1874663043031808/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826391 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826336.607946:0:162186:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f37e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee691 expref: -99 pid: 162186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826336.607960:0:162186:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d1a77c00 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:3.0:1787826336.607969:0:162186:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:3.0:1787826336.608006:0:162186:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 69632 00000080:00010000:3.0:1787826336.608013:0:162186:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f37e 00000080:00010000:3.0:1787826336.608020:0:162186:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:3.0:1787826336.608023:0:162186:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f37e from it ffffb97501d63b58 00010000:00010000:3.0:1787826336.608027:0:162186:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f37e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee691 expref: -99 pid: 162186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826336.608037:0:162186:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f37e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee691 expref: -99 pid: 162186 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826336.608065:0:162186:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826336.608088:0:162186:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=fsa-oleg347-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9181d080cb08), flags=8 00000080:00200000:3.0:1787826336.608093:0:162186:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=fsa-oleg347-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9181d080cb08), intent=getattr 00000002:00010000:3.0:1787826336.608120:0:162186:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: fsa-oleg347-client.virtnet,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 01 00010000:00010000:3.0:1787826336.608376:0:162186:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f385 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 162186 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826336.608386:0:162186:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f385 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 162186 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826336.608396:0:162186:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f385 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 162186 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826336.608420:0:162186:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635180 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162186:1874663043031936:192.168.203.147@tcp:101:rm.0 00000100:00100000:3.0:1787826336.608514:0:162186:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d22a2180 going to sleep for 50 seconds 00000100:00100000:3.0:1787826336.611249:0:162186:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635180 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162186:1874663043031936:192.168.203.147@tcp:101:rm.0 00010000:00010000:3.0:1787826336.611282:0:162186:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f385 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee6ad expref: -99 pid: 162186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826336.612458:0:162186:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f385 lrc: 4/1,0 mode: --/PR res: [0x200027cd1:0x6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee6ad expref: -99 pid: 162186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826336.613169:0:162186:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f385 lrc: 4/1,0 mode: --/PR res: [0x200027cd1:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee6ad expref: -99 pid: 162186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826336.613184:0:162186:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f385 lrc: 4/1,0 mode: PR/PR res: [0x200027cd1:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee6ad expref: -99 pid: 162186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826336.613198:0:162186:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f385 lrc: 4/1,0 mode: PR/PR res: [0x200027cd1:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee6ad expref: -99 pid: 162186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826336.613220:0:162186:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f385 lrc: 5/2,0 mode: PR/PR res: [0x200027cd1:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee6ad expref: -99 pid: 162186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826336.613235:0:162186:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f385 lrc: 5/2,0 mode: PR/PR res: [0x200027cd1:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee6ad expref: -99 pid: 162186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826336.613243:0:162186:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f385 lrc: 4/1,0 mode: PR/PR res: [0x200027cd1:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee6ad expref: -99 pid: 162186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826336.613261:0:162186:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c5635180 x1874663043031936/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 600/584 e 0 to 0 dl 1787826391 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826336.613277:0:162186:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f385 lrc: 3/1,0 mode: PR/PR res: [0x200027cd1:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee6ad expref: -99 pid: 162186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826336.613290:0:162186:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d1a77c00 type 13 mode 4 res 8590097617/6 (0 0) 00000080:00200000:3.0:1787826336.613299:0:162186:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:3.0:1787826336.613328:0:162186:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200027cd1:0x6:0x0], updating i_size 0 00000080:00200000:3.0:1787826336.613331:0:162186:0:(namei.c:147:ll_iget()) got inode: [0x200027cd1:0x6:0x0](ffff9181d080e3c8): rc = 0 00000080:00200000:3.0:1787826336.613352:0:162186:0:(file.c:6843:ll_layout_conf()) [0x200027cd1:0x6:0x0]: layout version change: 4294967294 -> 4294967295 00000080:00010000:3.0:1787826336.613359:0:162186:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200027cd1:0x6:0x0](ffff9181d080e3c8) for lock 0xbef42c0221d6f385 00000080:00010000:3.0:1787826336.613373:0:162186:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200027cd1:0x6:0x0](ffff9181d080e3c8) 00000080:00010000:3.0:1787826336.613377:0:162186:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f385 from it ffffb97501d63b20 00010000:00010000:3.0:1787826336.613380:0:162186:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f385 lrc: 3/1,0 mode: PR/PR res: [0x200027cd1:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00cee6ad expref: -99 pid: 162186 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826336.613391:0:162186:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f385 lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00cee6ad expref: -99 pid: 162186 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:3.0:1787826336.613443:0:162186:0:(file.c:6296:ll_getattr_dentry()) START file fsa-oleg347-client.virtnet:[0x200027cd1:0x6:0x0](ffff9181d080e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1787826336.613452:0:162186:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200027cd1:0x6:0x0](ffff9181d080e3c8),name=fsa-oleg347-client.virtnet 00000002:00010000:3.0:1787826336.613625:0:162186:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200027cd1:0x6:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826336.614100:0:162186:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f385 lrc: 2/1,0 mode: PR/PR res: [0x200027cd1:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00cee6ad expref: -99 pid: 162186 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826336.614271:0:162186:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f385 lrc: 2/1,0 mode: PR/PR res: [0x200027cd1:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00cee6ad expref: -99 pid: 162186 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826336.614284:0:162186:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826336.614292:0:162186:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200027cd1:0x6:0x0](ffff9181d080e3c8) 00000080:00010000:2.0:1787826336.614296:0:162186:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200027cd1:0x6:0x0](ffff9181d080e3c8) for lock 0xbef42c0221d6f385 00000080:00010000:2.0:1787826336.614301:0:162186:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f385 from it ffffb97501d63cb8 00010000:00010000:2.0:1787826336.614305:0:162186:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f385 lrc: 3/1,0 mode: PR/PR res: [0x200027cd1:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00cee6ad expref: -99 pid: 162186 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826336.614316:0:162186:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f385 lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00cee6ad expref: -99 pid: 162186 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1787826336.614390:0:162186:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:2.0:1787826336.614401:0:162186:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826336.614410:0:162186:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f37e lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee691 expref: -99 pid: 162186 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826336.614417:0:162186:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f37e lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee691 expref: -99 pid: 162186 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826336.614423:0:162186:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826336.614428:0:162186:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:2.0:1787826336.614431:0:162186:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f37e 00000080:00010000:2.0:1787826336.614435:0:162186:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f37e from it ffffb97501d63c10 00010000:00010000:2.0:1787826336.614438:0:162186:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f37e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee691 expref: -99 pid: 162186 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826336.614445:0:162186:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f37e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee691 expref: -99 pid: 162186 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1787826336.614452:0:162186:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:2.0:1787826336.614463:0:162186:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=fsa-oleg347-client.virtnet, flags=0 00000080:00200000:2.0:1787826336.614468:0:162186:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:2.0:1787826336.614475:0:162186:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826336.614481:0:162186:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f37e lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee691 expref: -99 pid: 162186 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787826336.614489:0:162186:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f37e lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee691 expref: -99 pid: 162186 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1787826336.614496:0:162186:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826336.614499:0:162186:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:2.0:1787826336.614502:0:162186:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f37e 00000080:00010000:2.0:1787826336.614505:0:162186:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f37e from it ffffb97501d63d68 00010000:00010000:2.0:1787826336.614508:0:162186:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f37e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee691 expref: -99 pid: 162186 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787826336.614516:0:162186:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f37e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee691 expref: -99 pid: 162186 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:2.0:1787826336.614529:0:162186:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 3 00000080:00200000:2.0:1787826336.614534:0:162186:0:(namei.c:2418:ll_unlink()) VFS Op:name=fsa-oleg347-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9181d080cb08) 00010000:00010000:2.0:1787826336.614563:0:162186:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f37e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00cee691 expref: -99 pid: 162186 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1787826336.614592:0:162186:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f385 lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x878a5eea00cee6ad expref: -99 pid: 162186 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1787826336.614603:0:162186:0:(vvp_object.c:117:vvp_conf_set()) [0x200027cd1:0x6:0x0]: losing layout lock 00010000:00010000:2.0:1787826336.614653:0:162186:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f385 lrc: 1/0,0 mode: --/PR res: [0x200027cd1:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cee6ad expref: -99 pid: 162186 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787826336.614662:0:162186:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f37e lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cee691 expref: -99 pid: 162186 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1787826336.614669:0:162186:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:2.0:1787826336.614672:0:162186:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480200/0xbef42c0221d6f385 lrc: 0/0,0 mode: --/PR res: [0x200027cd1:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cee6ad expref: -99 pid: 162186 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787826336.614692:0:162186:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480400/0xbef42c0221d6f37e lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cee691 expref: -99 pid: 162186 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000020:00100000:2.0:1787826336.614711:0:162186:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787826336.614724:0:162186:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181d0fc3b80 x1874663043032192/t0(0) o36->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 512/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826336.614782:0:162186:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc3b80 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162186:1874663043032192:192.168.203.147@tcp:36:rm.0 00000100:00100000:2.0:1787826336.616153:0:162186:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c32f6240 going to sleep for 50 seconds 00000100:00100000:2.0:1787826336.626241:0:162186:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc3b80 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162186:1874663043032192:192.168.203.147@tcp:36:rm.0 00000080:00200000:2.0:1787826336.626318:0:162186:0:(vvp_io.c:1826:vvp_io_init()) [0x200027cd1:0x6:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:2.0:1787826336.626330:0:162186:0:(vvp_io.c:304:vvp_io_fini()) [0x200027cd1:0x6:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826336.626340:0:162186:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200027cd1:0x6:0x0](ffff9181d080e3c8) 00010000:00010000:1.0:1787826337.379209:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826338.400926:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826338.849036:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:f40147eb-9b53-47d4-b77c-99ceedd9664c:2390:1874663043032320:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826338.849047:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663043032448:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826338.849151:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043032576:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826338.849151:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663043032704:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826338.849175:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043032832:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826338.850440:0:162075:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d22a2000 going to sleep for 25 seconds 00000100:00100000:0.0:1787826338.861031:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:f40147eb-9b53-47d4-b77c-99ceedd9664c:2390:1874663043032320:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826338.861060:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043032576:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826338.861321:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663043032448:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826338.862847:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663043032704:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826338.862909:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043032832:192.168.203.147@tcp:400:kworker.0 00010000:00010000:1.0:1787826339.425078:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826340.449043:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826341.473064:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826342.497047:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:0.0:1787826343.317329:0:162262:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: local REPLAY BARRIER on lustre-MDT0001 00010000:00010000:1.0:1787826343.521106:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826343.969314:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:f40147eb-9b53-47d4-b77c-99ceedd9664c:2389:1874663043032960:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826343.969441:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663043033216:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826343.969481:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663043033472:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826343.969513:0:2389:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1787826343.969519:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663043033088:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826343.969540:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663043033344:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826343.976670:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663043033088:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826343.976691:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663043033216:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826343.978330:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663043033344:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826343.978347:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:f40147eb-9b53-47d4-b77c-99ceedd9664c:2389:1874663043032960:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826343.978358:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663043033472:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826344.138529:0:162075:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162075:1874663043028224:192.168.203.147@tcp:36:lfs.0 00000080:00200000:3.0:1787826344.138589:0:162075:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448) 00000080:00200000:3.0:1787826344.138598:0:162075:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x240001f59:0x16:0x0], updating i_size 4096 00000080:00200008:3.0:1787826344.138636:0:162075:0:(file.c:479:ll_file_release()) START file d80b.replay-single:[0x200027cd1:0x4:0x0](ffff9181d0809148), flags 300000 00000002:00100000:3.0:1787826344.138711:0:162075:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181c5634000 x1874663043027968/t352187318284(352187318284) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 0 dl 1787826343 ref 1 fl Complete:RPQU/604/ffffffff rc 301/-1 job:'lfs.0' uid:0 gid:0 projid:0 00000020:00100000:3.0:1787826344.138724:0:162075:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:3.0:1787826344.138730:0:162075:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c5635180 x1874663043033728/t0(0) o35->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:3.0:1787826344.138809:0:162075:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635180 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162075:1874663043033728:192.168.203.147@tcp:35:lfs.0 00000100:00100000:3.0:1787826344.138938:0:162075:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d22a2000 going to sleep for 11 seconds 00000100:00100000:3.0:1787826344.144247:0:162075:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635180 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162075:1874663043033728:192.168.203.147@tcp:35:lfs.0 00000002:00100000:3.0:1787826344.144302:0:162075:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181c5634000 x1874663043027968/t352187318284(352187318284) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 0 dl 1787826343 ref 1 fl Complete:RQU/604/ffffffff rc 301/-1 job:'lfs.0' uid:0 gid:0 projid:0 00010000:00010000:0.0:1787826344.545154:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826345.569078:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826346.593953:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826347.617024:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826348.643607:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826349.089519:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:f40147eb-9b53-47d4-b77c-99ceedd9664c:2390:1874663043033856:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826349.089613:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043034112:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826349.089647:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043034368:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826349.089669:0:2390:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1787826349.089674:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043033984:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826349.089689:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043034240:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826349.102342:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043033984:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826349.102363:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043034240:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826349.102439:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:f40147eb-9b53-47d4-b77c-99ceedd9664c:2390:1874663043033856:192.168.203.147@tcp:400:kworker.0 00000100:00080000:0.0:1787826349.102455:0:2390:0:(recover.c:205:ptlrpc_request_handle_notconn()) import lustre-MDT0001-mdc-ffff9181c5620000 of lustre-MDT0001_UUID@192.168.203.147@tcp abruptly disconnected: reconnecting 00000100:00080000:0.0:1787826349.102460:0:2390:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from FULL to DISCONN 00000100:02000400:0.0:1787826349.102464:0:2390:0:(import.c:182:ptlrpc_set_import_discon()) lustre-MDT0001-mdc-ffff9181c5620000: Connection to lustre-MDT0001 (at 192.168.203.147@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00080000:0.0:1787826349.131909:0:2390:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826349.131928:0:2390:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8306 00000100:00080000:0.0:1787826349.131943:0:2390:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787826349.131955:0:2390:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826349.131987:0:2390:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5500 x1874663043034496/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826349.132027:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043034112:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826349.132065:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043034368:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826349.132737:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043034496:192.168.203.147@tcp:38:ptlrpcd_00_01.0 00000100:00080000:2.0:1787826349.142094:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826349.142111:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-19) 00000100:00100000:2.0:1787826349.142117:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043034496:192.168.203.147@tcp:38:ptlrpcd_00_01.0 00010000:00010000:1.0:1787826349.665116:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826350.688897:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826351.713444:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826352.736927:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826353.762304:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787826354.213784:0:138695:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826354.213790:0:138695:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787826354.213793:0:138695:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826354.213804:0:138695:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8420 00000100:00080000:3.0:1787826354.213815:0:138695:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 5s 00000100:00080000:3.0:1787826354.213821:0:138695:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826354.213831:0:138695:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826354.213847:0:138695:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635880 x1874663043034880/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826354.213908:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663043034752:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826354.213995:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663043035136:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826354.214025:0:2392:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1787826354.214031:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:f40147eb-9b53-47d4-b77c-99ceedd9664c:2392:1874663043034624:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826354.214061:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663043035008:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826354.214174:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043034880:192.168.203.147@tcp:38:kworker.0 00000100:00080000:2.0:1787826354.224189:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826354.224206:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-19) 00000100:00100000:2.0:1787826354.224211:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043034880:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787826354.224702:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663043034752:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826354.225091:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:f40147eb-9b53-47d4-b77c-99ceedd9664c:2392:1874663043034624:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826354.225104:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663043035008:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826354.225115:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663043035136:192.168.203.147@tcp:400:kworker.0 00010000:00010000:0.0:1787826354.784900:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826355.808964:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826356.833549:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826357.857156:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826358.880912:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826359.329131:0:2392:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787826359.329166:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:f40147eb-9b53-47d4-b77c-99ceedd9664c:2392:1874663043035264:192.168.203.147@tcp:400:kworker.0 00000100:00080000:3.0:1787826359.329342:0:138695:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826359.329349:0:138695:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787826359.329354:0:138695:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826359.329362:0:138695:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8425 00000100:00080000:3.0:1787826359.329371:0:138695:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 10s 00000100:00100000:2.0:1787826359.329377:0:2391:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00080000:3.0:1787826359.329382:0:138695:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787826359.329412:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663043035392:192.168.203.147@tcp:400:kworker.0 00000008:00100000:3.0:1787826359.329416:0:138695:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826359.329444:0:138695:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635180 x1874663043035520/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826359.329486:0:138695:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc3800 lustre-OST0000_UUID: changing import state from FULL to CONNECTING 00000100:00080000:3.0:1787826359.329490:0:138695:0:(import.c:2101:ptlrpc_disconnect_and_idle_import()) lustre-OST0000-osc-ffff9181c5620000: disconnect after 23s idle 00000100:00080000:3.0:1787826359.329502:0:138695:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc6000 lustre-OST0001_UUID: changing import state from FULL to CONNECTING 00000100:00080000:3.0:1787826359.329505:0:138695:0:(import.c:2101:ptlrpc_disconnect_and_idle_import()) lustre-OST0001-osc-ffff9181c5620000: disconnect after 23s idle 00000100:00100000:0.0:1787826359.329801:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043035520:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787826359.329843:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043035648:192.168.203.147@tcp:9:kworker.0 00000100:00100000:0.0:1787826359.329872:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043035776:192.168.203.147@tcp:9:kworker.0 00000100:00080000:1.0:1787826359.334514:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787826359.334532:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-19) 00000100:00100000:1.0:1787826359.334538:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043035520:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826359.334802:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663043035392:192.168.203.147@tcp:400:kworker.0 00000100:00080000:0.0:1787826359.337309:0:2388:0:(import.c:2011:ptlrpc_disconnect_idle_interpret()) @@@ inflight=1, refcount=3: rc = 0 req@ffff9181c5634000 x1874663043035648/t0(0) o9->lustre-OST0000-osc-ffff9181c5620000@192.168.203.147@tcp:28/4 lens 224/224 e 0 to 0 dl 1787826369 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826359.337325:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc3800 lustre-OST0000_UUID: changing import state from CONNECTING to IDLE 00000100:00100000:0.0:1787826359.337331:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043035648:192.168.203.147@tcp:9:kworker.0 00000100:00080000:0.0:1787826359.337348:0:2388:0:(import.c:2011:ptlrpc_disconnect_idle_interpret()) @@@ inflight=1, refcount=3: rc = 0 req@ffff9181c5634a80 x1874663043035776/t0(0) o9->lustre-OST0001-osc-ffff9181c5620000@192.168.203.147@tcp:28/4 lens 224/224 e 0 to 0 dl 1787826369 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826359.337355:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc6000 lustre-OST0001_UUID: changing import state from CONNECTING to IDLE 00000100:00100000:0.0:1787826359.337358:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043035776:192.168.203.147@tcp:9:kworker.0 00000100:00100000:3.0:1787826359.337967:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:f40147eb-9b53-47d4-b77c-99ceedd9664c:2392:1874663043035264:192.168.203.147@tcp:400:kworker.0 00010000:00010000:1.0:1787826359.905049:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826360.931217:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826361.953966:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826362.977188:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826364.001045:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787826364.449119:0:138695:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826364.449126:0:138695:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787826364.449130:0:138695:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826364.449142:0:138695:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8431 00000100:00080000:1.0:1787826364.449152:0:138695:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 15s 00000100:00080000:1.0:1787826364.449159:0:138695:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787826364.449171:0:138695:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787826364.449189:0:138695:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3b80 x1874663043036160/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826364.449554:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043036160:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787826364.449905:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:f40147eb-9b53-47d4-b77c-99ceedd9664c:2389:1874663043035904:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826364.449966:0:2389:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1787826364.449973:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663043036032:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826364.453354:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:f40147eb-9b53-47d4-b77c-99ceedd9664c:2389:1874663043035904:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826364.454324:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663043036032:192.168.203.147@tcp:400:kworker.0 00000100:00080000:2.0:1787826364.458460:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826364.458653:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-19) 00000100:00100000:2.0:1787826364.458660:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043036160:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787826365.035790:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826366.059242:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826367.075843:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826368.098484:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826369.121441:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826369.570692:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:f40147eb-9b53-47d4-b77c-99ceedd9664c:2391:1874663043036288:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826369.573023:0:2391:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787826369.573047:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663043036416:192.168.203.147@tcp:400:kworker.0 00000100:00080000:3.0:1787826369.575650:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826369.575665:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787826369.575669:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826369.575679:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8436 00000100:00080000:3.0:1787826369.575688:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826369.575695:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826369.575706:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826369.575735:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637b80 x1874663043036544/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826369.576220:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043036544:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826369.580961:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663043036416:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826369.580979:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:f40147eb-9b53-47d4-b77c-99ceedd9664c:2391:1874663043036288:192.168.203.147@tcp:400:kworker.0 00000100:00080000:2.0:1787826369.585166:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826369.585194:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-19) 00000100:00100000:2.0:1787826369.585199:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043036544:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787826370.145600:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826370.546322:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:1.0:1787826370.546669:0:159722:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049422848 00000100:00100000:1.0:1787826370.546680:0:159722:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049422848 00000100:00100000:1.0:1787826370.546739:0:159722:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 5 00000100:00100000:1.0:1787826370.546791:0:159722:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:f40147eb-9b53-47d4-b77c-99ceedd9664c+3:196432:x1874663049422848:12345-192.168.203.147@tcp:104: 00010000:00010000:1.0:1787826370.546807:0:159722:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f22e lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1400000000000 nid: local remote: 0x878a5eea00cedc50 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826370.547128:0:159722:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1787826370.547136:0:159722:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:f40147eb-9b53-47d4-b77c-99ceedd9664c+3:196432:x1874663049422848:12345-192.168.203.147@tcp:104: Request processed in 349us (828us total) trans 0 rc 0/0 00000100:00100000:1.0:1787826370.547144:0:159722:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 5 00010000:00010000:2.0:1787826370.547326:0:159732:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826370.547333:0:159732:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f22e lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1400000000000 nid: local remote: 0x878a5eea00cedc50 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826370.547341:0:159732:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4800 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:2.0:1787826370.547346:0:159732:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f22e lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1400400000000 nid: local remote: 0x878a5eea00cedc50 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826370.547354:0:159732:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f22e lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1408400000000 nid: local remote: 0x878a5eea00cedc50 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1787826370.547367:0:159732:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f22e lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1409400000000 nid: local remote: 0x878a5eea00cedc50 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787826370.547373:0:159732:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:2.0:1787826370.547417:0:159732:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f22e lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0x878a5eea00cedc50 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826370.547424:0:159732:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826370.547455:0:159732:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f22e lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0x878a5eea00cedc50 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826370.547461:0:159732:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f22e lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0x878a5eea00cedc50 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826370.547478:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826370.547480:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826370.547481:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826370.547489:0:2392:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1787826370.547502:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:f40147eb-9b53-47d4-b77c-99ceedd9664c:2392:1874663043036672:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:2.0:1787826370.561914:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:f40147eb-9b53-47d4-b77c-99ceedd9664c:2392:1874663043036672:192.168.203.147@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1787826371.169885:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826372.217016:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787826372.576853:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:1.0:1787826372.576862:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff9181c5620000 from 340 10000000:01000000:1.0:1787826372.576870:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:1.0:1787826372.576918:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f38c lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826372.576930:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f38c lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826372.576940:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f38c lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826372.576966:0:159735:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043036800:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00100000:1.0:1787826372.577105:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181e0156780 going to sleep for 11 seconds 00000100:00100000:1.0:1787826372.585479:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043036800:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787826372.585512:0:159735:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f38c lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00cee7f6 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826372.585524:0:159735:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f38c lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x878a5eea00cee7f6 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826372.585611:0:159735:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043036928:192.168.203.147@tcp:256:ll_cfg_requeue.0 00000100:00100000:1.0:1787826372.585698:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181e0156780 going to sleep for 11 seconds 00000100:00100000:1.0:1787826372.593326:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043036928:192.168.203.147@tcp:256:ll_cfg_requeue.0 00000100:00080000:1.0:1787826372.593411:0:159735:0:(import.c:214:ptlrpc_set_import_discon()) mdc: import ffff9181c5bc2800 already not connected (conn 6, was 0): DISCONN 00010000:00080000:1.0:1787826372.593429:0:159735:0:(ldlm_lib.c:85:import_set_conn()) imp ffff9181c5bc2800@lustre-MDT0001-mdc-ffff9181c5620000: found existing conn 192.168.203.147@tcp, moved to head 00000100:00080000:1.0:1787826372.593436:0:159735:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826372.593442:0:159735:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 0 00000100:00080000:1.0:1787826372.593452:0:159735:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787826372.593466:0:159735:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787826372.593504:0:159735:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3b80 x1874663043037056/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826372.593635:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043037056:192.168.203.147@tcp:38:ll_cfg_requeue.0 10000000:01000000:1.0:1787826372.594824:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-cliir' succeeded (0). 00010000:00010000:1.0:1787826372.594845:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f38c lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00cee7f6 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826372.594859:0:159735:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f38c lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x878a5eea00cee7f6 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00080000:2.0:1787826372.599246:0:2388:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffff9181c35b3b80 x1874663043037056/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/416 e 0 to 0 dl 1787826382 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826372.599277:0:2388:0:(import.c:1132:ptlrpc_connect_interpret()) lustre-MDT0001-mdc-ffff9181c5620000: connect to target with instance 5 00000100:00080000:2.0:1787826372.599304:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to REPLAY 00000100:00080000:2.0:1787826372.599312:0:2388:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0001_UUID 00000100:00100000:2.0:1787826372.599315:0:2388:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff9181c5620000: committing for last_committed 17179870055 gen 1 00000100:00080000:2.0:1787826372.599333:0:2388:0:(recover.c:53:ptlrpc_replay_next()) import ffff9181c5bc2800 from lustre-MDT0001_UUID committed 17179870055 last 0 00000100:00080000:2.0:1787826372.599338:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from REPLAY to REPLAY_LOCKS 00000100:00100000:0.0:1787826372.601970:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043037056:192.168.203.147@tcp:38:ll_cfg_requeue.0 00010000:00080000:2.0:1787826372.602806:0:162522:0:(ldlm_request.c:2972:ldlm_lock_replay_thread()) lock replay thread lustre-MDT0001-mdc-ffff9181c5620000 to lustre-MDT0001_UUID@192.168.203.147@tcp 00010000:00010000:2.0:1787826372.602815:0:162522:0:(ldlm_request.c:2888:ldlm_cancel_unused_locks_for_replay()) Dropping as many unused locks as possible before replay for namespace lustre-MDT0001-mdc-ffff9181c5620000 (0) 00010000:00010000:2.0:1787826372.602822:0:162522:0:(ldlm_request.c:2901:ldlm_cancel_unused_locks_for_replay()) Canceled 0 unused locks from namespace lustre-MDT0001-mdc-ffff9181c5620000 00000100:00080000:2.0:1787826372.604122:0:162522:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from REPLAY_LOCKS to REPLAY_WAIT 00000100:00100000:0.0:1787826372.604716:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043037184:192.168.203.147@tcp:400:ldlm_lock_repla.0 00010000:00010000:1.0:1787826373.217091:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826374.241317:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787826374.689063:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level REPLAY_WAIT/7 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826374.689072:0:160087:0:(pinger.c:247:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: not pinging (in recovery or recovery disabled: REPLAY_WAIT) 00000100:00100000:2.0:1787826374.689111:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663043037440:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826374.689394:0:2391:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787826374.689402:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:f40147eb-9b53-47d4-b77c-99ceedd9664c:2391:1874663043037312:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826374.708149:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:f40147eb-9b53-47d4-b77c-99ceedd9664c:2391:1874663043037312:192.168.203.147@tcp:400:kworker.0 00010000:00010000:2.0:1787826374.712989:0:2391:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 0/0 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1787826374.713017:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663043037440:192.168.203.147@tcp:400:kworker.0 00010000:00010000:1.0:1787826374.713538:0:159733:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 0 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1787826374.713568:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826374.713569:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826374.713580:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826375.264987:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00080000:0.0:1787826375.833932:0:2388:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffff9181d0fc0380 x1874663043037184/t0(0) o400->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 224/224 e 0 to 0 dl 1787826402 ref 1 fl Rpc:RQU/2c0/0 rc 0/0 job:'ldlm_lock_repla.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826375.833958:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from REPLAY_WAIT to RECOVER 00000100:00080000:0.0:1787826375.833965:0:2388:0:(client.c:3247:ptlrpc_resend_req()) @@@ going to resend req@ffff9181d0fc0380 x1874663043037184/t0(0) o400->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 224/224 e 0 to 0 dl 1787826402 ref 1 fl Interpret:RQU/2c0/0 rc 0/0 job:'ldlm_lock_repla.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826375.833973:0:2388:0:(client.c:3256:ptlrpc_resend_req()) @@@ it has reply, so skip it req@ffff9181d0fc0380 x1874663043037184/t0(0) o400->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 224/224 e 0 to 0 dl 1787826402 ref 1 fl Interpret:RQU/2c0/0 rc 0/0 job:'ldlm_lock_repla.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826375.833980:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from RECOVER to FULL 00000100:02000000:0.0:1787826375.846734:0:2388:0:(import.c:1743:ptlrpc_import_recovery_state_machine()) lustre-MDT0001-mdc-ffff9181c5620000: Connection restored to 192.168.203.147@tcp (at 192.168.203.147@tcp) 00000100:00100000:0.0:1787826375.846764:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043037184:192.168.203.147@tcp:400:ldlm_lock_repla.0 00010000:00010000:1.0:1787826376.289256:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826377.313579:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826377.698117:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:f40147eb-9b53-47d4-b77c-99ceedd9664c:2392:1874663043037568:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826377.698252:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663043037824:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826377.698292:0:2392:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1787826377.698298:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663043037696:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826377.702030:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663043037696:192.168.203.147@tcp:400:kworker.0 00010000:00080000:2.0:1787826377.702056:0:2392:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffff9181c5637100 x1874663043037824/t0(0) o400->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 224/224 e 0 to 0 dl 1787826407 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826377.702075:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663043037824:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826377.712375:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:f40147eb-9b53-47d4-b77c-99ceedd9664c:2392:1874663043037568:192.168.203.147@tcp:400:kworker.0 00010000:00010000:1.0:1787826378.341069:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826379.368906:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200008:2.0:1787826379.787588:0:162580:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181d080cb08), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1787826379.787606:0:162580:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:2.0:1787826379.787629:0:162580:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826379.787694:0:162580:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 162580 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826379.787704:0:162580:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 162580 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826379.787715:0:162580:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 162580 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826379.787741:0:162580:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc2680 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162580:1874663043037952:192.168.203.147@tcp:101:lfs.0 00000100:00100000:2.0:1787826379.787927:0:162580:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c32f6000 going to sleep for 25 seconds 00000100:00100000:2.0:1787826379.796361:0:162580:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc2680 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162580:1874663043037952:192.168.203.147@tcp:101:lfs.0 00010000:00010000:2.0:1787826379.796403:0:162580:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826379.796420:0:162580:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826379.796434:0:162580:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826379.796451:0:162580:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826379.796462:0:162580:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826379.796467:0:162580:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826379.796471:0:162580:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787826379.796480:0:162580:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181d0fc2680 x1874663043037952/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826409 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787826379.796489:0:162580:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826379.796498:0:162580:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d1a77c00 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:2.0:1787826379.796503:0:162580:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:2.0:1787826379.796527:0:162580:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 69632 00000080:00010000:2.0:1787826379.796531:0:162580:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f393 00000080:00010000:2.0:1787826379.796534:0:162580:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:2.0:1787826379.796550:0:162580:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f393 from it ffffb97502ccfcb8 00010000:00010000:2.0:1787826379.796553:0:162580:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826379.796562:0:162580:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:2.0:1787826379.797240:0:162580:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181d080cb08), request_mask 0, flags 0, foreign 0 00000080:00200000:2.0:1787826379.797245:0:162580:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:2.0:1787826379.797262:0:162580:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826379.797272:0:162580:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826379.797283:0:162580:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826379.797290:0:162580:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826379.797295:0:162580:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:2.0:1787826379.797298:0:162580:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f393 00000080:00010000:2.0:1787826379.797302:0:162580:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f393 from it ffffb97502ccfcb8 00010000:00010000:2.0:1787826379.797306:0:162580:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826379.797314:0:162580:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1787826379.797396:0:162580:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:2.0:1787826379.797403:0:162580:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826379.797408:0:162580:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787826379.797417:0:162580:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1787826379.797421:0:162580:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826379.797424:0:162580:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:2.0:1787826379.797426:0:162580:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f393 00000080:00010000:2.0:1787826379.797428:0:162580:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f393 from it ffffb97502ccfbb0 00010000:00010000:2.0:1787826379.797430:0:162580:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787826379.797434:0:162580:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:2.0:1787826379.797439:0:162580:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 44 00000080:00200008:2.0:1787826379.797445:0:162580:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9181d080cb08), flags 100000 00000080:00200080:2.0:1787826379.798013:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.798027:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00800000:00000004:2.0:1787826379.798032:0:162580:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9181c5620000: age 8408, max_age 8450 00000100:00100000:2.0:1787826379.798063:0:162580:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc2680 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162580:1874663043038080:192.168.203.147@tcp:41:lfs.0 00000100:00100000:2.0:1787826379.798164:0:162580:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c32f6000 going to sleep for 11 seconds 00000100:00100000:2.0:1787826379.806069:0:162580:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc2680 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162580:1874663043038080:192.168.203.147@tcp:41:lfs.0 00800000:00000004:2.0:1787826379.806100:0:162580:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0000-mdc-ffff9181c5620000: update ffff9181c354d178 cache blocks 317223/353529 objects 1023644/1024000 00000080:00200080:2.0:1787826379.806182:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.806200:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00800000:00000004:2.0:1787826379.806208:0:162580:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9181c5620000: age 8408, max_age 8450 00000100:00100000:2.0:1787826379.806242:0:162580:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc2680 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162580:1874663043038208:192.168.203.147@tcp:41:lfs.0 00000100:00100000:2.0:1787826379.806674:0:162580:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c32f6000 going to sleep for 25 seconds 00010000:00080000:2.0:1787826379.812490:0:162580:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffff9181d0fc2680 x1874663043038208/t0(0) o41->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 224/368 e 0 to 0 dl 1787826409 ref 2 fl Rpc:RQU/200/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826379.812511:0:162580:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc2680 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162580:1874663043038208:192.168.203.147@tcp:41:lfs.0 00800000:00000004:2.0:1787826379.812534:0:162580:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0001-mdc-ffff9181c5620000: update ffff9181c3548450 cache blocks 317326/353529 objects 1023702/1024000 00000080:00200080:2.0:1787826379.812710:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.812726:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.812735:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.812739:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.812771:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.812778:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.812783:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.812786:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.812790:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.812794:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.812798:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.812801:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.812804:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.812807:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.812811:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.812814:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.812817:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.812821:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.812826:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.812829:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.812833:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.812837:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.812841:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.812844:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.812849:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.812852:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.812856:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.812859:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.812863:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.812866:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.812870:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.812873:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.812877:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.812880:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.812885:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.812888:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.812892:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.812895:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.812899:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.812903:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.812906:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.812910:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.812913:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.812918:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.812922:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.812925:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.812930:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.812933:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.812937:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.812941:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.812944:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.812947:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.812950:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.812953:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.812956:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.812959:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.812962:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.812966:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.812969:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.812972:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.812975:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.812978:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.812982:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.812986:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.812989:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.812993:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.812997:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.813000:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.813004:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.813007:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.813011:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.813014:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.813017:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.813021:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.813024:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.813027:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.813032:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.813035:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.813040:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.813043:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.813047:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.813050:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.813054:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.813057:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.813060:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.813063:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.813067:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.813071:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.813074:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.813077:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.813080:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.813083:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.813087:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.813091:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.813095:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.813098:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.813102:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.813106:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.813109:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.813113:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.813116:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.813120:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.813124:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.813128:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.813131:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.813135:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.813139:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.813142:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.813146:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.813149:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.813153:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.813156:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.813159:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.813162:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.813166:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.813169:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.813172:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.813175:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.813179:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.813182:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.813185:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.813189:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.813192:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.813196:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.813200:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00800000:00000080:2.0:1787826379.813204:0:162580:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.813208:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.813214:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00020000:00000004:2.0:1787826379.813222:0:162580:0:(obd_class.h:1090:obd_statfs()) lustre-OST0000-osc-ffff9181c5620000: age 8408, max_age 8450 00000100:00080000:2.0:1787826379.813236:0:162580:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8381 00000100:00080000:2.0:1787826379.813248:0:162580:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-ffff9181c5620000: import ffff9181c5bc3800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826379.813269:0:162580:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683216 ocd_grant: 8437760, lost: 0. 00000100:00100000:2.0:1787826379.813292:0:162580:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181d0fc1c00 x1874663043038336/t0(0) o8->lustre-OST0000-osc-ffff9181c5620000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787826379.813331:0:162580:0:(client.c:1845:ptlrpc_send_new_req()) @@@ req waiting for recovery: (FULL != CONNECTING) req@ffff9181d0fc2680 x1874663043038464/t0(0) o13->lustre-OST0000-osc-ffff9181c5620000@192.168.203.147@tcp:7/4 lens 224/368 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826379.813345:0:162580:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c32f6000 going to sleep for 0 seconds 00000100:00100000:0.0:1787826379.813427:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043038336:192.168.203.147@tcp:8:lfs.0 00000100:00100000:0.0:1787826379.823630:0:2388:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-ffff9181c5620000: committing for last_committed 38654705666 gen 3 00000100:00080000:0.0:1787826379.823647:0:2388:0:(import.c:1132:ptlrpc_connect_interpret()) lustre-OST0000-osc-ffff9181c5620000: connect to target with instance 9 00000100:00080000:0.0:1787826379.823656:0:2388:0:(import.c:956:ptlrpc_connect_set_flags()) lustre-OST0000-osc-ffff9181c5620000: Resetting ns_connect_flags to server flags: 0xa0427af2e3440038 00000100:00080000:0.0:1787826379.823667:0:2388:0:(import.c:1213:ptlrpc_connect_interpret()) connected to replayable target: lustre-OST0000_UUID 00000100:00080000:0.0:1787826379.823671:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc3800 lustre-OST0000_UUID: changing import state from CONNECTING to FULL 00000080:00000004:0.0:1787826379.823680:0:2388:0:(lcommon_misc.c:94:cl_ocd_update()) Changing connect_flags: 0xa0427af2e3440038 -> 0xa0427af2e3440038 00000080:00080000:0.0:1787826379.823685:0:2388:0:(lcommon_misc.c:56:cl_init_ea_size()) updating def/max_easize: 72/65536 00000100:00080000:0.0:1787826379.823698:0:2388:0:(recover.c:191:ptlrpc_wake_delayed()) @@@ waking (set 000000002714f290): req@ffff9181d0fc2680 x1874663043038464/t0(0) o13->lustre-OST0000-osc-ffff9181c5620000@192.168.203.147@tcp:7/4 lens 224/368 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826379.823725:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043038336:192.168.203.147@tcp:8:lfs.0 00000100:00100000:2.0:1787826379.823952:0:162580:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c32f6000 going to sleep for 11 seconds 00000100:00100000:2.0:1787826379.827341:0:162580:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc2680 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162580:1874663043038464:192.168.203.147@tcp:13:lfs.0 00020000:00000004:2.0:1787826379.827372:0:162580:0:(obd_class.h:1116:obd_statfs()) lustre-OST0000-osc-ffff9181c5620000: update ffff9181c3549e08 cache blocks 901356/958279 objects 261710/262144 00000080:00200080:2.0:1787826379.827438:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.827456:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00020000:00000004:2.0:1787826379.827464:0:162580:0:(obd_class.h:1090:obd_statfs()) lustre-OST0001-osc-ffff9181c5620000: age 8408, max_age 8450 00000100:00080000:2.0:1787826379.827482:0:162580:0:(import.c:510:import_select_connection()) lustre-OST0001-osc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8381 00000100:00080000:2.0:1787826379.827495:0:162580:0:(import.c:624:import_select_connection()) lustre-OST0001-osc-ffff9181c5620000: import ffff9181c5bc6000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826379.827505:0:162580:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683216 ocd_grant: 8437760, lost: 0. 00000100:00100000:2.0:1787826379.827529:0:162580:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181d0fc3800 x1874663043038592/t0(0) o8->lustre-OST0001-osc-ffff9181c5620000@192.168.203.147@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826379.827623:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043038592:192.168.203.147@tcp:8:lfs.0 00000100:00080000:2.0:1787826379.829114:0:162580:0:(client.c:1845:ptlrpc_send_new_req()) @@@ req waiting for recovery: (FULL != CONNECTING) req@ffff9181d0fc2680 x1874663043038720/t0(0) o13->lustre-OST0001-osc-ffff9181c5620000@192.168.203.147@tcp:7/4 lens 224/368 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826379.829125:0:162580:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c32f6000 going to sleep for 0 seconds 00000100:00100000:0.0:1787826379.837445:0:2388:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-ffff9181c5620000: committing for last_committed 0 gen 3 00000100:00080000:0.0:1787826379.837469:0:2388:0:(import.c:1132:ptlrpc_connect_interpret()) lustre-OST0001-osc-ffff9181c5620000: connect to target with instance 1 00000100:00080000:0.0:1787826379.837479:0:2388:0:(import.c:956:ptlrpc_connect_set_flags()) lustre-OST0001-osc-ffff9181c5620000: Resetting ns_connect_flags to server flags: 0xa0427af2e3440038 00000100:00080000:0.0:1787826379.837491:0:2388:0:(import.c:1213:ptlrpc_connect_interpret()) connected to replayable target: lustre-OST0001_UUID 00000100:00080000:0.0:1787826379.837495:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc6000 lustre-OST0001_UUID: changing import state from CONNECTING to FULL 00000080:00000004:0.0:1787826379.837504:0:2388:0:(lcommon_misc.c:94:cl_ocd_update()) Changing connect_flags: 0xa0427af2e3440038 -> 0xa0427af2e3440038 00000080:00080000:0.0:1787826379.837510:0:2388:0:(lcommon_misc.c:56:cl_init_ea_size()) updating def/max_easize: 72/65536 00000100:00080000:0.0:1787826379.837524:0:2388:0:(recover.c:191:ptlrpc_wake_delayed()) @@@ waking (set 000000002714f290): req@ffff9181d0fc2680 x1874663043038720/t0(0) o13->lustre-OST0001-osc-ffff9181c5620000@192.168.203.147@tcp:7/4 lens 224/368 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826379.837552:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043038592:192.168.203.147@tcp:8:lfs.0 00000100:00100000:2.0:1787826379.837655:0:162580:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c32f6000 going to sleep for 11 seconds 00000100:00100000:2.0:1787826379.847112:0:162580:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc2680 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:162580:1874663043038720:192.168.203.147@tcp:13:lfs.0 00020000:00000004:2.0:1787826379.847142:0:162580:0:(obd_class.h:1116:obd_statfs()) lustre-OST0001-osc-ffff9181c5620000: update ffff9181c354b7c0 cache blocks 901360/958279 objects 261727/262144 00000080:00200080:2.0:1787826379.847217:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847237:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847246:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847251:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847256:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847260:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847265:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847270:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847274:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847279:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847284:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847288:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847292:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847296:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847300:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847304:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847308:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847312:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847318:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847322:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847327:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847331:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847336:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847340:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847344:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847348:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847352:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847356:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847361:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847365:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847370:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847375:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847379:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847383:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847388:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847392:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847397:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847402:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847407:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847411:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847416:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847420:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847424:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847429:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847435:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847439:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847448:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847453:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847458:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847462:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847467:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847472:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847476:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847482:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847487:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847491:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847500:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847504:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847510:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847515:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847521:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847524:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847529:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847533:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847549:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847554:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847558:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847563:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847570:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847575:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847582:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847586:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847591:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847595:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847598:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847602:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847606:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847610:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847614:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847618:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847622:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847625:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847629:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847633:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847637:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847641:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847645:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847649:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847653:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847657:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847661:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847665:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847669:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847673:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847677:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847680:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847684:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847688:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847692:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847696:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847700:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847703:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847707:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847710:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847715:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847718:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847722:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847726:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847730:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847733:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847737:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847741:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847793:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847799:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847805:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847812:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847816:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847821:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847826:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847831:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847836:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847841:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847846:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847850:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200080:2.0:1787826379.847856:0:162580:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffd5c0f0e30 00020000:00000080:2.0:1787826379.847860:0:162580:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e2000 uarg=0000000000000000 00000080:00200008:2.0:1787826379.847874:0:162580:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9181d080cb08), flags 100000 00010000:00010000:1.0:1787826380.386024:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826381.409025:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826382.433057:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826382.881202:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043038976:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826382.881322:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043039232:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826382.881361:0:2390:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:0.0:1787826382.881368:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:f40147eb-9b53-47d4-b77c-99ceedd9664c:2390:1874663043038848:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826382.881404:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043039104:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826382.881426:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043039360:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826382.890564:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:f40147eb-9b53-47d4-b77c-99ceedd9664c:2390:1874663043038848:192.168.203.147@tcp:400:kworker.0 00010000:00080000:0.0:1787826382.890604:0:2390:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffff9181c55a4e00 x1874663043039104/t0(0) o400->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 224/224 e 0 to 0 dl 1787826398 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826382.890618:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043039104:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826382.890631:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043039360:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826382.890644:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043038976:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826382.890654:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043039232:192.168.203.147@tcp:400:kworker.0 00010000:00010000:3.0:1787826383.457833:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826384.488240:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826385.511483:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:1.0:1787826386.079567:0:162861:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: oleg347-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 00010000:00010000:3.0:1787826386.530061:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826387.559148:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:2.0:1787826387.854329:0:163009:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec 00000100:00100000:0.0:1787826388.001283:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:f40147eb-9b53-47d4-b77c-99ceedd9664c:2390:1874663043039488:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826388.001379:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043039744:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826388.001409:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043040000:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826388.001439:0:2390:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1787826388.001445:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043039616:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826388.001466:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043039872:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826388.012491:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043039616:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826388.012513:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043039872:192.168.203.147@tcp:400:kworker.0 00010000:00080000:1.0:1787826388.012529:0:2390:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffff9181c55a5500 x1874663043039744/t0(0) o400->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 224/224 e 0 to 0 dl 1787826404 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787826388.012541:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043039744:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826388.012553:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043040000:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826388.020379:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:f40147eb-9b53-47d4-b77c-99ceedd9664c:2390:1874663043039488:192.168.203.147@tcp:400:kworker.0 00010000:00010000:3.0:1787826388.577000:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826389.604084:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200008:3.0:1787826390.065564:0:163048:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181d080cb08), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1787826390.065581:0:163048:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:3.0:1787826390.065676:0:163048:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826390.065697:0:163048:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:3.0:1787826390.065713:0:163048:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:3.0:1787826390.065722:0:163048:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826390.065732:0:163048:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:3.0:1787826390.065736:0:163048:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f393 00000080:00010000:3.0:1787826390.065936:0:163048:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f393 from it ffffb9750150bcb8 00010000:00010000:3.0:1787826390.065941:0:163048:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:3.0:1787826390.065951:0:163048:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00000080:00200008:3.0:1787826390.066668:0:163048:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181d080cb08), request_mask 0, flags 0, foreign 0 00000080:00200000:3.0:1787826390.066676:0:163048:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:3.0:1787826390.066689:0:163048:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826390.066699:0:163048:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:3.0:1787826390.066711:0:163048:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00000080:00200000:3.0:1787826390.066723:0:163048:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826390.066728:0:163048:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:3.0:1787826390.066731:0:163048:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f393 00000080:00010000:3.0:1787826390.066736:0:163048:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f393 from it ffffb9750150bcb8 00010000:00010000:3.0:1787826390.066739:0:163048:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:3.0:1787826390.066817:0:163048:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00000080:00200000:3.0:1787826390.066923:0:163048:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:3.0:1787826390.066932:0:163048:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826390.066940:0:163048:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00010000:00010000:3.0:1787826390.066949:0:163048:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00000080:00200000:3.0:1787826390.066955:0:163048:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826390.066959:0:163048:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:3.0:1787826390.066962:0:163048:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f393 00000080:00010000:3.0:1787826390.066966:0:163048:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f393 from it ffffb9750150bbb0 00010000:00010000:3.0:1787826390.066969:0:163048:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00010000:00010000:3.0:1787826390.066976:0:163048:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00000080:00200000:3.0:1787826390.066985:0:163048:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 44 00000080:00200008:3.0:1787826390.066992:0:163048:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9181d080cb08), flags 100000 00000080:00200080:3.0:1787826390.067124:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:3.0:1787826390.067139:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00800000:00000004:3.0:1787826390.067146:0:163048:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9181c5620000: age 8451, max_age 8460 00000100:00100000:3.0:1787826390.067188:0:163048:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163048:1874663043040128:192.168.203.147@tcp:41:lfs.0 00000100:00100000:3.0:1787826390.067342:0:163048:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d22a2e40 going to sleep for 11 seconds 00000100:00100000:3.0:1787826390.078530:0:163048:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163048:1874663043040128:192.168.203.147@tcp:41:lfs.0 00800000:00000004:3.0:1787826390.078562:0:163048:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0000-mdc-ffff9181c5620000: update ffff9181c354d178 cache blocks 317223/353529 objects 1023644/1024000 00000080:00200080:3.0:1787826390.078790:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:3.0:1787826390.078808:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00800000:00000004:3.0:1787826390.078816:0:163048:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9181c5620000: age 8451, max_age 8460 00000100:00100000:3.0:1787826390.078849:0:163048:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163048:1874663043040256:192.168.203.147@tcp:41:lfs.0 00000100:00100000:3.0:1787826390.079234:0:163048:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d22a2e40 going to sleep for 11 seconds 00010000:00080000:3.0:1787826390.087915:0:163048:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffff9181c5635f80 x1874663043040256/t0(0) o41->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 224/368 e 0 to 0 dl 1787826406 ref 2 fl Rpc:RQU/200/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826390.087937:0:163048:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163048:1874663043040256:192.168.203.147@tcp:41:lfs.0 00800000:00000004:3.0:1787826390.087963:0:163048:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0001-mdc-ffff9181c5620000: update ffff9181c3548450 cache blocks 317326/353529 objects 1023702/1024000 00000080:00200080:2.0:1787826390.088237:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.088264:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.088273:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.088277:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.088281:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.088284:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.088288:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.088290:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.088295:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.088298:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.088302:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.088305:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.088308:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.088310:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.088314:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.088317:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.088320:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.088323:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.088326:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.088330:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.088334:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.088337:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.088340:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.088344:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.088347:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.088350:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.088354:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.088357:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.088361:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.088364:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.088368:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.088371:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.088375:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.088379:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.088383:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.088387:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.088390:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.088393:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.088397:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.088402:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.088405:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.088409:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.088413:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.088416:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.088420:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.088423:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.088427:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.088431:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.088435:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.088438:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.088442:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.088446:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.088450:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.088453:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.088457:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.088460:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.088463:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.088478:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.088483:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.088486:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.088490:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.088493:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.088497:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.088501:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.088505:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.088508:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.088512:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.088524:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.088531:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.088535:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.088539:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.088542:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.088546:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.088550:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.088554:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.088557:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.088561:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.088564:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.088568:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.088571:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.088575:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.088578:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.088582:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.088585:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.088589:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.088593:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.088596:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.088600:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.088605:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.088608:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.088612:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.088615:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.088619:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.088623:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.088627:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.088631:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.088634:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.088638:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.088642:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.088645:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.088649:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.088652:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.088656:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.088659:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.088663:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.088667:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.088671:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.088674:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.089124:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.089135:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.089140:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.089144:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.089147:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.089150:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.089154:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.089158:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.089161:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.089165:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.089168:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.089172:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.089176:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.089179:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.089182:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.089185:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.089188:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00800000:00000080:2.0:1787826390.089191:0:163048:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.089195:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.089201:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00020000:00000004:2.0:1787826390.089209:0:163048:0:(obd_class.h:1090:obd_statfs()) lustre-OST0000-osc-ffff9181c5620000: age 8451, max_age 8460 00000100:00100000:2.0:1787826390.089269:0:163048:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc2680 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163048:1874663043040384:192.168.203.147@tcp:13:lfs.0 00000100:00100000:2.0:1787826390.089361:0:163048:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6ea8b40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826390.094316:0:163048:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc2680 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163048:1874663043040384:192.168.203.147@tcp:13:lfs.0 00020000:00000004:2.0:1787826390.094348:0:163048:0:(obd_class.h:1116:obd_statfs()) lustre-OST0000-osc-ffff9181c5620000: update ffff9181c3549e08 cache blocks 901356/958279 objects 261710/262144 00000080:00200080:2.0:1787826390.094666:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.094687:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00020000:00000004:2.0:1787826390.094695:0:163048:0:(obd_class.h:1090:obd_statfs()) lustre-OST0001-osc-ffff9181c5620000: age 8451, max_age 8460 00000100:00100000:2.0:1787826390.094732:0:163048:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc2680 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163048:1874663043040512:192.168.203.147@tcp:13:lfs.0 00000100:00100000:2.0:1787826390.094865:0:163048:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6ea8b40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826390.101403:0:163048:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc2680 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163048:1874663043040512:192.168.203.147@tcp:13:lfs.0 00020000:00000004:2.0:1787826390.101433:0:163048:0:(obd_class.h:1116:obd_statfs()) lustre-OST0001-osc-ffff9181c5620000: update ffff9181c354b7c0 cache blocks 901360/958279 objects 261727/262144 00000080:00200080:2.0:1787826390.101548:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.101566:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.101574:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.101578:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.101583:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.101587:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.101591:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.101594:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.101598:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.101602:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.101606:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.101610:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.101613:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.101617:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.101621:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.101625:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.101629:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.101633:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.101637:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.101640:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.101644:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.101647:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.101651:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.101655:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.101659:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.101663:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.101667:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.101670:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.101674:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.101677:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.101692:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.101697:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.101701:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.101705:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.101710:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.101714:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.101718:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.101721:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.101725:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.101729:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.101733:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.101736:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.101785:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.101791:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.101797:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.101801:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.101805:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.101808:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.101812:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.101816:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.101820:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.101822:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.101826:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.101829:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.101832:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.101835:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.101839:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.101842:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.101845:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.101848:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.101853:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.101856:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.101860:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.101864:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.101868:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.101871:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.101875:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.101879:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.101883:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.101886:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.101889:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.101893:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.101896:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.101900:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.101903:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.101906:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.101909:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.101913:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.101917:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.101921:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.101925:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.101929:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.101933:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.101936:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.101940:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.101945:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.101949:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.101953:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.101957:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.101961:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.101965:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.101969:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.101974:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.101978:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.101982:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.101985:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.101990:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.101994:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.101998:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.102002:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.102006:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.102010:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.102014:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.102018:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.102022:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.102026:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.102030:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.102034:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.102038:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.102041:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.102045:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.102049:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.102053:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.102057:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.102061:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.102065:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.102068:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.102073:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.102077:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.102081:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.102085:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.102089:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.102092:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.102096:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200080:2.0:1787826390.102100:0:163048:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7fff8f6b5a10 00020000:00000080:2.0:1787826390.102104:0:163048:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c50e3000 uarg=0000000000000000 00000080:00200008:2.0:1787826390.102120:0:163048:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9181d080cb08), flags 100000 00000080:00200000:1.0:1787826390.141005:0:163065:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:1.0:1787826390.141033:0:163065:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826390.141053:0:163065:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00010000:00010000:1.0:1787826390.141070:0:163065:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00000080:00200000:1.0:1787826390.141077:0:163065:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826390.141084:0:163065:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:1.0:1787826390.141087:0:163065:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f393 00000080:00010000:1.0:1787826390.141095:0:163065:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f393 from it ffffb97500da3bf8 00010000:00010000:1.0:1787826390.141099:0:163065:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00010000:00010000:1.0:1787826390.141108:0:163065:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 0 00000080:00200000:1.0:1787826390.141129:0:163065:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826390.141136:0:163065:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80b.replay-single, flags=27 00000080:00200000:1.0:1787826390.141142:0:163065:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x4:0x0](ffff9181d0809148), inode mode 41ed mask 1 00000080:00200000:1.0:1787826390.141153:0:163065:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=remote_dir, dir=[0x200027cd1:0x4:0x0](ffff9181d0809148), file ffff9181d3b56d80, open_flags 18800, mode 0 opened 0 00000080:00200000:1.0:1787826390.141160:0:163065:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=remote_dir, dir=[0x200027cd1:0x4:0x0](ffff9181d0809148), intent=open 00000002:00010000:1.0:1787826390.141174:0:163065:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: remote_dir,[0x0:0x0:0x0]) in obj [0x200027cd1:0x4:0x0], intent: open flags 0304001 00010000:00010000:1.0:1787826390.141239:0:163065:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4600/0xbef42c0221d6f39a lrc: 3/1,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 163065 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.141246:0:163065:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4600/0xbef42c0221d6f39a lrc: 3/1,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 163065 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:1.0:1787826390.143806:0:163065:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:1.0:1787826390.143826:0:163065:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c35b1f80 x1874663043040768/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 656/131856 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:1.0:1787826390.143839:0:163065:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4600/0xbef42c0221d6f39a lrc: 3/1,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 163065 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826390.143863:0:163065:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163065:1874663043040768:192.168.203.147@tcp:101:lfs.0 00000100:00100000:1.0:1787826390.145424:0:163065:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181e0156f00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826390.151792:0:163065:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163065:1874663043040768:192.168.203.147@tcp:101:lfs.0 00010000:00010000:2.0:1787826390.151822:0:163065:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4600/0xbef42c0221d6f39a lrc: 4/1,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee970 expref: -99 pid: 163065 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.151841:0:163065:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4600/0xbef42c0221d6f39a lrc: 4/1,0 mode: --/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee970 expref: -99 pid: 163065 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.151849:0:163065:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4600/0xbef42c0221d6f39a lrc: 4/1,0 mode: --/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee970 expref: -99 pid: 163065 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.151859:0:163065:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4600/0xbef42c0221d6f39a lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee970 expref: -99 pid: 163065 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.151871:0:163065:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4600/0xbef42c0221d6f39a lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee970 expref: -99 pid: 163065 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.151884:0:163065:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4600/0xbef42c0221d6f39a lrc: 5/2,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee970 expref: -99 pid: 163065 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.151893:0:163065:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4600/0xbef42c0221d6f39a lrc: 5/2,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee970 expref: -99 pid: 163065 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.151900:0:163065:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4600/0xbef42c0221d6f39a lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee970 expref: -99 pid: 163065 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787826390.151913:0:163065:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x1 disposition=0xb, status=0 req@ffff9181c35b1f80 x1874663043040768/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 656/608 e 0 to 0 dl 1787826406 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787826390.151927:0:163065:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4600/0xbef42c0221d6f39a lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee970 expref: -99 pid: 163065 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.151936:0:163065:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d1a77c00 type 13 mode 4 res 9663684441/22 (0 0) 00000002:00010000:2.0:1787826390.151950:0:163065:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200027cd1:0x4:0x0]) in obj [0x240001f59:0x16:0x0], intent: open flags 0304001 00010000:00010000:2.0:1787826390.151996:0:163065:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842c00/0xbef42c0221d6f3a1 lrc: 3/1,0 mode: --/CR res: [0x240001f59:0x16:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 163065 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.152003:0:163065:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842c00/0xbef42c0221d6f3a1 lrc: 3/1,0 mode: --/CR res: [0x240001f59:0x16:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163065 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1787826390.152014:0:163065:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1787826390.152020:0:163065:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181d0fc2680 x1874663043041024/t0(0) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/131856 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1787826390.152028:0:163065:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842c00/0xbef42c0221d6f3a1 lrc: 3/1,0 mode: --/CR res: [0x240001f59:0x16:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163065 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826390.152047:0:163065:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc2680 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163065:1874663043041024:192.168.203.147@tcp:101:lfs.0 00000100:00100000:2.0:1787826390.152531:0:163065:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6ea8b40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826390.206631:0:163065:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc2680 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163065:1874663043041024:192.168.203.147@tcp:101:lfs.0 00010000:00010000:2.0:1787826390.206658:0:163065:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842c00/0xbef42c0221d6f3a1 lrc: 4/1,0 mode: --/CR res: [0x240001f59:0x16:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163065 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.206673:0:163065:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842c00/0xbef42c0221d6f3a1 lrc: 4/1,0 mode: --/CR res: [0x240001f59:0x16:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 163065 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.206692:0:163065:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842c00/0xbef42c0221d6f3a1 lrc: 4/1,0 mode: --/CR res: [0x240001f59:0x16:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 163065 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.206700:0:163065:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842c00/0xbef42c0221d6f3a1 lrc: 3/0,0 mode: --/CR res: [0x240001f59:0x16:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 163065 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.206710:0:163065:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842c00/0xbef42c0221d6f3a1 lrc: 4/0,0 mode: --/CR res: [0x240001f59:0x16:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 163065 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.206716:0:163065:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181d1842c00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:2.0:1787826390.206721:0:163065:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842c00/0xbef42c0221d6f3a1 lrc: 4/0,0 mode: --/CR res: [0x240001f59:0x16:0x0].0x0 bits 0x1/0x1 rrc: 2 type: IBT flags: 0x206400000000 pid: 163065 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.206737:0:163065:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842c00/0xbef42c0221d6f3a1 lrc: 4/0,0 mode: --/CR res: [0x240001f59:0x16:0x0].0x0 bits 0x1/0x1 rrc: 2 type: IBT flags: 0x206400000000 pid: 163065 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.206771:0:163065:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842c00/0xbef42c0221d6f3a1 lrc: 5/0,0 mode: --/CR res: [0x240001f59:0x16:0x0].0x0 bits 0x1/0x1 rrc: 2 type: IBT flags: 0x20e400000000 pid: 163065 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.206779:0:163065:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787826390.206785:0:163065:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842c00/0xbef42c0221d6f3a1 lrc: 3/0,0 mode: --/CR res: [0x240001f59:0x16:0x0].0x0 bits 0x1/0x1 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 163065 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.206795:0:163065:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842c00/0xbef42c0221d6f3a1 lrc: 0/0,0 mode: --/CR res: [0x240001f59:0x16:0x0].0x0 bits 0x1/0x1 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 163065 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787826390.206818:0:163065:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x1 disposition=0x2b, status=0 req@ffff9181d0fc2680 x1874663043041024/t21474836481(21474836481) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00100000:2.0:1787826390.206831:0:163065:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181d0fc2680 x1874663043041024/t21474836481(21474836481) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1787826390.206888:0:163065:0:(namei.c:952:ll_intent_lock()) intent lock 1 on i1 [0x200027cd1:0x4:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1787826390.206918:0:163065:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x240001f59:0x16:0x0], updating i_size 4096 00000080:00200000:2.0:1787826390.206921:0:163065:0:(namei.c:147:ll_iget()) got inode: [0x240001f59:0x16:0x0](ffff9181d080f448): rc = 0 00000080:00010000:2.0:1787826390.206926:0:163065:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001f59:0x16:0x0](ffff9181d080f448) for lock 0xbef42c0221d6f39a 00000080:00010000:2.0:1787826390.206938:0:163065:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001f59:0x16:0x0](ffff9181d080f448) 00000080:00200008:2.0:1787826390.206949:0:163065:0:(file.c:1090:ll_file_open()) START file remote_dir:[0x240001f59:0x16:0x0](ffff9181d080f448), flags 304000 00000002:00100000:2.0:1787826390.206968:0:163065:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181d0fc2680 x1874663043041024/t21474836481(21474836481) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00010000:2.0:1787826390.207004:0:163065:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f39a from it ffff9181d1d8a780 00010000:00010000:2.0:1787826390.207009:0:163065:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4600/0xbef42c0221d6f39a lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee970 expref: -99 pid: 163065 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.207018:0:163065:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4600/0xbef42c0221d6f39a lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee970 expref: -99 pid: 163065 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826390.207037:0:163065:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 44 00000080:00200000:2.0:1787826390.207074:0:163065:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:2.0:1787826390.207088:0:163065:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826390.207101:0:163065:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 0 00010000:00010000:2.0:1787826390.207120:0:163065:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 0 00000080:00200000:2.0:1787826390.207126:0:163065:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826390.207131:0:163065:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:2.0:1787826390.207134:0:163065:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f393 00000080:00010000:2.0:1787826390.207137:0:163065:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f393 from it ffffb97500da3bf8 00010000:00010000:2.0:1787826390.207140:0:163065:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 0 00010000:00010000:2.0:1787826390.207147:0:163065:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00000080:00200000:2.0:1787826390.207156:0:163065:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.207161:0:163065:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80b.replay-single, flags=24 00000080:00200000:2.0:1787826390.207165:0:163065:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x4:0x0](ffff9181d0809148), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.207168:0:163065:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=remote_dir, flags=264 00000080:00200000:2.0:1787826390.207171:0:163065:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 44 00000080:00200008:2.0:1787826390.207174:0:163065:0:(file.c:1090:ll_file_open()) START file remote_dir:[0x240001f59:0x16:0x0](ffff9181d080f448), flags 504000 00000080:00200008:2.0:1787826390.207189:0:163065:0:(file.c:479:ll_file_release()) START file remote_dir:[0x240001f59:0x16:0x0](ffff9181d080f448), flags 304000 00000080:00200080:2.0:1787826390.207219:0:163065:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448) cmd=800466af arg=7ffdc1254eac 00000080:00200008:2.0:1787826390.207314:0:163065:0:(file.c:479:ll_file_release()) START file remote_dir:[0x240001f59:0x16:0x0](ffff9181d080f448), flags 504000 00000002:00100000:2.0:1787826390.207351:0:163065:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181d0fc2680 x1874663043041024/t21474836481(21474836481) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 0 dl 1787826406 ref 1 fl Complete:RPQU/604/ffffffff rc 301/-1 job:'lfs.0' uid:0 gid:0 projid:0 00000020:00100000:2.0:1787826390.207362:0:163065:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1787826390.207367:0:163065:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181d0fc1500 x1874663043041280/t0(0) o35->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:2.0:1787826390.207386:0:163065:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc1500 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163065:1874663043041280:192.168.203.147@tcp:35:lfs.0 00000100:00100000:2.0:1787826390.207453:0:163065:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6ea8b40 going to sleep for 11 seconds 00010000:00080000:2.0:1787826390.210783:0:163065:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffff9181d0fc1500 x1874663043041280/t0(0) o35->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/456 e 0 to 0 dl 1787826406 ref 2 fl Rpc:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000100:00100000:2.0:1787826390.210806:0:163065:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc1500 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163065:1874663043041280:192.168.203.147@tcp:35:lfs.0 00000002:00100000:2.0:1787826390.210838:0:163065:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181d0fc2680 x1874663043041024/t21474836481(21474836481) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 0 dl 1787826406 ref 1 fl Complete:RQU/604/ffffffff rc 301/-1 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787826390.222636:0:163071:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:1.0:1787826390.222965:0:163071:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826390.222984:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00010000:00010000:1.0:1787826390.223000:0:163071:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00000080:00200000:1.0:1787826390.223009:0:163071:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826390.223017:0:163071:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:1.0:1787826390.223020:0:163071:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f393 00000080:00010000:1.0:1787826390.223028:0:163071:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f393 from it ffffb97500e2bbf8 00010000:00010000:1.0:1787826390.223032:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00010000:00010000:1.0:1787826390.223040:0:163071:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00000080:00200000:1.0:1787826390.223056:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826390.223064:0:163071:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80b.replay-single, flags=25 00000080:00200000:1.0:1787826390.223069:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x4:0x0](ffff9181d0809148), inode mode 41ed mask 1 00000080:00200000:1.0:1787826390.223073:0:163071:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=remote_dir, flags=25 00000080:00200000:1.0:1787826390.223076:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 1 00000080:00200000:1.0:1787826390.223086:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 3 00000080:00200000:1.0:1787826390.223091:0:163071:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f-0, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), file ffff9181ca328600, open_flags 8042, mode 81a4 opened 0 00000080:00200000:1.0:1787826390.223097:0:163071:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f-0, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), intent=open|creat 00000002:00010000:1.0:1787826390.223119:0:163071:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f-0,[0x240001f59:0x17:0x0]) in obj [0x240001f59:0x16:0x0], intent: open|creat flags 0100103 00010000:00010000:1.0:1787826390.223166:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b4200/0xbef42c0221d6f3a8 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.223175:0:163071:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b4200/0xbef42c0221d6f3a8 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:1.0:1787826390.223187:0:163071:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:1.0:1787826390.223199:0:163071:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c35b3100 x1874663043041536/t0(0) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:1.0:1787826390.223209:0:163071:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b4200/0xbef42c0221d6f3a8 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826390.223228:0:163071:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043041536:192.168.203.147@tcp:101:createmany.0 00000100:00100000:1.0:1787826390.223363:0:163071:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181e01560c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826390.257697:0:163071:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043041536:192.168.203.147@tcp:101:createmany.0 00010000:00010000:1.0:1787826390.257727:0:163071:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b4200/0xbef42c0221d6f3a8 lrc: 4/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee9a8 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.257786:0:163071:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b4200/0xbef42c0221d6f3a8 lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x17:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee9a8 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.257797:0:163071:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b4200/0xbef42c0221d6f3a8 lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee9a8 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.257807:0:163071:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b4200/0xbef42c0221d6f3a8 lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee9a8 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.257820:0:163071:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b4200/0xbef42c0221d6f3a8 lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee9a8 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.257834:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b4200/0xbef42c0221d6f3a8 lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee9a8 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.257842:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b4200/0xbef42c0221d6f3a8 lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee9a8 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.257854:0:163071:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b4200/0xbef42c0221d6f3a8 lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee9a8 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826390.257865:0:163071:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9181c35b3100 x1874663043041536/t21474836483(21474836483) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:1.0:1787826390.257877:0:163071:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c35b3100 x1874663043041536/t21474836483(21474836483) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826390.257890:0:163071:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b4200/0xbef42c0221d6f3a8 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee9a8 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:1.0:1787826390.257901:0:163071:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b4200/0xbef42c0221d6f3a8 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee9a8 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.257910:0:163071:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c77f3000 type 13 mode 4 res 9663684441/23 (0 0) 00000080:00200000:1.0:1787826390.257917:0:163071:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x240001f59:0x16:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1787826390.257931:0:163071:0:(namei.c:1628:ll_create_it()) VFS Op:name=f-0, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), intent=open|creat 00000080:00200000:1.0:1787826390.257959:0:163071:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001f59:0x17:0x0](ffff9181c83e7448) 00000080:00200000:1.0:1787826390.258033:0:163071:0:(file.c:6843:ll_layout_conf()) [0x240001f59:0x17:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:1.0:1787826390.258037:0:163071:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x240001f59:0x17:0x0](ffff9181c83e7448) 00000080:00010000:1.0:1787826390.258040:0:163071:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001f59:0x17:0x0](ffff9181c83e7448) for lock 0xbef42c0221d6f3a8 00000080:00010000:1.0:1787826390.258049:0:163071:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001f59:0x17:0x0](ffff9181c83e7448) 00000080:00200008:1.0:1787826390.258061:0:163071:0:(file.c:1090:ll_file_open()) START file f-0:[0x240001f59:0x17:0x0](ffff9181c83e7448), flags 100102 00000002:00100000:1.0:1787826390.258082:0:163071:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c35b3100 x1874663043041536/t21474836483(21474836483) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:1.0:1787826390.258115:0:163071:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f3a8 from it ffff9181d1d8a180 00010000:00010000:1.0:1787826390.258120:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b4200/0xbef42c0221d6f3a8 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00cee9a8 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.258129:0:163071:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b4200/0xbef42c0221d6f3a8 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00cee9a8 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826390.258151:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x17:0x0](ffff9181c83e7448), inode mode 81a4 mask 40 00000080:00200008:1.0:1787826390.258176:0:163071:0:(file.c:479:ll_file_release()) START file f-0:[0x240001f59:0x17:0x0](ffff9181c83e7448), flags 100002 00000002:00100000:1.0:1787826390.258216:0:163071:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181c35b3100 x1874663043041536/t21474836483(21474836483) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:1.0:1787826390.258227:0:163071:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:1.0:1787826390.258232:0:163071:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c35b1500 x1874663043041792/t0(0) o35->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:1.0:1787826390.258252:0:163071:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043041792:192.168.203.147@tcp:35:createmany.0 00000100:00100000:1.0:1787826390.258341:0:163071:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181e0156b40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826390.270215:0:163071:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043041792:192.168.203.147@tcp:35:createmany.0 00000002:00100000:1.0:1787826390.270251:0:163071:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181c35b3100 x1874663043041536/t21474836483(21474836483) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787826390.270378:0:163071:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:1.0:1787826390.270396:0:163071:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826390.270416:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00010000:00010000:1.0:1787826390.270431:0:163071:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00000080:00200000:1.0:1787826390.270444:0:163071:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826390.270450:0:163071:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:1.0:1787826390.270453:0:163071:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f393 00000080:00010000:1.0:1787826390.270460:0:163071:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f393 from it ffffb97500e2bbf8 00010000:00010000:1.0:1787826390.270473:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00010000:00010000:1.0:1787826390.270484:0:163071:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00000080:00200000:1.0:1787826390.270500:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826390.270507:0:163071:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80b.replay-single, flags=25 00000080:00200000:1.0:1787826390.270513:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x4:0x0](ffff9181d0809148), inode mode 41ed mask 1 00000080:00200000:1.0:1787826390.270589:0:163071:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=remote_dir, flags=25 00000080:00200000:1.0:1787826390.270593:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 1 00000080:00200000:1.0:1787826390.270605:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 3 00000080:00200000:1.0:1787826390.270609:0:163071:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f-1, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), file ffff9181ca277c80, open_flags 8042, mode 81a4 opened 0 00000080:00200000:1.0:1787826390.270617:0:163071:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f-1, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), intent=open|creat 00000002:00010000:1.0:1787826390.270636:0:163071:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f-1,[0x240001f59:0x18:0x0]) in obj [0x240001f59:0x16:0x0], intent: open|creat flags 0100103 00010000:00010000:1.0:1787826390.270768:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b5200/0xbef42c0221d6f3af lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.270778:0:163071:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b5200/0xbef42c0221d6f3af lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:1.0:1787826390.270789:0:163071:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:1.0:1787826390.270796:0:163071:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c35b0380 x1874663043042048/t0(0) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:1.0:1787826390.270806:0:163071:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b5200/0xbef42c0221d6f3af lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826390.270825:0:163071:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043042048:192.168.203.147@tcp:101:createmany.0 00000100:00100000:1.0:1787826390.270934:0:163071:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181e0156b40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826390.286591:0:163071:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043042048:192.168.203.147@tcp:101:createmany.0 00010000:00010000:1.0:1787826390.286622:0:163071:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b5200/0xbef42c0221d6f3af lrc: 4/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee9cb expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.286653:0:163071:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b5200/0xbef42c0221d6f3af lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x18:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee9cb expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.286662:0:163071:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b5200/0xbef42c0221d6f3af lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee9cb expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.286673:0:163071:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b5200/0xbef42c0221d6f3af lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee9cb expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.286684:0:163071:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b5200/0xbef42c0221d6f3af lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee9cb expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.286698:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b5200/0xbef42c0221d6f3af lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee9cb expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.286706:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b5200/0xbef42c0221d6f3af lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee9cb expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.286718:0:163071:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b5200/0xbef42c0221d6f3af lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee9cb expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826390.286729:0:163071:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9181c35b0380 x1874663043042048/t21474836485(21474836485) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:1.0:1787826390.286771:0:163071:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c35b0380 x1874663043042048/t21474836485(21474836485) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826390.286787:0:163071:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b5200/0xbef42c0221d6f3af lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee9cb expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:1.0:1787826390.286800:0:163071:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b5200/0xbef42c0221d6f3af lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee9cb expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.286819:0:163071:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c77f3000 type 13 mode 4 res 9663684441/24 (0 0) 00000080:00200000:1.0:1787826390.286825:0:163071:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x240001f59:0x16:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1787826390.286839:0:163071:0:(namei.c:1628:ll_create_it()) VFS Op:name=f-1, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), intent=open|creat 00000080:00200000:1.0:1787826390.286869:0:163071:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001f59:0x18:0x0](ffff9181c83e21c8) 00000080:00200000:1.0:1787826390.286953:0:163071:0:(file.c:6843:ll_layout_conf()) [0x240001f59:0x18:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:1.0:1787826390.286958:0:163071:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x240001f59:0x18:0x0](ffff9181c83e21c8) 00000080:00010000:1.0:1787826390.286960:0:163071:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001f59:0x18:0x0](ffff9181c83e21c8) for lock 0xbef42c0221d6f3af 00000080:00010000:1.0:1787826390.286969:0:163071:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001f59:0x18:0x0](ffff9181c83e21c8) 00000080:00200008:1.0:1787826390.286982:0:163071:0:(file.c:1090:ll_file_open()) START file f-1:[0x240001f59:0x18:0x0](ffff9181c83e21c8), flags 100102 00000002:00100000:1.0:1787826390.287005:0:163071:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c35b0380 x1874663043042048/t21474836485(21474836485) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:1.0:1787826390.287035:0:163071:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f3af from it ffff9181d1d8a180 00010000:00010000:1.0:1787826390.287040:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b5200/0xbef42c0221d6f3af lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00cee9cb expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.287049:0:163071:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b5200/0xbef42c0221d6f3af lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00cee9cb expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826390.287070:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x18:0x0](ffff9181c83e21c8), inode mode 81a4 mask 40 00000080:00200008:1.0:1787826390.287099:0:163071:0:(file.c:479:ll_file_release()) START file f-1:[0x240001f59:0x18:0x0](ffff9181c83e21c8), flags 100002 00000002:00100000:1.0:1787826390.287140:0:163071:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181c35b0380 x1874663043042048/t21474836485(21474836485) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:1.0:1787826390.287151:0:163071:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:1.0:1787826390.287155:0:163071:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c35b1c00 x1874663043042304/t0(0) o35->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:1.0:1787826390.287174:0:163071:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043042304:192.168.203.147@tcp:35:createmany.0 00000100:00100000:1.0:1787826390.287245:0:163071:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181e0156c00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826390.292044:0:163071:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043042304:192.168.203.147@tcp:35:createmany.0 00000002:00100000:1.0:1787826390.292082:0:163071:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181c35b0380 x1874663043042048/t21474836485(21474836485) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787826390.292170:0:163071:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:1.0:1787826390.292189:0:163071:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826390.292207:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00010000:00010000:1.0:1787826390.292221:0:163071:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00000080:00200000:1.0:1787826390.292240:0:163071:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826390.292245:0:163071:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:1.0:1787826390.292249:0:163071:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f393 00000080:00010000:1.0:1787826390.292265:0:163071:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f393 from it ffffb97500e2bbf8 00010000:00010000:1.0:1787826390.292269:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00010000:00010000:1.0:1787826390.292278:0:163071:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 11 lru_type: 1 00000080:00200000:1.0:1787826390.292300:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826390.292308:0:163071:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80b.replay-single, flags=25 00000080:00200000:1.0:1787826390.292313:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x4:0x0](ffff9181d0809148), inode mode 41ed mask 1 00000080:00200000:1.0:1787826390.292316:0:163071:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=remote_dir, flags=25 00000080:00200000:1.0:1787826390.292318:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 1 00000080:00200000:1.0:1787826390.292330:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 3 00000080:00200000:1.0:1787826390.292334:0:163071:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f-2, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), file ffff9181c381d500, open_flags 8042, mode 81a4 opened 0 00000080:00200000:1.0:1787826390.292341:0:163071:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f-2, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), intent=open|creat 00000002:00010000:1.0:1787826390.292362:0:163071:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f-2,[0x240001f59:0x19:0x0]) in obj [0x240001f59:0x16:0x0], intent: open|creat flags 0100103 00010000:00010000:1.0:1787826390.292409:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b4000/0xbef42c0221d6f3b6 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.292417:0:163071:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b4000/0xbef42c0221d6f3b6 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:1.0:1787826390.292427:0:163071:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:1.0:1787826390.292433:0:163071:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c35b0000 x1874663043042560/t0(0) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:1.0:1787826390.292442:0:163071:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b4000/0xbef42c0221d6f3b6 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826390.292460:0:163071:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043042560:192.168.203.147@tcp:101:createmany.0 00000100:00100000:1.0:1787826390.292597:0:163071:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181e0156c00 going to sleep for 11 seconds 00000100:00100000:1.0:1787826390.303653:0:163071:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043042560:192.168.203.147@tcp:101:createmany.0 00010000:00010000:1.0:1787826390.304147:0:163071:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b4000/0xbef42c0221d6f3b6 lrc: 4/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee9ee expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.304177:0:163071:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b4000/0xbef42c0221d6f3b6 lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x19:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee9ee expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.304186:0:163071:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b4000/0xbef42c0221d6f3b6 lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee9ee expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.304198:0:163071:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b4000/0xbef42c0221d6f3b6 lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee9ee expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.304209:0:163071:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b4000/0xbef42c0221d6f3b6 lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee9ee expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.304223:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b4000/0xbef42c0221d6f3b6 lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee9ee expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.304231:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b4000/0xbef42c0221d6f3b6 lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee9ee expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.304243:0:163071:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b4000/0xbef42c0221d6f3b6 lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee9ee expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1787826390.304259:0:163071:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9181c35b0000 x1874663043042560/t21474836487(21474836487) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:1.0:1787826390.304273:0:163071:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c35b0000 x1874663043042560/t21474836487(21474836487) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1787826390.304286:0:163071:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b4000/0xbef42c0221d6f3b6 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee9ee expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:1.0:1787826390.304300:0:163071:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b4000/0xbef42c0221d6f3b6 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee9ee expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.304310:0:163071:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c77f3000 type 13 mode 4 res 9663684441/25 (0 0) 00000080:00200000:1.0:1787826390.304316:0:163071:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x240001f59:0x16:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1787826390.304332:0:163071:0:(namei.c:1628:ll_create_it()) VFS Op:name=f-2, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), intent=open|creat 00000080:00200000:1.0:1787826390.304360:0:163071:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001f59:0x19:0x0](ffff9181c83e1988) 00000080:00200000:1.0:1787826390.304445:0:163071:0:(file.c:6843:ll_layout_conf()) [0x240001f59:0x19:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:1.0:1787826390.304451:0:163071:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x240001f59:0x19:0x0](ffff9181c83e1988) 00000080:00010000:1.0:1787826390.304455:0:163071:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001f59:0x19:0x0](ffff9181c83e1988) for lock 0xbef42c0221d6f3b6 00000080:00010000:1.0:1787826390.304474:0:163071:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001f59:0x19:0x0](ffff9181c83e1988) 00000080:00200008:1.0:1787826390.304489:0:163071:0:(file.c:1090:ll_file_open()) START file f-2:[0x240001f59:0x19:0x0](ffff9181c83e1988), flags 100102 00000002:00100000:1.0:1787826390.304510:0:163071:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c35b0000 x1874663043042560/t21474836487(21474836487) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:1.0:1787826390.304806:0:163071:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f3b6 from it ffff9181d1d8a180 00010000:00010000:1.0:1787826390.304811:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b4000/0xbef42c0221d6f3b6 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00cee9ee expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.304822:0:163071:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b4000/0xbef42c0221d6f3b6 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00cee9ee expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826390.304844:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x19:0x0](ffff9181c83e1988), inode mode 81a4 mask 40 00000080:00200008:1.0:1787826390.304875:0:163071:0:(file.c:479:ll_file_release()) START file f-2:[0x240001f59:0x19:0x0](ffff9181c83e1988), flags 100002 00000002:00100000:1.0:1787826390.305141:0:163071:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181c35b0000 x1874663043042560/t21474836487(21474836487) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:1.0:1787826390.305152:0:163071:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:1.0:1787826390.305157:0:163071:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c35b1180 x1874663043042816/t0(0) o35->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:1.0:1787826390.305177:0:163071:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043042816:192.168.203.147@tcp:35:createmany.0 00000100:00100000:1.0:1787826390.305260:0:163071:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181e0156e40 going to sleep for 11 seconds 00000100:00100000:1.0:1787826390.314318:0:163071:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043042816:192.168.203.147@tcp:35:createmany.0 00000002:00100000:1.0:1787826390.314357:0:163071:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181c35b0000 x1874663043042560/t21474836487(21474836487) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787826390.314433:0:163071:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:1.0:1787826390.314451:0:163071:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826390.314482:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 11 lru_type: 1 00010000:00010000:1.0:1787826390.314497:0:163071:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 11 lru_type: 1 00000080:00200000:1.0:1787826390.314511:0:163071:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826390.314516:0:163071:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:1.0:1787826390.314520:0:163071:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f393 00000080:00010000:1.0:1787826390.314526:0:163071:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f393 from it ffffb97500e2bbf8 00010000:00010000:1.0:1787826390.314530:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 11 lru_type: 1 00010000:00010000:1.0:1787826390.314540:0:163071:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 12 lru_type: 1 00000080:00200000:1.0:1787826390.314555:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826390.314561:0:163071:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80b.replay-single, flags=25 00000080:00200000:1.0:1787826390.314567:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x4:0x0](ffff9181d0809148), inode mode 41ed mask 1 00000080:00200000:1.0:1787826390.314571:0:163071:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=remote_dir, flags=25 00000080:00200000:1.0:1787826390.314573:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 1 00000080:00200000:1.0:1787826390.314585:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 3 00000080:00200000:1.0:1787826390.314590:0:163071:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f-3, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), file ffff9181c381c180, open_flags 8042, mode 81a4 opened 0 00000080:00200000:1.0:1787826390.314599:0:163071:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f-3, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), intent=open|creat 00000002:00010000:1.0:1787826390.314618:0:163071:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f-3,[0x240001f59:0x1a:0x0]) in obj [0x240001f59:0x16:0x0], intent: open|creat flags 0100103 00010000:00010000:1.0:1787826390.314669:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b5400/0xbef42c0221d6f3bd lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.314689:0:163071:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b5400/0xbef42c0221d6f3bd lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:1.0:1787826390.314702:0:163071:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:1.0:1787826390.314708:0:163071:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c35b2680 x1874663043043072/t0(0) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:1.0:1787826390.314717:0:163071:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b5400/0xbef42c0221d6f3bd lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826390.314737:0:163071:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043043072:192.168.203.147@tcp:101:createmany.0 00000100:00100000:2.0:1787826390.316227:0:163071:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181e0156e40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826390.336501:0:163071:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043043072:192.168.203.147@tcp:101:createmany.0 00010000:00010000:2.0:1787826390.336543:0:163071:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b5400/0xbef42c0221d6f3bd lrc: 4/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceea11 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.336581:0:163071:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b5400/0xbef42c0221d6f3bd lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceea11 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.336591:0:163071:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b5400/0xbef42c0221d6f3bd lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x1a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceea11 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.336602:0:163071:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b5400/0xbef42c0221d6f3bd lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x1a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceea11 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.336613:0:163071:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b5400/0xbef42c0221d6f3bd lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x1a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceea11 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.336627:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b5400/0xbef42c0221d6f3bd lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x1a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceea11 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.336636:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b5400/0xbef42c0221d6f3bd lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x1a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceea11 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.336644:0:163071:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b5400/0xbef42c0221d6f3bd lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x1a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceea11 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787826390.336655:0:163071:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9181c35b2680 x1874663043043072/t21474836489(21474836489) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:2.0:1787826390.336668:0:163071:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c35b2680 x1874663043043072/t21474836489(21474836489) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787826390.336680:0:163071:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b5400/0xbef42c0221d6f3bd lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x1a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceea11 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:2.0:1787826390.336692:0:163071:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b5400/0xbef42c0221d6f3bd lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x1a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceea11 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.336703:0:163071:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c77f3000 type 13 mode 4 res 9663684441/26 (0 0) 00000080:00200000:2.0:1787826390.336709:0:163071:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x240001f59:0x16:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1787826390.336722:0:163071:0:(namei.c:1628:ll_create_it()) VFS Op:name=f-3, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), intent=open|creat 00000080:00200000:2.0:1787826390.336817:0:163071:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001f59:0x1a:0x0](ffff9181c830aa08) 00000080:00200000:2.0:1787826390.336942:0:163071:0:(file.c:6843:ll_layout_conf()) [0x240001f59:0x1a:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:2.0:1787826390.336948:0:163071:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x240001f59:0x1a:0x0](ffff9181c830aa08) 00000080:00010000:2.0:1787826390.336951:0:163071:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001f59:0x1a:0x0](ffff9181c830aa08) for lock 0xbef42c0221d6f3bd 00000080:00010000:2.0:1787826390.336960:0:163071:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001f59:0x1a:0x0](ffff9181c830aa08) 00000080:00200008:2.0:1787826390.336973:0:163071:0:(file.c:1090:ll_file_open()) START file f-3:[0x240001f59:0x1a:0x0](ffff9181c830aa08), flags 100102 00000002:00100000:2.0:1787826390.336995:0:163071:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c35b2680 x1874663043043072/t21474836489(21474836489) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:2.0:1787826390.337031:0:163071:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f3bd from it ffff9181d1d8a180 00010000:00010000:2.0:1787826390.337036:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b5400/0xbef42c0221d6f3bd lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x1a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00ceea11 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.337045:0:163071:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b5400/0xbef42c0221d6f3bd lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x1a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00ceea11 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826390.337067:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1a:0x0](ffff9181c830aa08), inode mode 81a4 mask 40 00000080:00200008:2.0:1787826390.337097:0:163071:0:(file.c:479:ll_file_release()) START file f-3:[0x240001f59:0x1a:0x0](ffff9181c830aa08), flags 100002 00000002:00100000:2.0:1787826390.337140:0:163071:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181c35b2680 x1874663043043072/t21474836489(21474836489) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:2.0:1787826390.337150:0:163071:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1787826390.337155:0:163071:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181d0fc1500 x1874663043043328/t0(0) o35->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:2.0:1787826390.337175:0:163071:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc1500 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043043328:192.168.203.147@tcp:35:createmany.0 00000100:00100000:2.0:1787826390.337240:0:163071:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6ea89c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826390.347913:0:163071:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc1500 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043043328:192.168.203.147@tcp:35:createmany.0 00000002:00100000:2.0:1787826390.347952:0:163071:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181c35b2680 x1874663043043072/t21474836489(21474836489) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1787826390.348030:0:163071:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:2.0:1787826390.348049:0:163071:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826390.348067:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 12 lru_type: 1 00010000:00010000:2.0:1787826390.348152:0:163071:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 12 lru_type: 1 00000080:00200000:2.0:1787826390.348162:0:163071:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826390.348168:0:163071:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:2.0:1787826390.348172:0:163071:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f393 00000080:00010000:2.0:1787826390.348179:0:163071:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f393 from it ffffb97500e2bbf8 00010000:00010000:2.0:1787826390.348183:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 12 lru_type: 1 00010000:00010000:2.0:1787826390.348190:0:163071:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 13 lru_type: 1 00000080:00200000:2.0:1787826390.348204:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.348210:0:163071:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80b.replay-single, flags=25 00000080:00200000:2.0:1787826390.348216:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x4:0x0](ffff9181d0809148), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.348219:0:163071:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=remote_dir, flags=25 00000080:00200000:2.0:1787826390.348221:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.348232:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 3 00000080:00200000:2.0:1787826390.348236:0:163071:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f-4, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), file ffff9181c4d04600, open_flags 8042, mode 81a4 opened 0 00000080:00200000:2.0:1787826390.348244:0:163071:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f-4, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), intent=open|creat 00000002:00010000:2.0:1787826390.348262:0:163071:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f-4,[0x240001f59:0x1b:0x0]) in obj [0x240001f59:0x16:0x0], intent: open|creat flags 0100103 00010000:00010000:2.0:1787826390.348312:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842c00/0xbef42c0221d6f3c4 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.348320:0:163071:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842c00/0xbef42c0221d6f3c4 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1787826390.348331:0:163071:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1787826390.348337:0:163071:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181d0fc2680 x1874663043043584/t0(0) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1787826390.348346:0:163071:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842c00/0xbef42c0221d6f3c4 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826390.348362:0:163071:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc2680 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043043584:192.168.203.147@tcp:101:createmany.0 00000100:00100000:2.0:1787826390.349033:0:163071:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6ea89c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826390.368205:0:163071:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc2680 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043043584:192.168.203.147@tcp:101:createmany.0 00010000:00010000:2.0:1787826390.368234:0:163071:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842c00/0xbef42c0221d6f3c4 lrc: 4/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceea34 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.368267:0:163071:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842c00/0xbef42c0221d6f3c4 lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceea34 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.368275:0:163071:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842c00/0xbef42c0221d6f3c4 lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceea34 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.368286:0:163071:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842c00/0xbef42c0221d6f3c4 lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceea34 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.368297:0:163071:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842c00/0xbef42c0221d6f3c4 lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceea34 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.368311:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842c00/0xbef42c0221d6f3c4 lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceea34 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.368325:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842c00/0xbef42c0221d6f3c4 lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceea34 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.368333:0:163071:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842c00/0xbef42c0221d6f3c4 lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceea34 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787826390.368345:0:163071:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9181d0fc2680 x1874663043043584/t21474836491(21474836491) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:2.0:1787826390.368358:0:163071:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181d0fc2680 x1874663043043584/t21474836491(21474836491) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787826390.368371:0:163071:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842c00/0xbef42c0221d6f3c4 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceea34 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:2.0:1787826390.368384:0:163071:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842c00/0xbef42c0221d6f3c4 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceea34 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.368520:0:163071:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c77f3000 type 13 mode 4 res 9663684441/27 (0 0) 00000080:00200000:2.0:1787826390.368527:0:163071:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x240001f59:0x16:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1787826390.368540:0:163071:0:(namei.c:1628:ll_create_it()) VFS Op:name=f-4, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), intent=open|creat 00000080:00200000:2.0:1787826390.368572:0:163071:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001f59:0x1b:0x0](ffff9181c830ec08) 00000080:00200000:2.0:1787826390.368642:0:163071:0:(file.c:6843:ll_layout_conf()) [0x240001f59:0x1b:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:2.0:1787826390.368648:0:163071:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x240001f59:0x1b:0x0](ffff9181c830ec08) 00000080:00010000:2.0:1787826390.368651:0:163071:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001f59:0x1b:0x0](ffff9181c830ec08) for lock 0xbef42c0221d6f3c4 00000080:00010000:2.0:1787826390.368660:0:163071:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001f59:0x1b:0x0](ffff9181c830ec08) 00000080:00200008:2.0:1787826390.368674:0:163071:0:(file.c:1090:ll_file_open()) START file f-4:[0x240001f59:0x1b:0x0](ffff9181c830ec08), flags 100102 00000002:00100000:2.0:1787826390.368938:0:163071:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181d0fc2680 x1874663043043584/t21474836491(21474836491) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:2.0:1787826390.368975:0:163071:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f3c4 from it ffff9181e003a3c0 00010000:00010000:2.0:1787826390.368981:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842c00/0xbef42c0221d6f3c4 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00ceea34 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.368990:0:163071:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842c00/0xbef42c0221d6f3c4 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00ceea34 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826390.369011:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1b:0x0](ffff9181c830ec08), inode mode 81a4 mask 40 00000080:00200008:2.0:1787826390.369039:0:163071:0:(file.c:479:ll_file_release()) START file f-4:[0x240001f59:0x1b:0x0](ffff9181c830ec08), flags 100002 00000002:00100000:2.0:1787826390.369083:0:163071:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181d0fc2680 x1874663043043584/t21474836491(21474836491) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:2.0:1787826390.369094:0:163071:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1787826390.369099:0:163071:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181d0fc1880 x1874663043043840/t0(0) o35->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:2.0:1787826390.369118:0:163071:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc1880 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043043840:192.168.203.147@tcp:35:createmany.0 00000100:00100000:2.0:1787826390.369197:0:163071:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6ea8c00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826390.371451:0:163071:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc1880 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043043840:192.168.203.147@tcp:35:createmany.0 00000002:00100000:2.0:1787826390.371540:0:163071:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181d0fc2680 x1874663043043584/t21474836491(21474836491) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1787826390.371613:0:163071:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:2.0:1787826390.371630:0:163071:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826390.371648:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 13 lru_type: 1 00010000:00010000:2.0:1787826390.371666:0:163071:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 13 lru_type: 1 00000080:00200000:2.0:1787826390.371674:0:163071:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826390.371679:0:163071:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:2.0:1787826390.372013:0:163071:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f393 00000080:00010000:2.0:1787826390.372023:0:163071:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f393 from it ffffb97500e2bbf8 00010000:00010000:2.0:1787826390.372028:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 13 lru_type: 1 00010000:00010000:2.0:1787826390.372037:0:163071:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 14 lru_type: 1 00000080:00200000:2.0:1787826390.372051:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.372059:0:163071:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80b.replay-single, flags=25 00000080:00200000:2.0:1787826390.372065:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x4:0x0](ffff9181d0809148), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.372069:0:163071:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=remote_dir, flags=25 00000080:00200000:2.0:1787826390.372071:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.372085:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 3 00000080:00200000:2.0:1787826390.372089:0:163071:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f-5, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), file ffff9181c4d05c80, open_flags 8042, mode 81a4 opened 0 00000080:00200000:2.0:1787826390.372096:0:163071:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f-5, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), intent=open|creat 00000002:00010000:2.0:1787826390.372117:0:163071:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f-5,[0x240001f59:0x1c:0x0]) in obj [0x240001f59:0x16:0x0], intent: open|creat flags 0100103 00010000:00010000:2.0:1787826390.372169:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843c00/0xbef42c0221d6f3cb lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.372177:0:163071:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843c00/0xbef42c0221d6f3cb lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1787826390.372188:0:163071:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1787826390.372195:0:163071:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181d0fc1180 x1874663043044096/t0(0) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1787826390.372204:0:163071:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843c00/0xbef42c0221d6f3cb lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826390.372222:0:163071:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc1180 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043044096:192.168.203.147@tcp:101:createmany.0 00000100:00100000:2.0:1787826390.372397:0:163071:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6ea8c00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826390.397036:0:163071:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc1180 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043044096:192.168.203.147@tcp:101:createmany.0 00010000:00010000:2.0:1787826390.397064:0:163071:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843c00/0xbef42c0221d6f3cb lrc: 4/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceea57 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.397095:0:163071:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843c00/0xbef42c0221d6f3cb lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x1c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceea57 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.397103:0:163071:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843c00/0xbef42c0221d6f3cb lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x1c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceea57 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.397114:0:163071:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843c00/0xbef42c0221d6f3cb lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x1c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceea57 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.397124:0:163071:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843c00/0xbef42c0221d6f3cb lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x1c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceea57 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.397136:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843c00/0xbef42c0221d6f3cb lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x1c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceea57 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.397150:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843c00/0xbef42c0221d6f3cb lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x1c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceea57 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.397157:0:163071:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843c00/0xbef42c0221d6f3cb lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x1c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceea57 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787826390.397168:0:163071:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9181d0fc1180 x1874663043044096/t21474836493(21474836493) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:2.0:1787826390.397181:0:163071:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181d0fc1180 x1874663043044096/t21474836493(21474836493) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787826390.397193:0:163071:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843c00/0xbef42c0221d6f3cb lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x1c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceea57 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:2.0:1787826390.397204:0:163071:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843c00/0xbef42c0221d6f3cb lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x1c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceea57 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.397214:0:163071:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c77f3000 type 13 mode 4 res 9663684441/28 (0 0) 00000080:00200000:2.0:1787826390.397220:0:163071:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x240001f59:0x16:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1787826390.397232:0:163071:0:(namei.c:1628:ll_create_it()) VFS Op:name=f-5, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), intent=open|creat 00000080:00200000:2.0:1787826390.397257:0:163071:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001f59:0x1c:0x0](ffff9181c830a1c8) 00000080:00200000:2.0:1787826390.397328:0:163071:0:(file.c:6843:ll_layout_conf()) [0x240001f59:0x1c:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:2.0:1787826390.397333:0:163071:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x240001f59:0x1c:0x0](ffff9181c830a1c8) 00000080:00010000:2.0:1787826390.397336:0:163071:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001f59:0x1c:0x0](ffff9181c830a1c8) for lock 0xbef42c0221d6f3cb 00000080:00010000:2.0:1787826390.397346:0:163071:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001f59:0x1c:0x0](ffff9181c830a1c8) 00000080:00200008:2.0:1787826390.397360:0:163071:0:(file.c:1090:ll_file_open()) START file f-5:[0x240001f59:0x1c:0x0](ffff9181c830a1c8), flags 100102 00000002:00100000:2.0:1787826390.397382:0:163071:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181d0fc1180 x1874663043044096/t21474836493(21474836493) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:2.0:1787826390.397418:0:163071:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f3cb from it ffff9181e003a3c0 00010000:00010000:2.0:1787826390.397423:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843c00/0xbef42c0221d6f3cb lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x1c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00ceea57 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.397432:0:163071:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843c00/0xbef42c0221d6f3cb lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x1c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00ceea57 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826390.397453:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1c:0x0](ffff9181c830a1c8), inode mode 81a4 mask 40 00000080:00200008:2.0:1787826390.397494:0:163071:0:(file.c:479:ll_file_release()) START file f-5:[0x240001f59:0x1c:0x0](ffff9181c830a1c8), flags 100002 00000002:00100000:2.0:1787826390.397537:0:163071:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181d0fc1180 x1874663043044096/t21474836493(21474836493) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:2.0:1787826390.397547:0:163071:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1787826390.397551:0:163071:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181d0fc2a00 x1874663043044352/t0(0) o35->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:2.0:1787826390.397571:0:163071:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc2a00 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043044352:192.168.203.147@tcp:35:createmany.0 00000100:00100000:2.0:1787826390.397651:0:163071:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6ea8d80 going to sleep for 11 seconds 00000100:00100000:2.0:1787826390.403156:0:163071:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc2a00 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043044352:192.168.203.147@tcp:35:createmany.0 00000002:00100000:2.0:1787826390.403194:0:163071:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181d0fc1180 x1874663043044096/t21474836493(21474836493) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1787826390.403272:0:163071:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:2.0:1787826390.403291:0:163071:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826390.403310:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 14 lru_type: 1 00010000:00010000:2.0:1787826390.403331:0:163071:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 14 lru_type: 1 00000080:00200000:2.0:1787826390.403339:0:163071:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826390.403345:0:163071:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:2.0:1787826390.403349:0:163071:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f393 00000080:00010000:2.0:1787826390.403355:0:163071:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f393 from it ffffb97500e2bbf8 00010000:00010000:2.0:1787826390.403359:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 14 lru_type: 1 00010000:00010000:2.0:1787826390.403368:0:163071:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 15 lru_type: 1 00000080:00200000:2.0:1787826390.403383:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.403390:0:163071:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80b.replay-single, flags=25 00000080:00200000:2.0:1787826390.403397:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x4:0x0](ffff9181d0809148), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.403401:0:163071:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=remote_dir, flags=25 00000080:00200000:2.0:1787826390.403404:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.403420:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 3 00000080:00200000:2.0:1787826390.403424:0:163071:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f-6, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), file ffff9181c4d04600, open_flags 8042, mode 81a4 opened 0 00000080:00200000:2.0:1787826390.403432:0:163071:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f-6, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), intent=open|creat 00000002:00010000:2.0:1787826390.403451:0:163071:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f-6,[0x240001f59:0x1d:0x0]) in obj [0x240001f59:0x16:0x0], intent: open|creat flags 0100103 00010000:00010000:2.0:1787826390.403512:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842600/0xbef42c0221d6f3d2 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.403520:0:163071:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842600/0xbef42c0221d6f3d2 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1787826390.403531:0:163071:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1787826390.403537:0:163071:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181d0fc0000 x1874663043044608/t0(0) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1787826390.403547:0:163071:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842600/0xbef42c0221d6f3d2 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826390.403566:0:163071:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc0000 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043044608:192.168.203.147@tcp:101:createmany.0 00000100:00100000:2.0:1787826390.403963:0:163071:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6ea8d80 going to sleep for 11 seconds 00000100:00100000:2.0:1787826390.420357:0:163071:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc0000 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043044608:192.168.203.147@tcp:101:createmany.0 00010000:00010000:2.0:1787826390.420393:0:163071:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842600/0xbef42c0221d6f3d2 lrc: 4/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceea7a expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.420437:0:163071:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842600/0xbef42c0221d6f3d2 lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x1d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceea7a expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.420447:0:163071:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842600/0xbef42c0221d6f3d2 lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceea7a expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.420459:0:163071:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842600/0xbef42c0221d6f3d2 lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceea7a expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.420483:0:163071:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842600/0xbef42c0221d6f3d2 lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceea7a expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.420498:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842600/0xbef42c0221d6f3d2 lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceea7a expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.420512:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842600/0xbef42c0221d6f3d2 lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceea7a expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.420519:0:163071:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842600/0xbef42c0221d6f3d2 lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceea7a expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787826390.420531:0:163071:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9181d0fc0000 x1874663043044608/t21474836495(21474836495) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:2.0:1787826390.420544:0:163071:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181d0fc0000 x1874663043044608/t21474836495(21474836495) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787826390.420557:0:163071:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842600/0xbef42c0221d6f3d2 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceea7a expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:2.0:1787826390.420570:0:163071:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842600/0xbef42c0221d6f3d2 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceea7a expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.420581:0:163071:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c77f3000 type 13 mode 4 res 9663684441/29 (0 0) 00000080:00200000:2.0:1787826390.420588:0:163071:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x240001f59:0x16:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1787826390.420602:0:163071:0:(namei.c:1628:ll_create_it()) VFS Op:name=f-6, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), intent=open|creat 00000080:00200000:2.0:1787826390.420632:0:163071:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001f59:0x1d:0x0](ffff9181c830cb08) 00000080:00200000:2.0:1787826390.422491:0:163071:0:(file.c:6843:ll_layout_conf()) [0x240001f59:0x1d:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:2.0:1787826390.422507:0:163071:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x240001f59:0x1d:0x0](ffff9181c830cb08) 00000080:00010000:2.0:1787826390.422512:0:163071:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001f59:0x1d:0x0](ffff9181c830cb08) for lock 0xbef42c0221d6f3d2 00000080:00010000:2.0:1787826390.422523:0:163071:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001f59:0x1d:0x0](ffff9181c830cb08) 00000080:00200008:2.0:1787826390.422537:0:163071:0:(file.c:1090:ll_file_open()) START file f-6:[0x240001f59:0x1d:0x0](ffff9181c830cb08), flags 100102 00000002:00100000:2.0:1787826390.422571:0:163071:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181d0fc0000 x1874663043044608/t21474836495(21474836495) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:2.0:1787826390.422611:0:163071:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f3d2 from it ffff9181e003a3c0 00010000:00010000:2.0:1787826390.422619:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842600/0xbef42c0221d6f3d2 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00ceea7a expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.422631:0:163071:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842600/0xbef42c0221d6f3d2 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00ceea7a expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826390.422655:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1d:0x0](ffff9181c830cb08), inode mode 81a4 mask 40 00000080:00200008:2.0:1787826390.422687:0:163071:0:(file.c:479:ll_file_release()) START file f-6:[0x240001f59:0x1d:0x0](ffff9181c830cb08), flags 100002 00000002:00100000:2.0:1787826390.424441:0:163071:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181d0fc0000 x1874663043044608/t21474836495(21474836495) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:2.0:1787826390.424459:0:163071:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1787826390.424474:0:163071:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181d0fc3100 x1874663043044864/t0(0) o35->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:2.0:1787826390.424505:0:163071:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc3100 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043044864:192.168.203.147@tcp:35:createmany.0 00000100:00100000:2.0:1787826390.424659:0:163071:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6ea8780 going to sleep for 11 seconds 00000100:00100000:2.0:1787826390.441325:0:163071:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc3100 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043044864:192.168.203.147@tcp:35:createmany.0 00000002:00100000:2.0:1787826390.441364:0:163071:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181d0fc0000 x1874663043044608/t21474836495(21474836495) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1787826390.441438:0:163071:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:2.0:1787826390.441457:0:163071:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826390.441488:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 15 lru_type: 1 00010000:00010000:2.0:1787826390.441508:0:163071:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 15 lru_type: 1 00000080:00200000:2.0:1787826390.441515:0:163071:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826390.441521:0:163071:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:2.0:1787826390.441525:0:163071:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f393 00000080:00010000:2.0:1787826390.441532:0:163071:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f393 from it ffffb97500e2bbf8 00010000:00010000:2.0:1787826390.441536:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 15 lru_type: 1 00010000:00010000:2.0:1787826390.441546:0:163071:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 16 lru_type: 1 00000080:00200000:2.0:1787826390.441561:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.441568:0:163071:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80b.replay-single, flags=25 00000080:00200000:2.0:1787826390.441574:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x4:0x0](ffff9181d0809148), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.441578:0:163071:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=remote_dir, flags=25 00000080:00200000:2.0:1787826390.441581:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.441594:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 3 00000080:00200000:2.0:1787826390.441598:0:163071:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f-7, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), file ffff9181c4d05c80, open_flags 8042, mode 81a4 opened 0 00000080:00200000:2.0:1787826390.441605:0:163071:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f-7, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), intent=open|creat 00000002:00010000:2.0:1787826390.441638:0:163071:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f-7,[0x240001f59:0x1e:0x0]) in obj [0x240001f59:0x16:0x0], intent: open|creat flags 0100103 00010000:00010000:2.0:1787826390.441689:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842400/0xbef42c0221d6f3d9 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.441698:0:163071:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842400/0xbef42c0221d6f3d9 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1787826390.441710:0:163071:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1787826390.441717:0:163071:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181d0fc2d80 x1874663043045120/t0(0) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1787826390.441725:0:163071:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842400/0xbef42c0221d6f3d9 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826390.441796:0:163071:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc2d80 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043045120:192.168.203.147@tcp:101:createmany.0 00000100:00100000:2.0:1787826390.441933:0:163071:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6ea8780 going to sleep for 11 seconds 00000100:00100000:2.0:1787826390.451408:0:163071:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc2d80 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043045120:192.168.203.147@tcp:101:createmany.0 00010000:00010000:2.0:1787826390.451437:0:163071:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842400/0xbef42c0221d6f3d9 lrc: 4/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceea9d expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.451478:0:163071:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842400/0xbef42c0221d6f3d9 lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceea9d expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.451487:0:163071:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842400/0xbef42c0221d6f3d9 lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x1e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceea9d expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.451497:0:163071:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842400/0xbef42c0221d6f3d9 lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x1e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceea9d expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.451509:0:163071:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842400/0xbef42c0221d6f3d9 lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x1e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceea9d expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.451522:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842400/0xbef42c0221d6f3d9 lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x1e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceea9d expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.451534:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842400/0xbef42c0221d6f3d9 lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x1e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceea9d expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.451540:0:163071:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842400/0xbef42c0221d6f3d9 lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x1e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceea9d expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787826390.451550:0:163071:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9181d0fc2d80 x1874663043045120/t21474836497(21474836497) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:2.0:1787826390.451561:0:163071:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181d0fc2d80 x1874663043045120/t21474836497(21474836497) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787826390.451574:0:163071:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842400/0xbef42c0221d6f3d9 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x1e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceea9d expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:2.0:1787826390.451587:0:163071:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842400/0xbef42c0221d6f3d9 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x1e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceea9d expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.451597:0:163071:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c77f3000 type 13 mode 4 res 9663684441/30 (0 0) 00000080:00200000:2.0:1787826390.451603:0:163071:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x240001f59:0x16:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1787826390.451615:0:163071:0:(namei.c:1628:ll_create_it()) VFS Op:name=f-7, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), intent=open|creat 00000080:00200000:2.0:1787826390.451643:0:163071:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001f59:0x1e:0x0](ffff9181c830b248) 00000080:00200000:2.0:1787826390.451717:0:163071:0:(file.c:6843:ll_layout_conf()) [0x240001f59:0x1e:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:2.0:1787826390.451722:0:163071:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x240001f59:0x1e:0x0](ffff9181c830b248) 00000080:00010000:2.0:1787826390.451726:0:163071:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001f59:0x1e:0x0](ffff9181c830b248) for lock 0xbef42c0221d6f3d9 00000080:00010000:2.0:1787826390.451735:0:163071:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001f59:0x1e:0x0](ffff9181c830b248) 00000080:00200008:2.0:1787826390.451804:0:163071:0:(file.c:1090:ll_file_open()) START file f-7:[0x240001f59:0x1e:0x0](ffff9181c830b248), flags 100102 00000002:00100000:2.0:1787826390.451826:0:163071:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181d0fc2d80 x1874663043045120/t21474836497(21474836497) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:2.0:1787826390.451862:0:163071:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f3d9 from it ffff9181e003a3c0 00010000:00010000:2.0:1787826390.451867:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842400/0xbef42c0221d6f3d9 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x1e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00ceea9d expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.451875:0:163071:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842400/0xbef42c0221d6f3d9 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x1e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00ceea9d expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826390.451895:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1e:0x0](ffff9181c830b248), inode mode 81a4 mask 40 00000080:00200008:2.0:1787826390.451923:0:163071:0:(file.c:479:ll_file_release()) START file f-7:[0x240001f59:0x1e:0x0](ffff9181c830b248), flags 100002 00000002:00100000:2.0:1787826390.451968:0:163071:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181d0fc2d80 x1874663043045120/t21474836497(21474836497) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:2.0:1787826390.451978:0:163071:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1787826390.451983:0:163071:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181d0fc3480 x1874663043045376/t0(0) o35->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:2.0:1787826390.452001:0:163071:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc3480 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043045376:192.168.203.147@tcp:35:createmany.0 00000100:00100000:2.0:1787826390.452075:0:163071:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6ea8480 going to sleep for 11 seconds 00000100:00100000:2.0:1787826390.456315:0:163071:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc3480 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043045376:192.168.203.147@tcp:35:createmany.0 00000002:00100000:2.0:1787826390.456352:0:163071:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181d0fc2d80 x1874663043045120/t21474836497(21474836497) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1787826390.456438:0:163071:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:2.0:1787826390.456456:0:163071:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826390.456483:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 16 lru_type: 1 00010000:00010000:2.0:1787826390.456503:0:163071:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 16 lru_type: 1 00000080:00200000:2.0:1787826390.456511:0:163071:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826390.456517:0:163071:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:2.0:1787826390.456530:0:163071:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f393 00000080:00010000:2.0:1787826390.456537:0:163071:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f393 from it ffffb97500e2bbf8 00010000:00010000:2.0:1787826390.456541:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 16 lru_type: 1 00010000:00010000:2.0:1787826390.456551:0:163071:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 17 lru_type: 1 00000080:00200000:2.0:1787826390.456566:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.456573:0:163071:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80b.replay-single, flags=25 00000080:00200000:2.0:1787826390.456579:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x4:0x0](ffff9181d0809148), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.456582:0:163071:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=remote_dir, flags=25 00000080:00200000:2.0:1787826390.456585:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.456597:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 3 00000080:00200000:2.0:1787826390.456601:0:163071:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f-8, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), file ffff9181c4d04600, open_flags 8042, mode 81a4 opened 0 00000080:00200000:2.0:1787826390.456609:0:163071:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f-8, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), intent=open|creat 00000002:00010000:2.0:1787826390.456627:0:163071:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f-8,[0x240001f59:0x1f:0x0]) in obj [0x240001f59:0x16:0x0], intent: open|creat flags 0100103 00010000:00010000:2.0:1787826390.456675:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843a00/0xbef42c0221d6f3e0 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.456683:0:163071:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843a00/0xbef42c0221d6f3e0 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1787826390.456694:0:163071:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1787826390.456701:0:163071:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181d0fc1f80 x1874663043045632/t0(0) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1787826390.456709:0:163071:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843a00/0xbef42c0221d6f3e0 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826390.456770:0:163071:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc1f80 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043045632:192.168.203.147@tcp:101:createmany.0 00000100:00100000:2.0:1787826390.456890:0:163071:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6ea8480 going to sleep for 11 seconds 00000100:00100000:2.0:1787826390.467820:0:163071:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc1f80 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043045632:192.168.203.147@tcp:101:createmany.0 00010000:00010000:2.0:1787826390.467850:0:163071:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843a00/0xbef42c0221d6f3e0 lrc: 4/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceeac0 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.467881:0:163071:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843a00/0xbef42c0221d6f3e0 lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x1f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceeac0 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.467890:0:163071:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843a00/0xbef42c0221d6f3e0 lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x1f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceeac0 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.467902:0:163071:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843a00/0xbef42c0221d6f3e0 lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x1f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceeac0 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.467913:0:163071:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843a00/0xbef42c0221d6f3e0 lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x1f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceeac0 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.467927:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843a00/0xbef42c0221d6f3e0 lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x1f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceeac0 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.467940:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843a00/0xbef42c0221d6f3e0 lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x1f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceeac0 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.467948:0:163071:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843a00/0xbef42c0221d6f3e0 lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x1f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceeac0 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787826390.467961:0:163071:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9181d0fc1f80 x1874663043045632/t21474836499(21474836499) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:2.0:1787826390.467974:0:163071:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181d0fc1f80 x1874663043045632/t21474836499(21474836499) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787826390.467988:0:163071:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843a00/0xbef42c0221d6f3e0 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x1f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceeac0 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:2.0:1787826390.468001:0:163071:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843a00/0xbef42c0221d6f3e0 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x1f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceeac0 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.468011:0:163071:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c77f3000 type 13 mode 4 res 9663684441/31 (0 0) 00000080:00200000:2.0:1787826390.468017:0:163071:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x240001f59:0x16:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1787826390.468030:0:163071:0:(namei.c:1628:ll_create_it()) VFS Op:name=f-8, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), intent=open|creat 00000080:00200000:2.0:1787826390.468057:0:163071:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001f59:0x1f:0x0](ffff9181c8309988) 00000080:00200000:2.0:1787826390.468125:0:163071:0:(file.c:6843:ll_layout_conf()) [0x240001f59:0x1f:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:2.0:1787826390.468130:0:163071:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x240001f59:0x1f:0x0](ffff9181c8309988) 00000080:00010000:2.0:1787826390.468133:0:163071:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001f59:0x1f:0x0](ffff9181c8309988) for lock 0xbef42c0221d6f3e0 00000080:00010000:2.0:1787826390.468141:0:163071:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001f59:0x1f:0x0](ffff9181c8309988) 00000080:00200008:2.0:1787826390.468153:0:163071:0:(file.c:1090:ll_file_open()) START file f-8:[0x240001f59:0x1f:0x0](ffff9181c8309988), flags 100102 00000002:00100000:2.0:1787826390.468173:0:163071:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181d0fc1f80 x1874663043045632/t21474836499(21474836499) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:2.0:1787826390.468215:0:163071:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f3e0 from it ffff9181e003a3c0 00010000:00010000:2.0:1787826390.468220:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843a00/0xbef42c0221d6f3e0 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x1f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00ceeac0 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.468229:0:163071:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843a00/0xbef42c0221d6f3e0 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x1f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00ceeac0 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826390.468248:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x1f:0x0](ffff9181c8309988), inode mode 81a4 mask 40 00000080:00200008:2.0:1787826390.468288:0:163071:0:(file.c:479:ll_file_release()) START file f-8:[0x240001f59:0x1f:0x0](ffff9181c8309988), flags 100002 00000002:00100000:2.0:1787826390.468331:0:163071:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181d0fc1f80 x1874663043045632/t21474836499(21474836499) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:2.0:1787826390.468342:0:163071:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1787826390.468347:0:163071:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181d0fc3800 x1874663043045888/t0(0) o35->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:2.0:1787826390.468368:0:163071:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc3800 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043045888:192.168.203.147@tcp:35:createmany.0 00000100:00100000:2.0:1787826390.468459:0:163071:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6ea8b40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826390.487559:0:163071:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc3800 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043045888:192.168.203.147@tcp:35:createmany.0 00000002:00100000:2.0:1787826390.487599:0:163071:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181d0fc1f80 x1874663043045632/t21474836499(21474836499) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1787826390.487682:0:163071:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:2.0:1787826390.487701:0:163071:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826390.487720:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 17 lru_type: 1 00010000:00010000:2.0:1787826390.487776:0:163071:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 17 lru_type: 1 00000080:00200000:2.0:1787826390.487785:0:163071:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826390.487792:0:163071:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:2.0:1787826390.487796:0:163071:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f393 00000080:00010000:2.0:1787826390.487803:0:163071:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f393 from it ffffb97500e2bbf8 00010000:00010000:2.0:1787826390.487807:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 17 lru_type: 1 00010000:00010000:2.0:1787826390.487817:0:163071:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 18 lru_type: 1 00000080:00200000:2.0:1787826390.487833:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.487841:0:163071:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80b.replay-single, flags=25 00000080:00200000:2.0:1787826390.487846:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x4:0x0](ffff9181d0809148), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.487851:0:163071:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=remote_dir, flags=25 00000080:00200000:2.0:1787826390.487854:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.487870:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 3 00000080:00200000:2.0:1787826390.487875:0:163071:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f-9, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), file ffff9181c4d05c80, open_flags 8042, mode 81a4 opened 0 00000080:00200000:2.0:1787826390.487883:0:163071:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f-9, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), intent=open|creat 00000002:00010000:2.0:1787826390.487902:0:163071:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f-9,[0x240001f59:0x20:0x0]) in obj [0x240001f59:0x16:0x0], intent: open|creat flags 0100103 00010000:00010000:2.0:1787826390.487954:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843e00/0xbef42c0221d6f3e7 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.487963:0:163071:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843e00/0xbef42c0221d6f3e7 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1787826390.487975:0:163071:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1787826390.487982:0:163071:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181d0fc1c00 x1874663043046144/t0(0) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1787826390.487991:0:163071:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843e00/0xbef42c0221d6f3e7 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826390.488010:0:163071:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc1c00 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043046144:192.168.203.147@tcp:101:createmany.0 00000100:00100000:2.0:1787826390.488131:0:163071:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6ea8b40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826390.493209:0:163071:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc1c00 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043046144:192.168.203.147@tcp:101:createmany.0 00010000:00010000:2.0:1787826390.493237:0:163071:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843e00/0xbef42c0221d6f3e7 lrc: 4/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceeae3 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.493335:0:163071:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843e00/0xbef42c0221d6f3e7 lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x20:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceeae3 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.493345:0:163071:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843e00/0xbef42c0221d6f3e7 lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceeae3 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.493355:0:163071:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843e00/0xbef42c0221d6f3e7 lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceeae3 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.493365:0:163071:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843e00/0xbef42c0221d6f3e7 lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceeae3 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.493380:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843e00/0xbef42c0221d6f3e7 lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceeae3 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.493403:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843e00/0xbef42c0221d6f3e7 lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceeae3 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.493411:0:163071:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843e00/0xbef42c0221d6f3e7 lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceeae3 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787826390.493422:0:163071:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9181d0fc1c00 x1874663043046144/t21474836501(21474836501) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:2.0:1787826390.493434:0:163071:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181d0fc1c00 x1874663043046144/t21474836501(21474836501) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787826390.493447:0:163071:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843e00/0xbef42c0221d6f3e7 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceeae3 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:2.0:1787826390.493460:0:163071:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843e00/0xbef42c0221d6f3e7 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceeae3 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.493477:0:163071:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c77f3000 type 13 mode 4 res 9663684441/32 (0 0) 00000080:00200000:2.0:1787826390.493483:0:163071:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x240001f59:0x16:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1787826390.493497:0:163071:0:(namei.c:1628:ll_create_it()) VFS Op:name=f-9, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), intent=open|creat 00000080:00200000:2.0:1787826390.493521:0:163071:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001f59:0x20:0x0](ffff9181c830ba88) 00000080:00200000:2.0:1787826390.493589:0:163071:0:(file.c:6843:ll_layout_conf()) [0x240001f59:0x20:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:2.0:1787826390.493593:0:163071:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x240001f59:0x20:0x0](ffff9181c830ba88) 00000080:00010000:2.0:1787826390.493597:0:163071:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001f59:0x20:0x0](ffff9181c830ba88) for lock 0xbef42c0221d6f3e7 00000080:00010000:2.0:1787826390.493606:0:163071:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001f59:0x20:0x0](ffff9181c830ba88) 00000080:00200008:2.0:1787826390.493620:0:163071:0:(file.c:1090:ll_file_open()) START file f-9:[0x240001f59:0x20:0x0](ffff9181c830ba88), flags 100102 00000002:00100000:2.0:1787826390.493640:0:163071:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181d0fc1c00 x1874663043046144/t21474836501(21474836501) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:2.0:1787826390.498683:0:163071:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f3e7 from it ffff9181e003a3c0 00010000:00010000:2.0:1787826390.498705:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843e00/0xbef42c0221d6f3e7 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00ceeae3 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.498719:0:163071:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843e00/0xbef42c0221d6f3e7 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00ceeae3 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826390.498764:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x20:0x0](ffff9181c830ba88), inode mode 81a4 mask 40 00000080:00200008:2.0:1787826390.498801:0:163071:0:(file.c:479:ll_file_release()) START file f-9:[0x240001f59:0x20:0x0](ffff9181c830ba88), flags 100002 00000002:00100000:2.0:1787826390.498872:0:163071:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181d0fc1c00 x1874663043046144/t21474836501(21474836501) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:2.0:1787826390.498884:0:163071:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1787826390.498890:0:163071:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181d0fc0380 x1874663043046400/t0(0) o35->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:2.0:1787826390.498914:0:163071:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc0380 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043046400:192.168.203.147@tcp:35:createmany.0 00000100:00100000:2.0:1787826390.499008:0:163071:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6ea8e40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826390.503143:0:163071:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc0380 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043046400:192.168.203.147@tcp:35:createmany.0 00000002:00100000:2.0:1787826390.503177:0:163071:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181d0fc1c00 x1874663043046144/t21474836501(21474836501) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1787826390.503263:0:163071:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:2.0:1787826390.503281:0:163071:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826390.503299:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 18 lru_type: 1 00010000:00010000:2.0:1787826390.503319:0:163071:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 18 lru_type: 1 00000080:00200000:2.0:1787826390.503327:0:163071:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826390.503333:0:163071:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:2.0:1787826390.503337:0:163071:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f393 00000080:00010000:2.0:1787826390.503344:0:163071:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f393 from it ffffb97500e2bbf8 00010000:00010000:2.0:1787826390.503348:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 18 lru_type: 1 00010000:00010000:2.0:1787826390.503358:0:163071:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 19 lru_type: 1 00000080:00200000:2.0:1787826390.503373:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.503379:0:163071:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80b.replay-single, flags=25 00000080:00200000:2.0:1787826390.503385:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x4:0x0](ffff9181d0809148), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.503388:0:163071:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=remote_dir, flags=25 00000080:00200000:2.0:1787826390.503391:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.503404:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 3 00000080:00200000:2.0:1787826390.503408:0:163071:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f-10, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), file ffff9181c4d04600, open_flags 8042, mode 81a4 opened 0 00000080:00200000:2.0:1787826390.503416:0:163071:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f-10, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), intent=open|creat 00000002:00010000:2.0:1787826390.503435:0:163071:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f-10,[0x240001f59:0x21:0x0]) in obj [0x240001f59:0x16:0x0], intent: open|creat flags 0100103 00010000:00010000:2.0:1787826390.503496:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843400/0xbef42c0221d6f3ee lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.503504:0:163071:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843400/0xbef42c0221d6f3ee lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1787826390.503524:0:163071:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1787826390.503530:0:163071:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181d0fc2300 x1874663043046656/t0(0) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1787826390.503539:0:163071:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843400/0xbef42c0221d6f3ee lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826390.503556:0:163071:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc2300 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043046656:192.168.203.147@tcp:101:createmany.0 00000100:00100000:2.0:1787826390.503701:0:163071:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6ea8e40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826390.513824:0:163071:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc2300 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043046656:192.168.203.147@tcp:101:createmany.0 00010000:00010000:2.0:1787826390.513855:0:163071:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843400/0xbef42c0221d6f3ee lrc: 4/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceeb06 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.513885:0:163071:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843400/0xbef42c0221d6f3ee lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x21:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceeb06 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.513894:0:163071:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843400/0xbef42c0221d6f3ee lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x21:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceeb06 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.513905:0:163071:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843400/0xbef42c0221d6f3ee lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x21:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceeb06 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.513916:0:163071:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843400/0xbef42c0221d6f3ee lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x21:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceeb06 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.513930:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843400/0xbef42c0221d6f3ee lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x21:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceeb06 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.513944:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843400/0xbef42c0221d6f3ee lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x21:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceeb06 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.513952:0:163071:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843400/0xbef42c0221d6f3ee lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x21:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceeb06 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787826390.513963:0:163071:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9181d0fc2300 x1874663043046656/t21474836503(21474836503) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:2.0:1787826390.513975:0:163071:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181d0fc2300 x1874663043046656/t21474836503(21474836503) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787826390.513989:0:163071:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843400/0xbef42c0221d6f3ee lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x21:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceeb06 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:2.0:1787826390.514002:0:163071:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843400/0xbef42c0221d6f3ee lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x21:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceeb06 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.514014:0:163071:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c77f3000 type 13 mode 4 res 9663684441/33 (0 0) 00000080:00200000:2.0:1787826390.514021:0:163071:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x240001f59:0x16:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1787826390.514036:0:163071:0:(namei.c:1628:ll_create_it()) VFS Op:name=f-10, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), intent=open|creat 00000080:00200000:2.0:1787826390.514073:0:163071:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001f59:0x21:0x0](ffff9181c830db88) 00000080:00200000:2.0:1787826390.514156:0:163071:0:(file.c:6843:ll_layout_conf()) [0x240001f59:0x21:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:2.0:1787826390.514161:0:163071:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x240001f59:0x21:0x0](ffff9181c830db88) 00000080:00010000:2.0:1787826390.514164:0:163071:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001f59:0x21:0x0](ffff9181c830db88) for lock 0xbef42c0221d6f3ee 00000080:00010000:2.0:1787826390.514174:0:163071:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001f59:0x21:0x0](ffff9181c830db88) 00000080:00200008:2.0:1787826390.514188:0:163071:0:(file.c:1090:ll_file_open()) START file f-10:[0x240001f59:0x21:0x0](ffff9181c830db88), flags 100102 00000002:00100000:2.0:1787826390.514210:0:163071:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181d0fc2300 x1874663043046656/t21474836503(21474836503) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:2.0:1787826390.514248:0:163071:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f3ee from it ffff9181e003a3c0 00010000:00010000:2.0:1787826390.514267:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843400/0xbef42c0221d6f3ee lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x21:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00ceeb06 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.514276:0:163071:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843400/0xbef42c0221d6f3ee lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x21:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00ceeb06 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826390.514297:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x21:0x0](ffff9181c830db88), inode mode 81a4 mask 40 00000080:00200008:2.0:1787826390.514329:0:163071:0:(file.c:479:ll_file_release()) START file f-10:[0x240001f59:0x21:0x0](ffff9181c830db88), flags 100002 00000002:00100000:2.0:1787826390.514373:0:163071:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181d0fc2300 x1874663043046656/t21474836503(21474836503) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:2.0:1787826390.514383:0:163071:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1787826390.514388:0:163071:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181d0fc0e00 x1874663043046912/t0(0) o35->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:2.0:1787826390.514409:0:163071:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc0e00 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043046912:192.168.203.147@tcp:35:createmany.0 00000100:00100000:3.0:1787826390.515059:0:163071:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6ea8540 going to sleep for 11 seconds 00000100:00100000:3.0:1787826390.527838:0:163071:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc0e00 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043046912:192.168.203.147@tcp:35:createmany.0 00000002:00100000:3.0:1787826390.527875:0:163071:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181d0fc2300 x1874663043046656/t21474836503(21474836503) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:3.0:1787826390.527966:0:163071:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:3.0:1787826390.527985:0:163071:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826390.528004:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 19 lru_type: 1 00010000:00010000:3.0:1787826390.528019:0:163071:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 19 lru_type: 1 00000080:00200000:3.0:1787826390.528026:0:163071:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826390.528033:0:163071:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:3.0:1787826390.528037:0:163071:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f393 00000080:00010000:3.0:1787826390.528044:0:163071:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f393 from it ffffb97500e2bbf8 00010000:00010000:3.0:1787826390.528048:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 19 lru_type: 1 00010000:00010000:3.0:1787826390.528057:0:163071:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 20 lru_type: 1 00000080:00200000:3.0:1787826390.528073:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826390.528079:0:163071:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80b.replay-single, flags=25 00000080:00200000:3.0:1787826390.528204:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x4:0x0](ffff9181d0809148), inode mode 41ed mask 1 00000080:00200000:3.0:1787826390.528211:0:163071:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=remote_dir, flags=25 00000080:00200000:3.0:1787826390.528214:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 1 00000080:00200000:3.0:1787826390.528227:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 3 00000080:00200000:3.0:1787826390.528232:0:163071:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f-11, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), file ffff9181c5456780, open_flags 8042, mode 81a4 opened 0 00000080:00200000:3.0:1787826390.528241:0:163071:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f-11, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), intent=open|creat 00000002:00010000:3.0:1787826390.528272:0:163071:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f-11,[0x240001f59:0x22:0x0]) in obj [0x240001f59:0x16:0x0], intent: open|creat flags 0100103 00010000:00010000:3.0:1787826390.528326:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481c00/0xbef42c0221d6f3f5 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.528334:0:163071:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481c00/0xbef42c0221d6f3f5 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:3.0:1787826390.528344:0:163071:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:3.0:1787826390.528351:0:163071:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c5635f80 x1874663043047168/t0(0) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:3.0:1787826390.528359:0:163071:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481c00/0xbef42c0221d6f3f5 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826390.528376:0:163071:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043047168:192.168.203.147@tcp:101:createmany.0 00000100:00100000:3.0:1787826390.528696:0:163071:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d22a2e40 going to sleep for 11 seconds 00000100:00100000:3.0:1787826390.535653:0:163071:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043047168:192.168.203.147@tcp:101:createmany.0 00010000:00010000:3.0:1787826390.535685:0:163071:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481c00/0xbef42c0221d6f3f5 lrc: 4/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceeb29 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.535718:0:163071:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481c00/0xbef42c0221d6f3f5 lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x22:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceeb29 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.535728:0:163071:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481c00/0xbef42c0221d6f3f5 lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x22:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceeb29 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.535739:0:163071:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481c00/0xbef42c0221d6f3f5 lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x22:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceeb29 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.535780:0:163071:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481c00/0xbef42c0221d6f3f5 lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x22:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceeb29 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.535794:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481c00/0xbef42c0221d6f3f5 lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x22:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceeb29 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.535803:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481c00/0xbef42c0221d6f3f5 lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x22:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceeb29 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.535811:0:163071:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481c00/0xbef42c0221d6f3f5 lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x22:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceeb29 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826390.535824:0:163071:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9181c5635f80 x1874663043047168/t21474836505(21474836505) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:3.0:1787826390.535837:0:163071:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c5635f80 x1874663043047168/t21474836505(21474836505) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826390.535851:0:163071:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481c00/0xbef42c0221d6f3f5 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x22:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceeb29 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:3.0:1787826390.535865:0:163071:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481c00/0xbef42c0221d6f3f5 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x22:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceeb29 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.535875:0:163071:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c77f3000 type 13 mode 4 res 9663684441/34 (0 0) 00000080:00200000:3.0:1787826390.535880:0:163071:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x240001f59:0x16:0x0] suppgids 0 -1: rc 0 00000080:00200000:3.0:1787826390.535894:0:163071:0:(namei.c:1628:ll_create_it()) VFS Op:name=f-11, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), intent=open|creat 00000080:00200000:3.0:1787826390.535920:0:163071:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001f59:0x22:0x0](ffff9181d080a1c8) 00000080:00200000:3.0:1787826390.535996:0:163071:0:(file.c:6843:ll_layout_conf()) [0x240001f59:0x22:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:3.0:1787826390.536001:0:163071:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x240001f59:0x22:0x0](ffff9181d080a1c8) 00000080:00010000:3.0:1787826390.536006:0:163071:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001f59:0x22:0x0](ffff9181d080a1c8) for lock 0xbef42c0221d6f3f5 00000080:00010000:3.0:1787826390.536016:0:163071:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001f59:0x22:0x0](ffff9181d080a1c8) 00000080:00200008:3.0:1787826390.536030:0:163071:0:(file.c:1090:ll_file_open()) START file f-11:[0x240001f59:0x22:0x0](ffff9181d080a1c8), flags 100102 00000002:00100000:3.0:1787826390.536051:0:163071:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c5635f80 x1874663043047168/t21474836505(21474836505) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:3.0:1787826390.536092:0:163071:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f3f5 from it ffff9181c6ede000 00010000:00010000:3.0:1787826390.536098:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481c00/0xbef42c0221d6f3f5 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x22:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00ceeb29 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.536107:0:163071:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481c00/0xbef42c0221d6f3f5 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x22:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00ceeb29 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826390.536129:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x22:0x0](ffff9181d080a1c8), inode mode 81a4 mask 40 00000080:00200008:3.0:1787826390.536160:0:163071:0:(file.c:479:ll_file_release()) START file f-11:[0x240001f59:0x22:0x0](ffff9181d080a1c8), flags 100002 00000002:00100000:3.0:1787826390.536206:0:163071:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181c5635f80 x1874663043047168/t21474836505(21474836505) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:3.0:1787826390.536218:0:163071:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:3.0:1787826390.536223:0:163071:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c5636a00 x1874663043047424/t0(0) o35->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:3.0:1787826390.536244:0:163071:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043047424:192.168.203.147@tcp:35:createmany.0 00000100:00100000:3.0:1787826390.536321:0:163071:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d22a2d80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826390.540959:0:163071:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043047424:192.168.203.147@tcp:35:createmany.0 00000002:00100000:3.0:1787826390.540998:0:163071:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181c5635f80 x1874663043047168/t21474836505(21474836505) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:3.0:1787826390.541070:0:163071:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:3.0:1787826390.541088:0:163071:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826390.541105:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 20 lru_type: 1 00010000:00010000:3.0:1787826390.541120:0:163071:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 20 lru_type: 1 00000080:00200000:3.0:1787826390.541127:0:163071:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826390.541133:0:163071:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:3.0:1787826390.541136:0:163071:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f393 00000080:00010000:3.0:1787826390.541143:0:163071:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f393 from it ffffb97500e2bbf8 00010000:00010000:3.0:1787826390.541146:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 20 lru_type: 1 00010000:00010000:3.0:1787826390.541155:0:163071:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 21 lru_type: 1 00000080:00200000:3.0:1787826390.541169:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826390.541175:0:163071:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80b.replay-single, flags=25 00000080:00200000:3.0:1787826390.541181:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x4:0x0](ffff9181d0809148), inode mode 41ed mask 1 00000080:00200000:3.0:1787826390.541184:0:163071:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=remote_dir, flags=25 00000080:00200000:3.0:1787826390.541187:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 1 00000080:00200000:3.0:1787826390.541199:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 3 00000080:00200000:3.0:1787826390.541203:0:163071:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f-12, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), file ffff9181c5456c00, open_flags 8042, mode 81a4 opened 0 00000080:00200000:3.0:1787826390.541210:0:163071:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f-12, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), intent=open|creat 00000002:00010000:3.0:1787826390.541229:0:163071:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f-12,[0x240001f59:0x23:0x0]) in obj [0x240001f59:0x16:0x0], intent: open|creat flags 0100103 00010000:00010000:3.0:1787826390.541293:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f3fc lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.541306:0:163071:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f3fc lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:3.0:1787826390.541317:0:163071:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:3.0:1787826390.541323:0:163071:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c5634e00 x1874663043047680/t0(0) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:3.0:1787826390.541332:0:163071:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f3fc lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826390.541349:0:163071:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043047680:192.168.203.147@tcp:101:createmany.0 00000100:00100000:3.0:1787826390.541470:0:163071:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d22a2d80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826390.558482:0:163071:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043047680:192.168.203.147@tcp:101:createmany.0 00010000:00010000:3.0:1787826390.558512:0:163071:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f3fc lrc: 4/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceeb4c expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.558542:0:163071:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f3fc lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x23:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceeb4c expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.558551:0:163071:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f3fc lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceeb4c expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.558562:0:163071:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f3fc lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceeb4c expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.558573:0:163071:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f3fc lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceeb4c expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.558592:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f3fc lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceeb4c expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.558601:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f3fc lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceeb4c expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.558608:0:163071:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f3fc lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceeb4c expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826390.558620:0:163071:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9181c5634e00 x1874663043047680/t21474836507(21474836507) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:3.0:1787826390.558632:0:163071:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c5634e00 x1874663043047680/t21474836507(21474836507) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826390.558646:0:163071:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f3fc lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceeb4c expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:3.0:1787826390.558659:0:163071:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f3fc lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceeb4c expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.558669:0:163071:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c77f3000 type 13 mode 4 res 9663684441/35 (0 0) 00000080:00200000:3.0:1787826390.558675:0:163071:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x240001f59:0x16:0x0] suppgids 0 -1: rc 0 00000080:00200000:3.0:1787826390.558689:0:163071:0:(namei.c:1628:ll_create_it()) VFS Op:name=f-12, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), intent=open|creat 00000080:00200000:3.0:1787826390.558719:0:163071:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001f59:0x23:0x0](ffff9181d0809988) 00000080:00200000:3.0:1787826390.558904:0:163071:0:(file.c:6843:ll_layout_conf()) [0x240001f59:0x23:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:3.0:1787826390.558911:0:163071:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x240001f59:0x23:0x0](ffff9181d0809988) 00000080:00010000:3.0:1787826390.558915:0:163071:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001f59:0x23:0x0](ffff9181d0809988) for lock 0xbef42c0221d6f3fc 00000080:00010000:3.0:1787826390.558924:0:163071:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001f59:0x23:0x0](ffff9181d0809988) 00000080:00200008:3.0:1787826390.558939:0:163071:0:(file.c:1090:ll_file_open()) START file f-12:[0x240001f59:0x23:0x0](ffff9181d0809988), flags 100102 00000002:00100000:3.0:1787826390.558961:0:163071:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c5634e00 x1874663043047680/t21474836507(21474836507) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:3.0:1787826390.558995:0:163071:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f3fc from it ffff9181c6ede000 00010000:00010000:3.0:1787826390.559000:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f3fc lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00ceeb4c expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.559009:0:163071:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f3fc lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00ceeb4c expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826390.559030:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x23:0x0](ffff9181d0809988), inode mode 81a4 mask 40 00000080:00200008:3.0:1787826390.559058:0:163071:0:(file.c:479:ll_file_release()) START file f-12:[0x240001f59:0x23:0x0](ffff9181d0809988), flags 100002 00000002:00100000:3.0:1787826390.559098:0:163071:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181c5634e00 x1874663043047680/t21474836507(21474836507) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:3.0:1787826390.559109:0:163071:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:3.0:1787826390.559114:0:163071:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c5634380 x1874663043047936/t0(0) o35->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:3.0:1787826390.559134:0:163071:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043047936:192.168.203.147@tcp:35:createmany.0 00000100:00100000:3.0:1787826390.559213:0:163071:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d22a2180 going to sleep for 11 seconds 00000100:00100000:3.0:1787826390.567008:0:163071:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043047936:192.168.203.147@tcp:35:createmany.0 00000002:00100000:3.0:1787826390.567046:0:163071:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181c5634e00 x1874663043047680/t21474836507(21474836507) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:3.0:1787826390.567124:0:163071:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:3.0:1787826390.567144:0:163071:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826390.567161:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 21 lru_type: 1 00010000:00010000:3.0:1787826390.567177:0:163071:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 21 lru_type: 1 00000080:00200000:3.0:1787826390.567184:0:163071:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826390.567190:0:163071:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:3.0:1787826390.567193:0:163071:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f393 00000080:00010000:3.0:1787826390.567199:0:163071:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f393 from it ffffb97500e2bbf8 00010000:00010000:3.0:1787826390.567203:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 21 lru_type: 1 00010000:00010000:3.0:1787826390.567211:0:163071:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 22 lru_type: 1 00000080:00200000:3.0:1787826390.567227:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826390.567234:0:163071:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80b.replay-single, flags=25 00000080:00200000:3.0:1787826390.567239:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x4:0x0](ffff9181d0809148), inode mode 41ed mask 1 00000080:00200000:3.0:1787826390.567243:0:163071:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=remote_dir, flags=25 00000080:00200000:3.0:1787826390.567246:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 1 00000080:00200000:3.0:1787826390.568249:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 3 00000080:00200000:3.0:1787826390.568272:0:163071:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f-13, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), file ffff9181c5456780, open_flags 8042, mode 81a4 opened 0 00000080:00200000:3.0:1787826390.568281:0:163071:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f-13, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), intent=open|creat 00000002:00010000:3.0:1787826390.568314:0:163071:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f-13,[0x240001f59:0x24:0x0]) in obj [0x240001f59:0x16:0x0], intent: open|creat flags 0100103 00010000:00010000:3.0:1787826390.568374:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481e00/0xbef42c0221d6f403 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.568400:0:163071:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481e00/0xbef42c0221d6f403 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:3.0:1787826390.568412:0:163071:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:3.0:1787826390.568421:0:163071:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c5637480 x1874663043048192/t0(0) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:3.0:1787826390.568430:0:163071:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481e00/0xbef42c0221d6f403 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826390.568451:0:163071:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043048192:192.168.203.147@tcp:101:createmany.0 00000100:00100000:3.0:1787826390.568657:0:163071:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d22a2180 going to sleep for 11 seconds 00000100:00100000:3.0:1787826390.578473:0:163071:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043048192:192.168.203.147@tcp:101:createmany.0 00010000:00010000:3.0:1787826390.578503:0:163071:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481e00/0xbef42c0221d6f403 lrc: 4/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceeb6f expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.578533:0:163071:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481e00/0xbef42c0221d6f403 lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x24:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceeb6f expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.578541:0:163071:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481e00/0xbef42c0221d6f403 lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x24:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceeb6f expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.578552:0:163071:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481e00/0xbef42c0221d6f403 lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x24:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceeb6f expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.578564:0:163071:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481e00/0xbef42c0221d6f403 lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x24:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceeb6f expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.578583:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481e00/0xbef42c0221d6f403 lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x24:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceeb6f expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.578591:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481e00/0xbef42c0221d6f403 lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x24:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceeb6f expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.578598:0:163071:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481e00/0xbef42c0221d6f403 lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x24:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceeb6f expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826390.578611:0:163071:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9181c5637480 x1874663043048192/t21474836509(21474836509) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:3.0:1787826390.578624:0:163071:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c5637480 x1874663043048192/t21474836509(21474836509) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826390.578636:0:163071:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481e00/0xbef42c0221d6f403 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x24:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceeb6f expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:3.0:1787826390.578648:0:163071:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481e00/0xbef42c0221d6f403 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x24:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceeb6f expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.578657:0:163071:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c77f3000 type 13 mode 4 res 9663684441/36 (0 0) 00000080:00200000:3.0:1787826390.578663:0:163071:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x240001f59:0x16:0x0] suppgids 0 -1: rc 0 00000080:00200000:3.0:1787826390.578676:0:163071:0:(namei.c:1628:ll_create_it()) VFS Op:name=f-13, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), intent=open|creat 00000080:00200000:3.0:1787826390.578706:0:163071:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001f59:0x24:0x0](ffff9181d080db88) 00000080:00200000:3.0:1787826390.578826:0:163071:0:(file.c:6843:ll_layout_conf()) [0x240001f59:0x24:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:3.0:1787826390.578832:0:163071:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x240001f59:0x24:0x0](ffff9181d080db88) 00000080:00010000:3.0:1787826390.578836:0:163071:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001f59:0x24:0x0](ffff9181d080db88) for lock 0xbef42c0221d6f403 00000080:00010000:3.0:1787826390.578846:0:163071:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001f59:0x24:0x0](ffff9181d080db88) 00000080:00200008:3.0:1787826390.578860:0:163071:0:(file.c:1090:ll_file_open()) START file f-13:[0x240001f59:0x24:0x0](ffff9181d080db88), flags 100102 00000002:00100000:3.0:1787826390.578881:0:163071:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c5637480 x1874663043048192/t21474836509(21474836509) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:3.0:1787826390.578924:0:163071:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f403 from it ffff9181c6ede000 00010000:00010000:3.0:1787826390.578928:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481e00/0xbef42c0221d6f403 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x24:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00ceeb6f expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.578937:0:163071:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481e00/0xbef42c0221d6f403 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x24:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00ceeb6f expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826390.578957:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x24:0x0](ffff9181d080db88), inode mode 81a4 mask 40 00000080:00200008:3.0:1787826390.578984:0:163071:0:(file.c:479:ll_file_release()) START file f-13:[0x240001f59:0x24:0x0](ffff9181d080db88), flags 100002 00000002:00100000:3.0:1787826390.579024:0:163071:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181c5637480 x1874663043048192/t21474836509(21474836509) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:3.0:1787826390.579035:0:163071:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:3.0:1787826390.579040:0:163071:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c5634700 x1874663043048448/t0(0) o35->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:3.0:1787826390.579060:0:163071:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634700 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043048448:192.168.203.147@tcp:35:createmany.0 00000100:00100000:3.0:1787826390.579134:0:163071:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d22a2000 going to sleep for 11 seconds 00000100:00100000:3.0:1787826390.583405:0:163071:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634700 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043048448:192.168.203.147@tcp:35:createmany.0 00000002:00100000:2.0:1787826390.583543:0:163071:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181c5637480 x1874663043048192/t21474836509(21474836509) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1787826390.583628:0:163071:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:2.0:1787826390.583647:0:163071:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826390.583665:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 22 lru_type: 1 00010000:00010000:2.0:1787826390.583680:0:163071:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 22 lru_type: 1 00000080:00200000:2.0:1787826390.583689:0:163071:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826390.583695:0:163071:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:2.0:1787826390.583698:0:163071:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f393 00000080:00010000:2.0:1787826390.583704:0:163071:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f393 from it ffffb97500e2bbf8 00010000:00010000:2.0:1787826390.583709:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 22 lru_type: 1 00010000:00010000:2.0:1787826390.583717:0:163071:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 23 lru_type: 1 00000080:00200000:2.0:1787826390.583733:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.583739:0:163071:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80b.replay-single, flags=25 00000080:00200000:2.0:1787826390.583771:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x4:0x0](ffff9181d0809148), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.583776:0:163071:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=remote_dir, flags=25 00000080:00200000:2.0:1787826390.583780:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.583793:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 3 00000080:00200000:2.0:1787826390.583798:0:163071:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f-14, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), file ffff9181c4d05c80, open_flags 8042, mode 81a4 opened 0 00000080:00200000:2.0:1787826390.583806:0:163071:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f-14, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), intent=open|creat 00000002:00010000:2.0:1787826390.583825:0:163071:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f-14,[0x240001f59:0x25:0x0]) in obj [0x240001f59:0x16:0x0], intent: open|creat flags 0100103 00010000:00010000:2.0:1787826390.583875:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842800/0xbef42c0221d6f40a lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.583884:0:163071:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842800/0xbef42c0221d6f40a lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1787826390.583896:0:163071:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1787826390.583903:0:163071:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181d0fc0a80 x1874663043048704/t0(0) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1787826390.583913:0:163071:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842800/0xbef42c0221d6f40a lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826390.583934:0:163071:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181d0fc0a80 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043048704:192.168.203.147@tcp:101:createmany.0 00000100:00100000:2.0:1787826390.584079:0:163071:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6ea8f00 going to sleep for 11 seconds 00000100:00100000:2.0:1787826390.590972:0:163071:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181d0fc0a80 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043048704:192.168.203.147@tcp:101:createmany.0 00010000:00010000:2.0:1787826390.591000:0:163071:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842800/0xbef42c0221d6f40a lrc: 4/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceeb92 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.591030:0:163071:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842800/0xbef42c0221d6f40a lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x25:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceeb92 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.591038:0:163071:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842800/0xbef42c0221d6f40a lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x25:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceeb92 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.591047:0:163071:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842800/0xbef42c0221d6f40a lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x25:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceeb92 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.591058:0:163071:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842800/0xbef42c0221d6f40a lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x25:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceeb92 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.591069:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842800/0xbef42c0221d6f40a lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x25:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceeb92 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.591077:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842800/0xbef42c0221d6f40a lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x25:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceeb92 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.591088:0:163071:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842800/0xbef42c0221d6f40a lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x25:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceeb92 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787826390.591100:0:163071:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9181d0fc0a80 x1874663043048704/t21474836511(21474836511) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:2.0:1787826390.591111:0:163071:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181d0fc0a80 x1874663043048704/t21474836511(21474836511) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787826390.591124:0:163071:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842800/0xbef42c0221d6f40a lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x25:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceeb92 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:2.0:1787826390.591136:0:163071:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842800/0xbef42c0221d6f40a lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x25:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceeb92 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.591146:0:163071:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c77f3000 type 13 mode 4 res 9663684441/37 (0 0) 00000080:00200000:2.0:1787826390.591153:0:163071:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x240001f59:0x16:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1787826390.591165:0:163071:0:(namei.c:1628:ll_create_it()) VFS Op:name=f-14, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), intent=open|creat 00000080:00200000:2.0:1787826390.591191:0:163071:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001f59:0x25:0x0](ffff9181c830f448) 00000080:00200000:2.0:1787826390.591278:0:163071:0:(file.c:6843:ll_layout_conf()) [0x240001f59:0x25:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:2.0:1787826390.591292:0:163071:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x240001f59:0x25:0x0](ffff9181c830f448) 00000080:00010000:2.0:1787826390.591296:0:163071:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001f59:0x25:0x0](ffff9181c830f448) for lock 0xbef42c0221d6f40a 00000080:00010000:2.0:1787826390.591304:0:163071:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001f59:0x25:0x0](ffff9181c830f448) 00000080:00200008:2.0:1787826390.591317:0:163071:0:(file.c:1090:ll_file_open()) START file f-14:[0x240001f59:0x25:0x0](ffff9181c830f448), flags 100102 00000002:00100000:2.0:1787826390.591339:0:163071:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181d0fc0a80 x1874663043048704/t21474836511(21474836511) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:2.0:1787826390.591374:0:163071:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f40a from it ffff9181e003a3c0 00010000:00010000:2.0:1787826390.591380:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842800/0xbef42c0221d6f40a lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x25:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00ceeb92 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.591403:0:163071:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842800/0xbef42c0221d6f40a lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x25:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00ceeb92 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826390.591424:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x25:0x0](ffff9181c830f448), inode mode 81a4 mask 40 00000080:00200008:2.0:1787826390.591455:0:163071:0:(file.c:479:ll_file_release()) START file f-14:[0x240001f59:0x25:0x0](ffff9181c830f448), flags 100002 00000002:00100000:2.0:1787826390.591510:0:163071:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181d0fc0a80 x1874663043048704/t21474836511(21474836511) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:2.0:1787826390.591520:0:163071:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1787826390.591526:0:163071:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c49e5180 x1874663043048960/t0(0) o35->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:2.0:1787826390.591546:0:163071:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c49e5180 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043048960:192.168.203.147@tcp:35:createmany.0 00000100:00100000:2.0:1787826390.591614:0:163071:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6ea8300 going to sleep for 11 seconds 00000100:00100000:2.0:1787826390.597448:0:163071:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c49e5180 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043048960:192.168.203.147@tcp:35:createmany.0 00000002:00100000:2.0:1787826390.597500:0:163071:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181d0fc0a80 x1874663043048704/t21474836511(21474836511) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1787826390.597581:0:163071:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:2.0:1787826390.597600:0:163071:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826390.597618:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 23 lru_type: 1 00010000:00010000:2.0:1787826390.597632:0:163071:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 23 lru_type: 1 00000080:00200000:2.0:1787826390.597645:0:163071:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826390.597651:0:163071:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:2.0:1787826390.597655:0:163071:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f393 00000080:00010000:2.0:1787826390.597661:0:163071:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f393 from it ffffb97500e2bbf8 00010000:00010000:2.0:1787826390.597665:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 23 lru_type: 1 00010000:00010000:2.0:1787826390.597673:0:163071:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 24 lru_type: 1 00000080:00200000:2.0:1787826390.597687:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.597694:0:163071:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80b.replay-single, flags=25 00000080:00200000:2.0:1787826390.597699:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x4:0x0](ffff9181d0809148), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.597703:0:163071:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=remote_dir, flags=25 00000080:00200000:2.0:1787826390.597707:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.597722:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 3 00000080:00200000:2.0:1787826390.597727:0:163071:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f-15, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), file ffff9181c4d04000, open_flags 8042, mode 81a4 opened 0 00000080:00200000:2.0:1787826390.597734:0:163071:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f-15, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), intent=open|creat 00000002:00010000:2.0:1787826390.597772:0:163071:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f-15,[0x240001f59:0x26:0x0]) in obj [0x240001f59:0x16:0x0], intent: open|creat flags 0100103 00010000:00010000:2.0:1787826390.597822:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842200/0xbef42c0221d6f411 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.597831:0:163071:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842200/0xbef42c0221d6f411 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1787826390.597842:0:163071:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1787826390.597849:0:163071:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c49e4700 x1874663043049216/t0(0) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1787826390.597859:0:163071:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842200/0xbef42c0221d6f411 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826390.597877:0:163071:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c49e4700 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043049216:192.168.203.147@tcp:101:createmany.0 00000100:00100000:2.0:1787826390.598011:0:163071:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6ea8300 going to sleep for 11 seconds 00000100:00100000:2.0:1787826390.602914:0:163071:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c49e4700 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043049216:192.168.203.147@tcp:101:createmany.0 00010000:00010000:2.0:1787826390.602943:0:163071:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842200/0xbef42c0221d6f411 lrc: 4/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceebb5 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.602973:0:163071:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842200/0xbef42c0221d6f411 lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x26:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceebb5 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.602981:0:163071:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842200/0xbef42c0221d6f411 lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x26:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceebb5 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.603583:0:163071:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842200/0xbef42c0221d6f411 lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x26:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceebb5 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.603595:0:163071:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842200/0xbef42c0221d6f411 lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x26:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceebb5 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.603611:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842200/0xbef42c0221d6f411 lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x26:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceebb5 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.603619:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842200/0xbef42c0221d6f411 lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x26:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceebb5 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.603632:0:163071:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842200/0xbef42c0221d6f411 lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x26:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceebb5 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787826390.603644:0:163071:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9181c49e4700 x1874663043049216/t21474836513(21474836513) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:2.0:1787826390.603658:0:163071:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c49e4700 x1874663043049216/t21474836513(21474836513) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787826390.603672:0:163071:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842200/0xbef42c0221d6f411 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x26:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceebb5 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:2.0:1787826390.603688:0:163071:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842200/0xbef42c0221d6f411 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x26:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceebb5 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.603698:0:163071:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c77f3000 type 13 mode 4 res 9663684441/38 (0 0) 00000080:00200000:2.0:1787826390.603703:0:163071:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x240001f59:0x16:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1787826390.603716:0:163071:0:(namei.c:1628:ll_create_it()) VFS Op:name=f-15, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), intent=open|creat 00000080:00200000:2.0:1787826390.603907:0:163071:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001f59:0x26:0x0](ffff9181c830e3c8) 00000080:00200000:2.0:1787826390.603997:0:163071:0:(file.c:6843:ll_layout_conf()) [0x240001f59:0x26:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:2.0:1787826390.604003:0:163071:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x240001f59:0x26:0x0](ffff9181c830e3c8) 00000080:00010000:2.0:1787826390.604006:0:163071:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001f59:0x26:0x0](ffff9181c830e3c8) for lock 0xbef42c0221d6f411 00000080:00010000:2.0:1787826390.604016:0:163071:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001f59:0x26:0x0](ffff9181c830e3c8) 00000080:00200008:2.0:1787826390.604030:0:163071:0:(file.c:1090:ll_file_open()) START file f-15:[0x240001f59:0x26:0x0](ffff9181c830e3c8), flags 100102 00000002:00100000:2.0:1787826390.604066:0:163071:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c49e4700 x1874663043049216/t21474836513(21474836513) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:2.0:1787826390.604106:0:163071:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f411 from it ffff9181e003a3c0 00010000:00010000:2.0:1787826390.604111:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842200/0xbef42c0221d6f411 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x26:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00ceebb5 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.604119:0:163071:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842200/0xbef42c0221d6f411 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x26:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00ceebb5 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826390.604140:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x26:0x0](ffff9181c830e3c8), inode mode 81a4 mask 40 00000080:00200008:2.0:1787826390.604171:0:163071:0:(file.c:479:ll_file_release()) START file f-15:[0x240001f59:0x26:0x0](ffff9181c830e3c8), flags 100002 00000002:00100000:2.0:1787826390.604213:0:163071:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181c49e4700 x1874663043049216/t21474836513(21474836513) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:2.0:1787826390.604224:0:163071:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1787826390.604229:0:163071:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c49e7b80 x1874663043049472/t0(0) o35->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:2.0:1787826390.604251:0:163071:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c49e7b80 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043049472:192.168.203.147@tcp:35:createmany.0 00000100:00100000:2.0:1787826390.604482:0:163071:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6ea8cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826390.609115:0:163071:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c49e7b80 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043049472:192.168.203.147@tcp:35:createmany.0 00000002:00100000:2.0:1787826390.609152:0:163071:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181c49e4700 x1874663043049216/t21474836513(21474836513) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1787826390.609227:0:163071:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:2.0:1787826390.609244:0:163071:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826390.609263:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 24 lru_type: 1 00010000:00010000:2.0:1787826390.609276:0:163071:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 24 lru_type: 1 00000080:00200000:2.0:1787826390.609305:0:163071:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826390.609311:0:163071:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:2.0:1787826390.609314:0:163071:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f393 00000080:00010000:2.0:1787826390.609322:0:163071:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f393 from it ffffb97500e2bbf8 00010000:00010000:2.0:1787826390.609326:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 24 lru_type: 1 00010000:00010000:2.0:1787826390.609335:0:163071:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 25 lru_type: 1 00000080:00200000:2.0:1787826390.609349:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.609356:0:163071:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80b.replay-single, flags=25 00000080:00200000:2.0:1787826390.609362:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x4:0x0](ffff9181d0809148), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.609367:0:163071:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=remote_dir, flags=25 00000080:00200000:2.0:1787826390.609370:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.609383:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 3 00000080:00200000:2.0:1787826390.609388:0:163071:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f-16, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), file ffff9181c4d05c80, open_flags 8042, mode 81a4 opened 0 00000080:00200000:2.0:1787826390.609395:0:163071:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f-16, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), intent=open|creat 00000002:00010000:2.0:1787826390.609415:0:163071:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f-16,[0x240001f59:0x27:0x0]) in obj [0x240001f59:0x16:0x0], intent: open|creat flags 0100103 00010000:00010000:2.0:1787826390.609475:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842000/0xbef42c0221d6f418 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.609484:0:163071:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842000/0xbef42c0221d6f418 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1787826390.609495:0:163071:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1787826390.609501:0:163071:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c49e5f80 x1874663043049728/t0(0) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1787826390.609511:0:163071:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842000/0xbef42c0221d6f418 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826390.609529:0:163071:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c49e5f80 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043049728:192.168.203.147@tcp:101:createmany.0 00000100:00100000:2.0:1787826390.609659:0:163071:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6ea8cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826390.620025:0:163071:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c49e5f80 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043049728:192.168.203.147@tcp:101:createmany.0 00010000:00010000:2.0:1787826390.620064:0:163071:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842000/0xbef42c0221d6f418 lrc: 4/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceebd8 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.620292:0:163071:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842000/0xbef42c0221d6f418 lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x27:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceebd8 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.620303:0:163071:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842000/0xbef42c0221d6f418 lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceebd8 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.620318:0:163071:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842000/0xbef42c0221d6f418 lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceebd8 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.620330:0:163071:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842000/0xbef42c0221d6f418 lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceebd8 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.620347:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842000/0xbef42c0221d6f418 lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceebd8 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.620356:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842000/0xbef42c0221d6f418 lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceebd8 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.620370:0:163071:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842000/0xbef42c0221d6f418 lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceebd8 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787826390.620385:0:163071:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9181c49e5f80 x1874663043049728/t21474836515(21474836515) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:2.0:1787826390.620400:0:163071:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c49e5f80 x1874663043049728/t21474836515(21474836515) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787826390.620416:0:163071:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842000/0xbef42c0221d6f418 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceebd8 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:2.0:1787826390.620432:0:163071:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842000/0xbef42c0221d6f418 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceebd8 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.620444:0:163071:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c77f3000 type 13 mode 4 res 9663684441/39 (0 0) 00000080:00200000:2.0:1787826390.620452:0:163071:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x240001f59:0x16:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1787826390.620481:0:163071:0:(namei.c:1628:ll_create_it()) VFS Op:name=f-16, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), intent=open|creat 00000080:00200000:2.0:1787826390.620594:0:163071:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001f59:0x27:0x0](ffff9181c8309148) 00000080:00200000:2.0:1787826390.620925:0:163071:0:(file.c:6843:ll_layout_conf()) [0x240001f59:0x27:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:2.0:1787826390.620932:0:163071:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x240001f59:0x27:0x0](ffff9181c8309148) 00000080:00010000:2.0:1787826390.620936:0:163071:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001f59:0x27:0x0](ffff9181c8309148) for lock 0xbef42c0221d6f418 00000080:00010000:2.0:1787826390.620950:0:163071:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001f59:0x27:0x0](ffff9181c8309148) 00000080:00200008:2.0:1787826390.620968:0:163071:0:(file.c:1090:ll_file_open()) START file f-16:[0x240001f59:0x27:0x0](ffff9181c8309148), flags 100102 00000002:00100000:0.0:1787826390.621077:0:163071:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c49e5f80 x1874663043049728/t21474836515(21474836515) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:0.0:1787826390.621616:0:163071:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f418 from it ffff9181e003a3c0 00010000:00010000:0.0:1787826390.621625:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842000/0xbef42c0221d6f418 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00ceebd8 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.621642:0:163071:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842000/0xbef42c0221d6f418 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00ceebd8 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826390.621672:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x27:0x0](ffff9181c8309148), inode mode 81a4 mask 40 00000080:00200008:0.0:1787826390.621925:0:163071:0:(file.c:479:ll_file_release()) START file f-16:[0x240001f59:0x27:0x0](ffff9181c8309148), flags 100002 00000002:00100000:0.0:1787826390.622000:0:163071:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181c49e5f80 x1874663043049728/t21474836515(21474836515) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:0.0:1787826390.622012:0:163071:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:0.0:1787826390.622017:0:163071:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c55a6d80 x1874663043049984/t0(0) o35->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:0.0:1787826390.622045:0:163071:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043049984:192.168.203.147@tcp:35:createmany.0 00000100:00100000:0.0:1787826390.622710:0:163071:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d01a5600 going to sleep for 11 seconds 00010000:00010000:3.0:1787826390.635447:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1787826390.636425:0:163071:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043049984:192.168.203.147@tcp:35:createmany.0 00000002:00100000:0.0:1787826390.636484:0:163071:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181c49e5f80 x1874663043049728/t21474836515(21474836515) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787826390.636602:0:163071:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:0.0:1787826390.636625:0:163071:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826390.636648:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 25 lru_type: 1 00010000:00010000:0.0:1787826390.636666:0:163071:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 25 lru_type: 1 00000080:00200000:0.0:1787826390.636675:0:163071:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826390.636683:0:163071:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:0.0:1787826390.636687:0:163071:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f393 00000080:00010000:0.0:1787826390.636695:0:163071:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f393 from it ffffb97500e2bbf8 00010000:00010000:0.0:1787826390.636700:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 25 lru_type: 1 00010000:00010000:0.0:1787826390.636709:0:163071:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 26 lru_type: 1 00000080:00200000:0.0:1787826390.636726:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826390.636735:0:163071:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80b.replay-single, flags=25 00000080:00200000:0.0:1787826390.636784:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x4:0x0](ffff9181d0809148), inode mode 41ed mask 1 00000080:00200000:0.0:1787826390.636791:0:163071:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=remote_dir, flags=25 00000080:00200000:0.0:1787826390.636795:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 1 00000080:00200000:0.0:1787826390.636812:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 3 00000080:00200000:0.0:1787826390.636817:0:163071:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f-17, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), file ffff9181c51e8480, open_flags 8042, mode 81a4 opened 0 00000080:00200000:0.0:1787826390.636828:0:163071:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f-17, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), intent=open|creat 00000002:00010000:0.0:1787826390.636853:0:163071:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f-17,[0x240001f59:0x28:0x0]) in obj [0x240001f59:0x16:0x0], intent: open|creat flags 0100103 00010000:00010000:0.0:1787826390.636918:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0512a00/0xbef42c0221d6f41f lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.636927:0:163071:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0512a00/0xbef42c0221d6f41f lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:0.0:1787826390.636938:0:163071:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:0.0:1787826390.636946:0:163071:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c55a6680 x1874663043050240/t0(0) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:0.0:1787826390.636955:0:163071:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0512a00/0xbef42c0221d6f41f lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826390.636978:0:163071:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043050240:192.168.203.147@tcp:101:createmany.0 00000100:00100000:0.0:1787826390.639362:0:163071:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d01a5600 going to sleep for 11 seconds 00000100:00100000:0.0:1787826390.644540:0:163071:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043050240:192.168.203.147@tcp:101:createmany.0 00010000:00010000:0.0:1787826390.644579:0:163071:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0512a00/0xbef42c0221d6f41f lrc: 4/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceebfb expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.644616:0:163071:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0512a00/0xbef42c0221d6f41f lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x28:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceebfb expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.644631:0:163071:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0512a00/0xbef42c0221d6f41f lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceebfb expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.644645:0:163071:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0512a00/0xbef42c0221d6f41f lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceebfb expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.644656:0:163071:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0512a00/0xbef42c0221d6f41f lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceebfb expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.644671:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0512a00/0xbef42c0221d6f41f lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceebfb expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.644679:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0512a00/0xbef42c0221d6f41f lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceebfb expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.644688:0:163071:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0512a00/0xbef42c0221d6f41f lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceebfb expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826390.644716:0:163071:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9181c55a6680 x1874663043050240/t21474836517(21474836517) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:0.0:1787826390.644732:0:163071:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c55a6680 x1874663043050240/t21474836517(21474836517) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787826390.644771:0:163071:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0512a00/0xbef42c0221d6f41f lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceebfb expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:0.0:1787826390.644789:0:163071:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0512a00/0xbef42c0221d6f41f lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceebfb expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.644801:0:163071:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c77f3000 type 13 mode 4 res 9663684441/40 (0 0) 00000080:00200000:0.0:1787826390.644808:0:163071:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x240001f59:0x16:0x0] suppgids 0 -1: rc 0 00000080:00200000:0.0:1787826390.644826:0:163071:0:(namei.c:1628:ll_create_it()) VFS Op:name=f-17, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), intent=open|creat 00000080:00200000:0.0:1787826390.644862:0:163071:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001f59:0x28:0x0](ffff9181c83f2a08) 00000080:00200000:0.0:1787826390.644970:0:163071:0:(file.c:6843:ll_layout_conf()) [0x240001f59:0x28:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:0.0:1787826390.644976:0:163071:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x240001f59:0x28:0x0](ffff9181c83f2a08) 00000080:00010000:0.0:1787826390.644980:0:163071:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001f59:0x28:0x0](ffff9181c83f2a08) for lock 0xbef42c0221d6f41f 00000080:00010000:0.0:1787826390.644993:0:163071:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001f59:0x28:0x0](ffff9181c83f2a08) 00000080:00200008:0.0:1787826390.645011:0:163071:0:(file.c:1090:ll_file_open()) START file f-17:[0x240001f59:0x28:0x0](ffff9181c83f2a08), flags 100102 00000002:00100000:0.0:1787826390.645045:0:163071:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c55a6680 x1874663043050240/t21474836517(21474836517) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:0.0:1787826390.645666:0:163071:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f41f from it ffff9181c5c09cc0 00010000:00010000:0.0:1787826390.645676:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0512a00/0xbef42c0221d6f41f lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00ceebfb expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.645694:0:163071:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0512a00/0xbef42c0221d6f41f lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00ceebfb expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1787826390.646359:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x28:0x0](ffff9181c83f2a08), inode mode 81a4 mask 40 00000080:00200008:1.0:1787826390.646414:0:163071:0:(file.c:479:ll_file_release()) START file f-17:[0x240001f59:0x28:0x0](ffff9181c83f2a08), flags 100002 00000002:00100000:1.0:1787826390.646502:0:163071:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181c55a6680 x1874663043050240/t21474836517(21474836517) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:1.0:1787826390.646516:0:163071:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:1.0:1787826390.646525:0:163071:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c35b0e00 x1874663043050496/t0(0) o35->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:1.0:1787826390.646552:0:163071:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043050496:192.168.203.147@tcp:35:createmany.0 00000100:00100000:1.0:1787826390.646672:0:163071:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181e0156e40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826390.654278:0:163071:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043050496:192.168.203.147@tcp:35:createmany.0 00000002:00100000:2.0:1787826390.654332:0:163071:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181c55a6680 x1874663043050240/t21474836517(21474836517) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1787826390.654449:0:163071:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:2.0:1787826390.654483:0:163071:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826390.654508:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 26 lru_type: 1 00010000:00010000:2.0:1787826390.654525:0:163071:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 26 lru_type: 1 00000080:00200000:2.0:1787826390.654538:0:163071:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826390.654553:0:163071:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:2.0:1787826390.654557:0:163071:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f393 00000080:00010000:2.0:1787826390.654566:0:163071:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f393 from it ffffb97500e2bbf8 00010000:00010000:2.0:1787826390.654571:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 26 lru_type: 1 00010000:00010000:2.0:1787826390.654581:0:163071:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 27 lru_type: 1 00000080:00200000:2.0:1787826390.654600:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.654608:0:163071:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80b.replay-single, flags=25 00000080:00200000:2.0:1787826390.654615:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x4:0x0](ffff9181d0809148), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.654618:0:163071:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=remote_dir, flags=25 00000080:00200000:2.0:1787826390.654621:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.654637:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 3 00000080:00200000:2.0:1787826390.654643:0:163071:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f-18, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), file ffff9181c4d04000, open_flags 8042, mode 81a4 opened 0 00000080:00200000:2.0:1787826390.654653:0:163071:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f-18, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), intent=open|creat 00000002:00010000:2.0:1787826390.654689:0:163071:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f-18,[0x240001f59:0x29:0x0]) in obj [0x240001f59:0x16:0x0], intent: open|creat flags 0100103 00010000:00010000:2.0:1787826390.654799:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f426 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.654809:0:163071:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f426 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1787826390.654821:0:163071:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1787826390.654829:0:163071:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c49e5880 x1874663043050752/t0(0) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1787826390.654848:0:163071:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f426 lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826390.654873:0:163071:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c49e5880 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043050752:192.168.203.147@tcp:101:createmany.0 00000100:00100000:2.0:1787826390.655370:0:163071:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6ea8d80 going to sleep for 11 seconds 00000100:00100000:2.0:1787826390.661818:0:163071:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c49e5880 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043050752:192.168.203.147@tcp:101:createmany.0 00010000:00010000:2.0:1787826390.661923:0:163071:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f426 lrc: 4/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceec1e expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.661960:0:163071:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f426 lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x29:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceec1e expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.661968:0:163071:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f426 lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x29:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceec1e expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.661980:0:163071:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f426 lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x29:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceec1e expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.661991:0:163071:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f426 lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x29:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceec1e expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.662006:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f426 lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x29:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceec1e expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.662014:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f426 lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x29:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceec1e expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.662021:0:163071:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f426 lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x29:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceec1e expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787826390.662035:0:163071:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9181c49e5880 x1874663043050752/t21474836519(21474836519) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:2.0:1787826390.662173:0:163071:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c49e5880 x1874663043050752/t21474836519(21474836519) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787826390.662191:0:163071:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f426 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x29:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceec1e expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:2.0:1787826390.662205:0:163071:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f426 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x29:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceec1e expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.662217:0:163071:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c77f3000 type 13 mode 4 res 9663684441/41 (0 0) 00000080:00200000:2.0:1787826390.662225:0:163071:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x240001f59:0x16:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1787826390.662241:0:163071:0:(namei.c:1628:ll_create_it()) VFS Op:name=f-18, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), intent=open|creat 00000080:00200000:2.0:1787826390.662278:0:163071:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001f59:0x29:0x0](ffff9181c83080c8) 00000080:00200000:2.0:1787826390.662380:0:163071:0:(file.c:6843:ll_layout_conf()) [0x240001f59:0x29:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:2.0:1787826390.662385:0:163071:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x240001f59:0x29:0x0](ffff9181c83080c8) 00000080:00010000:2.0:1787826390.662389:0:163071:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001f59:0x29:0x0](ffff9181c83080c8) for lock 0xbef42c0221d6f426 00000080:00010000:2.0:1787826390.662400:0:163071:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001f59:0x29:0x0](ffff9181c83080c8) 00000080:00200008:2.0:1787826390.662417:0:163071:0:(file.c:1090:ll_file_open()) START file f-18:[0x240001f59:0x29:0x0](ffff9181c83080c8), flags 100102 00000002:00100000:2.0:1787826390.662442:0:163071:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c49e5880 x1874663043050752/t21474836519(21474836519) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:2.0:1787826390.662499:0:163071:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f426 from it ffff9181e003a480 00010000:00010000:2.0:1787826390.662504:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f426 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x29:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00ceec1e expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.662515:0:163071:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f426 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x29:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00ceec1e expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826390.662541:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x29:0x0](ffff9181c83080c8), inode mode 81a4 mask 40 00000080:00200008:2.0:1787826390.664312:0:163071:0:(file.c:479:ll_file_release()) START file f-18:[0x240001f59:0x29:0x0](ffff9181c83080c8), flags 100002 00000002:00100000:2.0:1787826390.664422:0:163071:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181c49e5880 x1874663043050752/t21474836519(21474836519) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:2.0:1787826390.664437:0:163071:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1787826390.664443:0:163071:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c49e5500 x1874663043051008/t0(0) o35->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:2.0:1787826390.664484:0:163071:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c49e5500 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043051008:192.168.203.147@tcp:35:createmany.0 00000100:00100000:2.0:1787826390.664621:0:163071:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6ea8540 going to sleep for 11 seconds 00000100:00100000:2.0:1787826390.671919:0:163071:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c49e5500 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043051008:192.168.203.147@tcp:35:createmany.0 00000002:00100000:2.0:1787826390.671970:0:163071:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181c49e5880 x1874663043050752/t21474836519(21474836519) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1787826390.672281:0:163071:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:2.0:1787826390.672327:0:163071:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826390.672349:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 27 lru_type: 1 00010000:00010000:2.0:1787826390.672364:0:163071:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 27 lru_type: 1 00000080:00200000:2.0:1787826390.672372:0:163071:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826390.672379:0:163071:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:2.0:1787826390.672383:0:163071:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f393 00000080:00010000:2.0:1787826390.672392:0:163071:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f393 from it ffffb97500e2bbf8 00010000:00010000:2.0:1787826390.672396:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 27 lru_type: 1 00010000:00010000:2.0:1787826390.672406:0:163071:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 28 lru_type: 1 00000080:00200000:2.0:1787826390.672425:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.672433:0:163071:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80b.replay-single, flags=25 00000080:00200000:2.0:1787826390.672444:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x4:0x0](ffff9181d0809148), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.672448:0:163071:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=remote_dir, flags=25 00000080:00200000:2.0:1787826390.672452:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.672698:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 3 00000080:00200000:2.0:1787826390.672705:0:163071:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f-19, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), file ffff9181c4d04480, open_flags 8042, mode 81a4 opened 0 00000080:00200000:2.0:1787826390.672715:0:163071:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f-19, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), intent=open|creat 00000002:00010000:2.0:1787826390.672742:0:163071:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f-19,[0x240001f59:0x2a:0x0]) in obj [0x240001f59:0x16:0x0], intent: open|creat flags 0100103 00010000:00010000:2.0:1787826390.673061:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843600/0xbef42c0221d6f42d lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.673076:0:163071:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843600/0xbef42c0221d6f42d lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1787826390.673096:0:163071:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1787826390.673107:0:163071:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c49e6a00 x1874663043051264/t0(0) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1787826390.673117:0:163071:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843600/0xbef42c0221d6f42d lrc: 3/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163071 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826390.673142:0:163071:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c49e6a00 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043051264:192.168.203.147@tcp:101:createmany.0 00000100:00100000:2.0:1787826390.673341:0:163071:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c6ea8540 going to sleep for 11 seconds 00000100:00100000:2.0:1787826390.680077:0:163071:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c49e6a00 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043051264:192.168.203.147@tcp:101:createmany.0 00010000:00010000:2.0:1787826390.680122:0:163071:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843600/0xbef42c0221d6f42d lrc: 4/0,1 mode: --/CW res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceec41 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.680159:0:163071:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843600/0xbef42c0221d6f42d lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceec41 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.680168:0:163071:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843600/0xbef42c0221d6f42d lrc: 4/0,1 mode: --/PR res: [0x240001f59:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceec41 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.680189:0:163071:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843600/0xbef42c0221d6f42d lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceec41 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.680202:0:163071:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843600/0xbef42c0221d6f42d lrc: 4/0,1 mode: PR/PR res: [0x240001f59:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceec41 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.680223:0:163071:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843600/0xbef42c0221d6f42d lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceec41 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.680231:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843600/0xbef42c0221d6f42d lrc: 5/1,1 mode: PR/PR res: [0x240001f59:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceec41 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.680239:0:163071:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843600/0xbef42c0221d6f42d lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceec41 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787826390.680252:0:163071:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9181c49e6a00 x1874663043051264/t21474836521(21474836521) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:2.0:1787826390.680267:0:163071:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c49e6a00 x1874663043051264/t21474836521(21474836521) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787826390.680284:0:163071:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843600/0xbef42c0221d6f42d lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceec41 expref: -99 pid: 163071 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:2.0:1787826390.680298:0:163071:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843600/0xbef42c0221d6f42d lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceec41 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.680311:0:163071:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c77f3000 type 13 mode 4 res 9663684441/42 (0 0) 00000080:00200000:2.0:1787826390.680318:0:163071:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x240001f59:0x16:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1787826390.680335:0:163071:0:(namei.c:1628:ll_create_it()) VFS Op:name=f-19, dir=[0x240001f59:0x16:0x0](ffff9181d080f448), intent=open|creat 00000080:00200000:2.0:1787826390.680367:0:163071:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001f59:0x2a:0x0](ffff9181c830d348) 00000080:00200000:2.0:1787826390.680469:0:163071:0:(file.c:6843:ll_layout_conf()) [0x240001f59:0x2a:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:2.0:1787826390.680476:0:163071:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x240001f59:0x2a:0x0](ffff9181c830d348) 00000080:00010000:2.0:1787826390.680480:0:163071:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001f59:0x2a:0x0](ffff9181c830d348) for lock 0xbef42c0221d6f42d 00000080:00010000:2.0:1787826390.680491:0:163071:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001f59:0x2a:0x0](ffff9181c830d348) 00000080:00200008:2.0:1787826390.680510:0:163071:0:(file.c:1090:ll_file_open()) START file f-19:[0x240001f59:0x2a:0x0](ffff9181c830d348), flags 100102 00000002:00100000:2.0:1787826390.680536:0:163071:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c49e6a00 x1874663043051264/t21474836521(21474836521) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:2.0:1787826390.680590:0:163071:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f42d from it ffff9181e003a480 00010000:00010000:2.0:1787826390.680596:0:163071:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843600/0xbef42c0221d6f42d lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00ceec41 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.680606:0:163071:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843600/0xbef42c0221d6f42d lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00ceec41 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826390.680631:0:163071:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x2a:0x0](ffff9181c830d348), inode mode 81a4 mask 40 00000080:00200008:2.0:1787826390.680670:0:163071:0:(file.c:479:ll_file_release()) START file f-19:[0x240001f59:0x2a:0x0](ffff9181c830d348), flags 100002 00000002:00100000:2.0:1787826390.680729:0:163071:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181c49e6a00 x1874663043051264/t21474836521(21474836521) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:2.0:1787826390.680740:0:163071:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1787826390.680810:0:163071:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c49e7800 x1874663043051520/t0(0) o35->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:2.0:1787826390.680848:0:163071:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c49e7800 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043051520:192.168.203.147@tcp:35:createmany.0 00000100:00100000:2.0:1787826390.680944:0:163071:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d81419c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826390.685313:0:163071:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c49e7800 pname:cluuid:pid:xid:nid:opc:job createmany:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163071:1874663043051520:192.168.203.147@tcp:35:createmany.0 00000002:00100000:2.0:1787826390.685363:0:163071:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181c49e6a00 x1874663043051264/t21474836521(21474836521) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787826390.721740:0:163076:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:0.0:1787826390.722040:0:163076:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826390.722065:0:163076:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 28 lru_type: 1 00010000:00010000:0.0:1787826390.722242:0:163076:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 28 lru_type: 1 00000080:00200000:0.0:1787826390.722254:0:163076:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826390.722264:0:163076:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:0.0:1787826390.722269:0:163076:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f393 00000080:00010000:0.0:1787826390.722278:0:163076:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f393 from it ffffb97501cb3bf8 00010000:00010000:0.0:1787826390.722282:0:163076:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 28 lru_type: 1 00010000:00010000:0.0:1787826390.722292:0:163076:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 29 lru_type: 1 00000080:00200000:0.0:1787826390.722310:0:163076:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826390.722318:0:163076:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80b.replay-single, flags=27 00000080:00200000:0.0:1787826390.722324:0:163076:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x4:0x0](ffff9181d0809148), inode mode 41ed mask 1 00000080:00200000:0.0:1787826390.722328:0:163076:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=remote_dir, flags=27 00000080:00200000:0.0:1787826390.722331:0:163076:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 1 00000080:00200000:0.0:1787826390.722336:0:163076:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f-1, flags=267 00000080:00200000:0.0:1787826390.722388:0:163076:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:0.0:1787826390.722396:0:163076:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826390.722404:0:163076:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 29 lru_type: 1 00010000:00010000:0.0:1787826390.722422:0:163076:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 29 lru_type: 1 00000080:00200000:0.0:1787826390.722429:0:163076:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826390.722433:0:163076:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:0.0:1787826390.722435:0:163076:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f393 00000080:00010000:0.0:1787826390.722439:0:163076:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f393 from it ffffb97501cb3bf8 00010000:00010000:0.0:1787826390.722442:0:163076:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 29 lru_type: 1 00010000:00010000:0.0:1787826390.722449:0:163076:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 30 lru_type: 1 00000080:00200000:0.0:1787826390.722457:0:163076:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826390.722468:0:163076:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80b.replay-single, flags=27 00000080:00200000:0.0:1787826390.722472:0:163076:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x4:0x0](ffff9181d0809148), inode mode 41ed mask 1 00000080:00200000:0.0:1787826390.722474:0:163076:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=remote_dir, flags=267 00000080:00200000:0.0:1787826390.722477:0:163076:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 44 00000080:00200008:0.0:1787826390.722487:0:163076:0:(file.c:1090:ll_file_open()) START file remote_dir:[0x240001f59:0x16:0x0](ffff9181d080f448), flags 304000 00000002:00010000:0.0:1787826390.722503:0:163076:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001f59:0x16:0x0]) in obj [0x200027cd1:0x4:0x0], intent: open flags 0100240304001 00010000:00010000:0.0:1787826390.722984:0:163076:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f434 lrc: 3/1,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 163076 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.722994:0:163076:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f434 lrc: 3/1,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163076 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:0.0:1787826390.723010:0:163076:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:0.0:1787826390.723025:0:163076:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c55a5f80 x1874663043051776/t0(0) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/131856 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:0.0:1787826390.723035:0:163076:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f434 lrc: 3/1,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163076 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826390.723135:0:163076:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163076:1874663043051776:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787826390.723425:0:163076:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d01a5780 going to sleep for 11 seconds 00000100:00100000:0.0:1787826390.728451:0:163076:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163076:1874663043051776:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787826390.728491:0:163076:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f434 lrc: 4/1,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163076 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.728508:0:163076:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f434 lrc: 4/1,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 163076 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.733612:0:163076:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f434 lrc: 4/1,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 163076 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.733624:0:163076:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f434 lrc: 3/0,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 163076 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.733635:0:163076:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f434 lrc: 4/0,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 163076 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.733641:0:163076:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0513200 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:0.0:1787826390.733646:0:163076:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f434 lrc: 4/0,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x1/0x1 rrc: 2 type: IBT flags: 0x206400000000 pid: 163076 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.733659:0:163076:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f434 lrc: 4/0,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x1/0x1 rrc: 2 type: IBT flags: 0x206400000000 pid: 163076 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.733670:0:163076:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f434 lrc: 5/0,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x1/0x1 rrc: 2 type: IBT flags: 0x20e400000000 pid: 163076 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.733678:0:163076:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787826390.733686:0:163076:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f434 lrc: 3/0,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x1/0x1 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 163076 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.733693:0:163076:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f434 lrc: 0/0,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x1/0x1 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 163076 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826390.733737:0:163076:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x1 disposition=0x2b, status=0 req@ffff9181c55a5f80 x1874663043051776/t21474836523(21474836523) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00100000:0.0:1787826390.733787:0:163076:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c55a5f80 x1874663043051776/t21474836523(21474836523) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787826390.733805:0:163076:0:(namei.c:952:ll_intent_lock()) intent lock 1 on i1 [0x200027cd1:0x4:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1787826390.733844:0:163076:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x240001f59:0x16:0x0], updating i_size 4096 00000002:00100000:0.0:1787826390.733868:0:163076:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c55a5f80 x1874663043051776/t21474836523(21474836523) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1787826390.733992:0:163076:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:0.0:1787826390.734011:0:163076:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826390.734027:0:163076:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 30 lru_type: 1 00010000:00010000:0.0:1787826390.734041:0:163076:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 30 lru_type: 1 00000080:00200000:0.0:1787826390.734049:0:163076:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826390.734054:0:163076:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:0.0:1787826390.734058:0:163076:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f393 00000080:00010000:0.0:1787826390.734063:0:163076:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f393 from it ffffb97501cb3bf8 00010000:00010000:0.0:1787826390.734067:0:163076:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 30 lru_type: 1 00010000:00010000:0.0:1787826390.734077:0:163076:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 31 lru_type: 1 00000080:00200000:0.0:1787826390.734092:0:163076:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826390.734099:0:163076:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80b.replay-single, flags=25 00000080:00200000:0.0:1787826390.734104:0:163076:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x4:0x0](ffff9181d0809148), inode mode 41ed mask 1 00000080:00200000:0.0:1787826390.734107:0:163076:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=remote_dir, flags=25 00000080:00200000:0.0:1787826390.734110:0:163076:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 1 00000080:00200000:0.0:1787826390.734115:0:163076:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f-1, flags=265 00000080:00200000:0.0:1787826390.734120:0:163076:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x18:0x0](ffff9181c83e21c8), inode mode 81a4 mask 44 00000080:00200008:0.0:1787826390.734127:0:163076:0:(file.c:1090:ll_file_open()) START file f-1:[0x240001f59:0x18:0x0](ffff9181c83e21c8), flags 100400 00000002:00010000:0.0:1787826390.734142:0:163076:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001f59:0x18:0x0]) in obj [0x240001f59:0x16:0x0], intent: open flags 0240100401 00010000:00010000:0.0:1787826390.734197:0:163076:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0513a00/0xbef42c0221d6f43b lrc: 3/1,0 mode: --/CR res: [0x240001f59:0x16:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 163076 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.734205:0:163076:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0513a00/0xbef42c0221d6f43b lrc: 3/1,0 mode: --/CR res: [0x240001f59:0x16:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163076 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:0.0:1787826390.734218:0:163076:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:0.0:1787826390.734224:0:163076:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c55a4380 x1874663043052032/t0(0) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/131856 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:0.0:1787826390.734234:0:163076:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0513a00/0xbef42c0221d6f43b lrc: 3/1,0 mode: --/CR res: [0x240001f59:0x16:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163076 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826390.734258:0:163076:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163076:1874663043052032:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787826390.734482:0:163076:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d01a5780 going to sleep for 11 seconds 00000100:00100000:0.0:1787826390.743410:0:163076:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163076:1874663043052032:192.168.203.147@tcp:101:lfs.0 00010000:00010000:0.0:1787826390.743446:0:163076:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0513a00/0xbef42c0221d6f43b lrc: 4/1,0 mode: --/CR res: [0x240001f59:0x16:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceec64 expref: -99 pid: 163076 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.743487:0:163076:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0513a00/0xbef42c0221d6f43b lrc: 4/1,0 mode: --/PR res: [0x240001f59:0x18:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceec64 expref: -99 pid: 163076 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.743497:0:163076:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0513a00/0xbef42c0221d6f43b lrc: 4/1,0 mode: --/PR res: [0x240001f59:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceec64 expref: -99 pid: 163076 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.743621:0:163076:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0513a00/0xbef42c0221d6f43b lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceec64 expref: -99 pid: 163076 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.743637:0:163076:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0513a00/0xbef42c0221d6f43b lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceec64 expref: -99 pid: 163076 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.743656:0:163076:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0513a00/0xbef42c0221d6f43b lrc: 5/2,0 mode: PR/PR res: [0x240001f59:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceec64 expref: -99 pid: 163076 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.743664:0:163076:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0513a00/0xbef42c0221d6f43b lrc: 5/2,0 mode: PR/PR res: [0x240001f59:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceec64 expref: -99 pid: 163076 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.743987:0:163076:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0513a00/0xbef42c0221d6f43b lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceec64 expref: -99 pid: 163076 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826390.744004:0:163076:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x1 disposition=0x2b, status=0 req@ffff9181c55a4380 x1874663043052032/t21474836524(21474836524) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00100000:0.0:1787826390.744019:0:163076:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c55a4380 x1874663043052032/t21474836524(21474836524) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787826390.744034:0:163076:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open, lvb_len: 56 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0513a00/0xbef42c0221d6f43b lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceec64 expref: -99 pid: 163076 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:0.0:1787826390.744050:0:163076:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0513a00/0xbef42c0221d6f43b lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceec64 expref: -99 pid: 163076 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.744065:0:163076:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b5200/0xbef42c0221d6f3af lrc: 2/1,0 mode: PR/PR res: [0x240001f59:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00cee9cb expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826390.744080:0:163076:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b5200/0xbef42c0221d6f3af lrc: 2/1,0 mode: PR/PR res: [0x240001f59:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00cee9cb expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826390.744090:0:163076:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0513a00/0xbef42c0221d6f43b lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceec64 expref: -99 pid: 163076 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.744096:0:163076:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0513a00/0xbef42c0221d6f43b lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000400000000 nid: local remote: 0x878a5eea00ceec64 expref: -99 pid: 163076 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.744103:0:163076:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0513a00/0xbef42c0221d6f43b lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000400000000 nid: local remote: 0x878a5eea00ceec64 expref: -99 pid: 163076 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826390.744137:0:163076:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000080:00200000:0.0:1787826390.744145:0:163076:0:(namei.c:952:ll_intent_lock()) intent lock 1 on i1 [0x240001f59:0x16:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1787826390.744213:0:163076:0:(file.c:6843:ll_layout_conf()) [0x240001f59:0x18:0x0]: layout version change: 0 -> 0 00000080:00010000:0.0:1787826390.744219:0:163076:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001f59:0x18:0x0](ffff9181c83e21c8) for lock 0xbef42c0221d6f3af 00000080:00010000:0.0:1787826390.744227:0:163076:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f3af from it ffffb97501cb3bf0 00010000:00010000:0.0:1787826390.744230:0:163076:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b5200/0xbef42c0221d6f3af lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00cee9cb expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826390.744240:0:163076:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b5200/0xbef42c0221d6f3af lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00cee9cb expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000002:00100000:0.0:1787826390.744271:0:163076:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c55a4380 x1874663043052032/t21474836524(21474836524) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200080:0.0:1787826390.744380:0:163076:0:(file.c:4646:ll_file_ioctl()) VFS Op:inode=[0x240001f59:0x18:0x0](ffff9181c83e21c8) cmd=800466af arg=7ffdf019590c 00000080:00200008:0.0:1787826390.744411:0:163076:0:(file.c:479:ll_file_release()) START file f-1:[0x240001f59:0x18:0x0](ffff9181c83e21c8), flags 100000 00000002:00100000:0.0:1787826390.744471:0:163076:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181c55a4380 x1874663043052032/t21474836524(21474836524) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:0 00000020:00100000:0.0:1787826390.744482:0:163076:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:0.0:1787826390.744487:0:163076:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c55a4700 x1874663043052288/t0(0) o35->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:0.0:1787826390.744510:0:163076:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163076:1874663043052288:192.168.203.147@tcp:35:lfs.0 00000100:00100000:0.0:1787826390.744685:0:163076:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d01a50c0 going to sleep for 11 seconds 00010000:00010000:2.0:1787826390.745333:0:159732:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826390.745344:0:159732:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0513a00/0xbef42c0221d6f43b lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x400000000 nid: local remote: 0x878a5eea00ceec64 expref: -99 pid: 163076 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.745353:0:159732:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0513a00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:2.0:1787826390.745358:0:159732:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0513a00/0xbef42c0221d6f43b lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x18:0x0].0x0 bits 0x1b/0x1b rrc: 3 type: IBT gid 0 flags: 0x400000000 nid: local remote: 0x878a5eea00ceec64 expref: -99 pid: 163076 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.745367:0:159732:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0513a00/0xbef42c0221d6f43b lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x18:0x0].0x0 bits 0x1b/0x1b rrc: 3 type: IBT gid 0 flags: 0x400000000 nid: local remote: 0x878a5eea00ceec64 expref: -99 pid: 163076 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.745376:0:159732:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0513a00/0xbef42c0221d6f43b lrc: 3/0,0 mode: PR/PR res: [0x240001f59:0x18:0x0].0x0 bits 0x1b/0x1b rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ceec64 expref: -99 pid: 163076 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.745397:0:159732:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b5200/0xbef42c0221d6f3af lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00cee9cb expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1787826390.745438:0:159732:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0513a00/0xbef42c0221d6f43b lrc: 2/0,0 mode: --/PR res: [0x240001f59:0x18:0x0].0x0 bits 0x1b/0x1b rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ceec64 expref: -99 pid: 163076 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.745447:0:159732:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826390.745482:0:159732:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0513a00/0xbef42c0221d6f43b lrc: 1/0,0 mode: --/PR res: [0x240001f59:0x18:0x0].0x0 bits 0x1b/0x1b rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ceec64 expref: -99 pid: 163076 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.745489:0:159732:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0513a00/0xbef42c0221d6f43b lrc: 0/0,0 mode: --/PR res: [0x240001f59:0x18:0x0].0x0 bits 0x1b/0x1b rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ceec64 expref: -99 pid: 163076 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.745507:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826390.745509:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826390.745521:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826390.745542:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c49e6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663043052416:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:2.0:1787826390.747933:0:163076:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163076:1874663043052288:192.168.203.147@tcp:35:lfs.0 00000002:00100000:2.0:1787826390.747974:0:163076:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181c55a4380 x1874663043052032/t21474836524(21474836524) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/664 e 0 to 0 dl 1787826406 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200008:2.0:1787826390.748303:0:163076:0:(file.c:479:ll_file_release()) START file remote_dir:[0x240001f59:0x16:0x0](ffff9181d080f448), flags 304000 00000002:00100000:2.0:1787826390.748370:0:163076:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181c55a5f80 x1874663043051776/t21474836523(21474836523) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 0 dl 1787826406 ref 1 fl Complete:RPQU/604/ffffffff rc 301/-1 job:'lfs.0' uid:0 gid:0 projid:0 00000020:00100000:2.0:1787826390.748382:0:163076:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1787826390.748388:0:163076:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c49e4000 x1874663043052672/t0(0) o35->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:2.0:1787826390.748411:0:163076:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c49e4000 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163076:1874663043052672:192.168.203.147@tcp:35:lfs.0 00000100:00100000:2.0:1787826390.748544:0:163076:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d81419c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826390.751106:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c49e6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663043052416:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:2.0:1787826390.752208:0:163076:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c49e4000 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163076:1874663043052672:192.168.203.147@tcp:35:lfs.0 00000002:00100000:2.0:1787826390.752251:0:163076:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181c55a5f80 x1874663043051776/t21474836523(21474836523) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 0 dl 1787826406 ref 1 fl Complete:RQU/604/ffffffff rc 301/-1 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1787826390.764128:0:163077:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:1.0:1787826390.764168:0:163077:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826390.764194:0:163077:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 31 lru_type: 1 00010000:00010000:1.0:1787826390.764214:0:163077:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 31 lru_type: 1 00000080:00200000:1.0:1787826390.764222:0:163077:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826390.764231:0:163077:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:1.0:1787826390.764236:0:163077:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f393 00000080:00010000:1.0:1787826390.764246:0:163077:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f393 from it ffffb97502bafb58 00010000:00010000:1.0:1787826390.764251:0:163077:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 31 lru_type: 1 00010000:00010000:1.0:1787826390.764261:0:163077:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 32 lru_type: 1 00000080:00200000:1.0:1787826390.764281:0:163077:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826390.764290:0:163077:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80b.replay-single, flags=8 00000080:00200008:1.0:1787826390.764300:0:163077:0:(file.c:6296:ll_getattr_dentry()) START file d80b.replay-single:[0x200027cd1:0x4:0x0](ffff9181d0809148), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1787826390.764306:0:163077:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200027cd1:0x4:0x0](ffff9181d0809148),name=d80b.replay-single 00000002:00010000:1.0:1787826390.764318:0:163077:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200027cd1:0x4:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826390.764325:0:163077:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d1a77c00 type 13 mode 30 res 8590097617/4 (0 0) 00010000:00010000:1.0:1787826390.764381:0:163077:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5600/0xbef42c0221d6f442 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 163077 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.764390:0:163077:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5600/0xbef42c0221d6f442 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 163077 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826390.764403:0:163077:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5600/0xbef42c0221d6f442 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 163077 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826390.764438:0:163077:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043052800:192.168.203.147@tcp:101:rm.0 00000100:00100000:1.0:1787826390.764772:0:163077:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181e0156f00 going to sleep for 11 seconds 00000100:00100000:3.0:1787826390.769185:0:163077:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043052800:192.168.203.147@tcp:101:rm.0 00010000:00010000:3.0:1787826390.769221:0:163077:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5600/0xbef42c0221d6f442 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceec79 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.769245:0:163077:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5600/0xbef42c0221d6f442 lrc: 4/1,0 mode: --/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceec79 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.769255:0:163077:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5600/0xbef42c0221d6f442 lrc: 4/1,0 mode: --/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceec79 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.769269:0:163077:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5600/0xbef42c0221d6f442 lrc: 4/1,0 mode: PR/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceec79 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.769282:0:163077:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5600/0xbef42c0221d6f442 lrc: 4/1,0 mode: PR/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceec79 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.769298:0:163077:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5600/0xbef42c0221d6f442 lrc: 5/2,0 mode: PR/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceec79 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.769308:0:163077:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5600/0xbef42c0221d6f442 lrc: 5/2,0 mode: PR/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceec79 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.769318:0:163077:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5600/0xbef42c0221d6f442 lrc: 4/1,0 mode: PR/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceec79 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826390.769332:0:163077:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c35b0700 x1874663043052800/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826406 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826390.769349:0:163077:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5600/0xbef42c0221d6f442 lrc: 3/1,0 mode: PR/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceec79 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.769360:0:163077:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d1a77c00 type 13 mode 4 res 8590097617/4 (0 0) 00000080:00200000:3.0:1787826390.769369:0:163077:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:3.0:1787826390.769405:0:163077:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200027cd1:0x4:0x0], updating i_size 4096 00000080:00010000:3.0:1787826390.769412:0:163077:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200027cd1:0x4:0x0](ffff9181d0809148) for lock 0xbef42c0221d6f442 00000080:00010000:3.0:1787826390.769422:0:163077:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200027cd1:0x4:0x0](ffff9181d0809148) 00000080:00010000:3.0:1787826390.769426:0:163077:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f442 from it ffffb97502bafcb8 00010000:00010000:3.0:1787826390.769429:0:163077:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5600/0xbef42c0221d6f442 lrc: 3/1,0 mode: PR/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceec79 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.769440:0:163077:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5600/0xbef42c0221d6f442 lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceec79 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826390.769572:0:163077:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:3.0:1787826390.769596:0:163077:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826390.769608:0:163077:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 32 lru_type: 1 00010000:00010000:3.0:1787826390.769620:0:163077:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 32 lru_type: 1 00000080:00200000:3.0:1787826390.769628:0:163077:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826390.769633:0:163077:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:3.0:1787826390.769636:0:163077:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f393 00000080:00010000:3.0:1787826390.769640:0:163077:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f393 from it ffffb97502bafbf8 00010000:00010000:3.0:1787826390.769643:0:163077:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 32 lru_type: 1 00010000:00010000:3.0:1787826390.769651:0:163077:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 33 lru_type: 1 00000080:00200000:3.0:1787826390.769665:0:163077:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826390.769673:0:163077:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80b.replay-single, flags=266 00000080:00200000:3.0:1787826390.769689:0:163077:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x4:0x0](ffff9181d0809148), inode mode 41ed mask 44 00000080:00200008:3.0:1787826390.769707:0:163077:0:(file.c:1090:ll_file_open()) START file d80b.replay-single:[0x200027cd1:0x4:0x0](ffff9181d0809148), flags 704400 00000002:00010000:3.0:1787826390.769725:0:163077:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200027cd1:0x4:0x0]) in obj [0x200000007:0x1:0x0], intent: open flags 0100240704401 00010000:00010000:3.0:1787826390.769814:0:163077:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480a00/0xbef42c0221d6f449 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 163077 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.769822:0:163077:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480a00/0xbef42c0221d6f449 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 163077 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:3.0:1787826390.769845:0:163077:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:3.0:1787826390.769853:0:163077:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c5635500 x1874663043053056/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/131856 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:3.0:1787826390.769862:0:163077:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480a00/0xbef42c0221d6f449 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 163077 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826390.769887:0:163077:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043053056:192.168.203.147@tcp:101:rm.0 00000100:00100000:3.0:1787826390.770114:0:163077:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d22a2000 going to sleep for 11 seconds 00000100:00100000:3.0:1787826390.775530:0:163077:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043053056:192.168.203.147@tcp:101:rm.0 00010000:00010000:3.0:1787826390.775568:0:163077:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480a00/0xbef42c0221d6f449 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 163077 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.775585:0:163077:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480a00/0xbef42c0221d6f449 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 163077 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.775594:0:163077:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480a00/0xbef42c0221d6f449 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 163077 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.775604:0:163077:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480a00/0xbef42c0221d6f449 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 163077 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.775614:0:163077:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480a00/0xbef42c0221d6f449 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 163077 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.775620:0:163077:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3480a00 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:3.0:1787826390.775625:0:163077:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480a00/0xbef42c0221d6f449 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x206400000000 pid: 163077 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.775633:0:163077:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480a00/0xbef42c0221d6f449 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x206400000000 pid: 163077 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.775640:0:163077:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480a00/0xbef42c0221d6f449 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x20e400000000 pid: 163077 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.775649:0:163077:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826390.775657:0:163077:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480a00/0xbef42c0221d6f449 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 163077 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826390.775663:0:163077:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480a00/0xbef42c0221d6f449 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 163077 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826390.775695:0:163077:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x1 disposition=0x2b, status=0 req@ffff9181c5635500 x1874663043053056/t352187318295(352187318295) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'rm.0' uid:0 gid:0 projid:0 00000002:00100000:3.0:1787826390.775711:0:163077:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c5635500 x1874663043053056/t352187318295(352187318295) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'rm.0' uid:0 gid:0 projid:0 00000080:00200000:3.0:1787826390.775729:0:163077:0:(namei.c:952:ll_intent_lock()) intent lock 1 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:3.0:1787826390.776005:0:163077:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200027cd1:0x4:0x0], updating i_size 4096 00000002:00100000:3.0:1787826390.776034:0:163077:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c5635500 x1874663043053056/t352187318295(352187318295) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'rm.0' uid:0 gid:0 projid:0 00000080:00200008:3.0:1787826390.776187:0:163077:0:(file.c:6296:ll_getattr_dentry()) START file d80b.replay-single:[0x200027cd1:0x4:0x0](ffff9181d0809148), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1787826390.776196:0:163077:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200027cd1:0x4:0x0](ffff9181d0809148),name=d80b.replay-single 00000002:00010000:3.0:1787826390.776223:0:163077:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200027cd1:0x4:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826390.776244:0:163077:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5600/0xbef42c0221d6f442 lrc: 2/1,0 mode: PR/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceec79 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826390.776258:0:163077:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5600/0xbef42c0221d6f442 lrc: 2/1,0 mode: PR/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceec79 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826390.776267:0:163077:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826390.776274:0:163077:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200027cd1:0x4:0x0](ffff9181d0809148) 00000080:00010000:3.0:1787826390.776278:0:163077:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200027cd1:0x4:0x0](ffff9181d0809148) for lock 0xbef42c0221d6f442 00000080:00010000:3.0:1787826390.776285:0:163077:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f442 from it ffffb97502bafcd8 00010000:00010000:3.0:1787826390.776288:0:163077:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5600/0xbef42c0221d6f442 lrc: 3/1,0 mode: PR/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceec79 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826390.776297:0:163077:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5600/0xbef42c0221d6f442 lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceec79 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:3.0:1787826390.776447:0:163077:0:(dir.c:332:ll_iterate()) VFS Op:inode=[0x200027cd1:0x4:0x0](ffff9181d0809148) pos/size0/4096 32bit_api 0 00000002:00010000:3.0:1787826390.776475:0:163077:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200027cd1:0x4:0x0]) in obj [0x200027cd1:0x4:0x0], intent: readdir flags 0 00010000:00010000:3.0:1787826390.776484:0:163077:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5600/0xbef42c0221d6f442 lrc: 2/1,0 mode: PR/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceec79 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826390.776494:0:163077:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5600/0xbef42c0221d6f442 lrc: 2/1,0 mode: PR/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceec79 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000100:00100000:3.0:1787826390.778260:0:163077:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043053184:192.168.203.147@tcp:37:rm.0 00000100:00100000:3.0:1787826390.778396:0:163077:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d22a2000 going to sleep for 11 seconds 00000100:00100000:3.0:1787826390.795372:0:163077:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043053184:192.168.203.147@tcp:37:rm.0 00010000:00010000:3.0:1787826390.796218:0:163077:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5600/0xbef42c0221d6f442 lrc: 3/1,0 mode: PR/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceec79 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826390.796238:0:163077:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5600/0xbef42c0221d6f442 lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceec79 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200008:3.0:1787826390.796411:0:163077:0:(file.c:479:ll_file_release()) START file d80b.replay-single:[0x200027cd1:0x4:0x0](ffff9181d0809148), flags 704000 00000002:00100000:3.0:1787826390.796519:0:163077:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181c5635500 x1874663043053056/t352187318295(352187318295) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 0 dl 1787826406 ref 1 fl Complete:RPQU/604/ffffffff rc 301/-1 job:'rm.0' uid:0 gid:0 projid:0 00000020:00100000:3.0:1787826390.796533:0:163077:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:3.0:1787826390.796540:0:163077:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c5637100 x1874663043053440/t0(0) o35->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:3.0:1787826390.796573:0:163077:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043053440:192.168.203.147@tcp:35:rm.0 00000100:00100000:3.0:1787826390.796678:0:163077:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d22a2000 going to sleep for 11 seconds 00000100:00100000:3.0:1787826390.800722:0:163077:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043053440:192.168.203.147@tcp:35:rm.0 00000002:00100000:3.0:1787826390.800815:0:163077:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181c5635500 x1874663043053056/t352187318295(352187318295) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 0 dl 1787826406 ref 1 fl Complete:RQU/604/ffffffff rc 301/-1 job:'rm.0' uid:0 gid:0 projid:0 00000080:00200000:3.0:1787826390.800932:0:163077:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:3.0:1787826390.800956:0:163077:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826390.800980:0:163077:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 33 lru_type: 1 00010000:00010000:3.0:1787826390.800998:0:163077:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 33 lru_type: 1 00000080:00200000:3.0:1787826390.801007:0:163077:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826390.801014:0:163077:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:3.0:1787826390.801018:0:163077:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f393 00000080:00010000:3.0:1787826390.801028:0:163077:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f393 from it ffffb97502bafbf8 00010000:00010000:3.0:1787826390.801034:0:163077:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 33 lru_type: 1 00010000:00010000:3.0:1787826390.801044:0:163077:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 34 lru_type: 1 00000080:00200000:3.0:1787826390.801062:0:163077:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826390.801070:0:163077:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80b.replay-single, flags=266 00000080:00200000:3.0:1787826390.801968:0:163077:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x4:0x0](ffff9181d0809148), inode mode 41ed mask 44 00000080:00200008:3.0:1787826390.801993:0:163077:0:(file.c:1090:ll_file_open()) START file d80b.replay-single:[0x200027cd1:0x4:0x0](ffff9181d0809148), flags 704400 00000002:00010000:2.0:1787826390.802284:0:163077:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200027cd1:0x4:0x0]) in obj [0x200000007:0x1:0x0], intent: open flags 0104240704401 00010000:00010000:2.0:1787826390.802367:0:163077:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843200/0xbef42c0221d6f450 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 163077 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.802376:0:163077:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843200/0xbef42c0221d6f450 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 163077 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1787826390.802388:0:163077:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1787826390.802400:0:163077:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c49e4000 x1874663043053696/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/131856 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1787826390.802409:0:163077:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843200/0xbef42c0221d6f450 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 163077 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826390.802432:0:163077:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c49e4000 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043053696:192.168.203.147@tcp:101:rm.0 00000100:00100000:2.0:1787826390.803207:0:163077:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d81419c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826390.807996:0:163077:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c49e4000 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043053696:192.168.203.147@tcp:101:rm.0 00010000:00010000:2.0:1787826390.809271:0:163077:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843200/0xbef42c0221d6f450 lrc: 4/1,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceec95 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.809290:0:163077:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843200/0xbef42c0221d6f450 lrc: 4/1,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceec95 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.809308:0:163077:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843200/0xbef42c0221d6f450 lrc: 4/1,0 mode: CR/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceec95 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.809322:0:163077:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843200/0xbef42c0221d6f450 lrc: 4/1,0 mode: CR/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceec95 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787826390.809353:0:163077:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x1 disposition=0x200002b, status=0 req@ffff9181c49e4000 x1874663043053696/t352187318297(352187318297) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00100000:2.0:1787826390.809369:0:163077:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c49e4000 x1874663043053696/t352187318297(352187318297) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787826390.809388:0:163077:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843200/0xbef42c0221d6f450 lrc: 3/1,0 mode: CR/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceec95 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.809408:0:163077:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d1a77c00 type 13 mode 16 res 8590097617/4 (0 0) 00000080:00200000:2.0:1787826390.809417:0:163077:0:(namei.c:952:ll_intent_lock()) intent lock 1 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:2.0:1787826390.809476:0:163077:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200027cd1:0x4:0x0], updating i_size 4096 00000080:00010000:2.0:1787826390.809485:0:163077:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200027cd1:0x4:0x0](ffff9181d0809148) for lock 0xbef42c0221d6f450 00000080:00010000:2.0:1787826390.809495:0:163077:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f450 from it ffffb97502bafbd0 00010000:00010000:2.0:1787826390.809500:0:163077:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843200/0xbef42c0221d6f450 lrc: 3/1,0 mode: CR/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceec95 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.809521:0:163077:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843200/0xbef42c0221d6f450 lrc: 2/0,0 mode: CR/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceec95 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000002:00100000:2.0:1787826390.809572:0:163077:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c49e4000 x1874663043053696/t352187318297(352187318297) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000080:00200008:2.0:1787826390.809666:0:163077:0:(file.c:6296:ll_getattr_dentry()) START file d80b.replay-single:[0x200027cd1:0x4:0x0](ffff9181d0809148), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1787826390.809687:0:163077:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200027cd1:0x4:0x0](ffff9181d0809148),name=d80b.replay-single 00000002:00010000:2.0:1787826390.809729:0:163077:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200027cd1:0x4:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826390.809762:0:163077:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5600/0xbef42c0221d6f442 lrc: 2/1,0 mode: PR/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceec79 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1787826390.809775:0:163077:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5600/0xbef42c0221d6f442 lrc: 2/1,0 mode: PR/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceec79 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:2.0:1787826390.809781:0:163077:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826390.809787:0:163077:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200027cd1:0x4:0x0](ffff9181d0809148) 00000080:00010000:2.0:1787826390.809791:0:163077:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200027cd1:0x4:0x0](ffff9181d0809148) for lock 0xbef42c0221d6f442 00000080:00010000:2.0:1787826390.809795:0:163077:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f442 from it ffffb97502bafcd8 00010000:00010000:2.0:1787826390.809798:0:163077:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5600/0xbef42c0221d6f442 lrc: 3/1,0 mode: PR/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceec79 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1787826390.809805:0:163077:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5600/0xbef42c0221d6f442 lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceec79 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00000080:00200000:2.0:1787826390.809920:0:163077:0:(dir.c:332:ll_iterate()) VFS Op:inode=[0x200027cd1:0x4:0x0](ffff9181d0809148) pos/size0/4096 32bit_api 0 00000080:00200000:2.0:1787826390.809951:0:163077:0:(dir.c:332:ll_iterate()) VFS Op:inode=[0x200027cd1:0x4:0x0](ffff9181d0809148) pos/size18446744073709551614/4096 32bit_api 0 00000080:00200000:2.0:1787826390.809980:0:163077:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x4:0x0](ffff9181d0809148), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.809988:0:163077:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=remote_dir, flags=0 00000080:00200008:2.0:1787826390.809997:0:163077:0:(file.c:6296:ll_getattr_dentry()) START file remote_dir:[0x240001f59:0x16:0x0](ffff9181d080f448), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1787826390.811704:0:163077:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448),name=remote_dir 00000002:00010000:2.0:1787826390.811739:0:163077:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001f59:0x16:0x0]) in obj [0x200027cd1:0x4:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826390.811956:0:163077:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843800/0xbef42c0221d6f457 lrc: 3/1,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 163077 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.811969:0:163077:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843800/0xbef42c0221d6f457 lrc: 3/1,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163077 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.811983:0:163077:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843800/0xbef42c0221d6f457 lrc: 3/1,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163077 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826390.812014:0:163077:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c49e6680 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043053824:192.168.203.147@tcp:101:rm.0 00000100:00100000:2.0:1787826390.812189:0:163077:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d8141e40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826390.816492:0:163077:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c49e6680 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043053824:192.168.203.147@tcp:101:rm.0 00010000:00010000:2.0:1787826390.816540:0:163077:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843800/0xbef42c0221d6f457 lrc: 4/1,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceecaa expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.816590:0:163077:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843800/0xbef42c0221d6f457 lrc: 4/1,0 mode: --/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceecaa expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.816603:0:163077:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843800/0xbef42c0221d6f457 lrc: 4/1,0 mode: --/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceecaa expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.816616:0:163077:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843800/0xbef42c0221d6f457 lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceecaa expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.816630:0:163077:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843800/0xbef42c0221d6f457 lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceecaa expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.816646:0:163077:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843800/0xbef42c0221d6f457 lrc: 5/2,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceecaa expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.816656:0:163077:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843800/0xbef42c0221d6f457 lrc: 5/2,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceecaa expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.816663:0:163077:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843800/0xbef42c0221d6f457 lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceecaa expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787826390.816687:0:163077:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c49e6680 x1874663043053824/t0(0) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826406 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787826390.817006:0:163077:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843800/0xbef42c0221d6f457 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceecaa expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.817022:0:163077:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c77f3000 type 13 mode 4 res 9663684441/22 (0 0) 00000080:00200000:2.0:1787826390.817031:0:163077:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200027cd1:0x4:0x0] suppgids 0 0: rc 0 00000080:00200000:2.0:1787826390.817072:0:163077:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x240001f59:0x16:0x0], updating i_size 4096 00000080:00010000:2.0:1787826390.817092:0:163077:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001f59:0x16:0x0](ffff9181d080f448) for lock 0xbef42c0221d6f457 00000080:00010000:2.0:1787826390.817100:0:163077:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001f59:0x16:0x0](ffff9181d080f448) 00000080:00010000:2.0:1787826390.817104:0:163077:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f457 from it ffffb97502bafcb8 00010000:00010000:2.0:1787826390.817108:0:163077:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843800/0xbef42c0221d6f457 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceecaa expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.817117:0:163077:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843800/0xbef42c0221d6f457 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceecaa expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826390.817208:0:163077:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x4:0x0](ffff9181d0809148), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.817219:0:163077:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=remote_dir, flags=258 00000080:00200000:2.0:1787826390.817236:0:163077:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 44 00000080:00200008:2.0:1787826390.817246:0:163077:0:(file.c:1090:ll_file_open()) START file remote_dir:[0x240001f59:0x16:0x0](ffff9181d080f448), flags 704400 00000002:00010000:2.0:1787826390.817274:0:163077:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001f59:0x16:0x0]) in obj [0x200027cd1:0x4:0x0], intent: open flags 0104240704401 00010000:00010000:2.0:1787826390.817336:0:163077:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842e00/0xbef42c0221d6f45e lrc: 3/1,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 163077 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.817345:0:163077:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842e00/0xbef42c0221d6f45e lrc: 3/1,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163077 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1787826390.817357:0:163077:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1787826390.817367:0:163077:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c49e6680 x1874663043054080/t0(0) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/131856 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1787826390.817378:0:163077:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842e00/0xbef42c0221d6f45e lrc: 3/1,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163077 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826390.817403:0:163077:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c49e6680 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043054080:192.168.203.147@tcp:101:rm.0 00000100:00100000:2.0:1787826390.817664:0:163077:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d8141e40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826390.822098:0:163077:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c49e6680 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043054080:192.168.203.147@tcp:101:rm.0 00010000:00010000:2.0:1787826390.822149:0:163077:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842e00/0xbef42c0221d6f45e lrc: 4/1,0 mode: --/CR res: [0x240001f59:0x16:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceecb8 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.822162:0:163077:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842e00/0xbef42c0221d6f45e lrc: 4/1,0 mode: --/CR res: [0x240001f59:0x16:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceecb8 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.822174:0:163077:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842e00/0xbef42c0221d6f45e lrc: 4/1,0 mode: CR/CR res: [0x240001f59:0x16:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceecb8 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.822185:0:163077:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842e00/0xbef42c0221d6f45e lrc: 4/1,0 mode: CR/CR res: [0x240001f59:0x16:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceecb8 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787826390.822201:0:163077:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x1 disposition=0x200002b, status=0 req@ffff9181c49e6680 x1874663043054080/t21474836527(21474836527) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00100000:2.0:1787826390.822216:0:163077:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c49e6680 x1874663043054080/t21474836527(21474836527) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787826390.822231:0:163077:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842e00/0xbef42c0221d6f45e lrc: 3/1,0 mode: CR/CR res: [0x240001f59:0x16:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceecb8 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.822241:0:163077:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c77f3000 type 13 mode 16 res 9663684441/22 (0 0) 00000080:00200000:2.0:1787826390.822249:0:163077:0:(namei.c:952:ll_intent_lock()) intent lock 1 on i1 [0x200027cd1:0x4:0x0] suppgids 0 0: rc 0 00000080:00200000:2.0:1787826390.822283:0:163077:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x240001f59:0x16:0x0], updating i_size 4096 00000080:00010000:2.0:1787826390.822289:0:163077:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001f59:0x16:0x0](ffff9181d080f448) for lock 0xbef42c0221d6f45e 00000080:00010000:2.0:1787826390.822296:0:163077:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f45e from it ffffb97502bafbd0 00010000:00010000:2.0:1787826390.822301:0:163077:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842e00/0xbef42c0221d6f45e lrc: 3/1,0 mode: CR/CR res: [0x240001f59:0x16:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceecb8 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.822317:0:163077:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842e00/0xbef42c0221d6f45e lrc: 2/0,0 mode: CR/CR res: [0x240001f59:0x16:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceecb8 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000002:00100000:2.0:1787826390.822346:0:163077:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c49e6680 x1874663043054080/t21474836527(21474836527) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 0 dl 1787826406 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000080:00200008:2.0:1787826390.822422:0:163077:0:(file.c:6296:ll_getattr_dentry()) START file remote_dir:[0x240001f59:0x16:0x0](ffff9181d080f448), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1787826390.822430:0:163077:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448),name=remote_dir 00000002:00010000:2.0:1787826390.822453:0:163077:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001f59:0x16:0x0]) in obj [0x200027cd1:0x4:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826390.822474:0:163077:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843800/0xbef42c0221d6f457 lrc: 2/1,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceecaa expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826390.822487:0:163077:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843800/0xbef42c0221d6f457 lrc: 2/1,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceecaa expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826390.822494:0:163077:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200027cd1:0x4:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826390.822499:0:163077:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001f59:0x16:0x0](ffff9181d080f448) 00000080:00010000:2.0:1787826390.822503:0:163077:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001f59:0x16:0x0](ffff9181d080f448) for lock 0xbef42c0221d6f457 00000080:00010000:2.0:1787826390.822507:0:163077:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f457 from it ffffb97502bafcd8 00010000:00010000:2.0:1787826390.822510:0:163077:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843800/0xbef42c0221d6f457 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceecaa expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826390.822518:0:163077:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843800/0xbef42c0221d6f457 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceecaa expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1787826390.822547:0:163077:0:(dir.c:332:ll_iterate()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448) pos/size0/4096 32bit_api 0 00000002:00010000:2.0:1787826390.822560:0:163077:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001f59:0x16:0x0]) in obj [0x240001f59:0x16:0x0], intent: readdir flags 0 00010000:00010000:2.0:1787826390.822565:0:163077:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843800/0xbef42c0221d6f457 lrc: 2/1,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceecaa expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787826390.822574:0:163077:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843800/0xbef42c0221d6f457 lrc: 2/1,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceecaa expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000100:00100000:2.0:1787826390.823658:0:163077:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c49e6300 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043054208:192.168.203.147@tcp:37:rm.0 00000100:00100000:2.0:1787826390.824408:0:163077:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d8141e40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826390.835161:0:163077:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c49e6300 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043054208:192.168.203.147@tcp:37:rm.0 00010000:00010000:2.0:1787826390.836386:0:163077:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843800/0xbef42c0221d6f457 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceecaa expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787826390.836411:0:163077:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843800/0xbef42c0221d6f457 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceecaa expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200008:2.0:1787826390.836505:0:163077:0:(file.c:479:ll_file_release()) START file remote_dir:[0x240001f59:0x16:0x0](ffff9181d080f448), flags 704000 00010000:00010000:2.0:1787826390.836668:0:163077:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842e00/0xbef42c0221d6f45e lrc: 2/0,0 mode: CR/CR res: [0x240001f59:0x16:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceecb8 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1787826390.841044:0:163077:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x4:0x0](ffff9181d0809148), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.841058:0:163077:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=remote_dir, flags=258 00000080:00200000:2.0:1787826390.841069:0:163077:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 44 00000080:00200008:2.0:1787826390.841079:0:163077:0:(file.c:1090:ll_file_open()) START file remote_dir:[0x240001f59:0x16:0x0](ffff9181d080f448), flags 704400 00000080:00200008:2.0:1787826390.841263:0:163077:0:(file.c:6296:ll_getattr_dentry()) START file remote_dir:[0x240001f59:0x16:0x0](ffff9181d080f448), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1787826390.841271:0:163077:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448),name=remote_dir 00000002:00010000:2.0:1787826390.841305:0:163077:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x240001f59:0x16:0x0]) in obj [0x200027cd1:0x4:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826390.841328:0:163077:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843800/0xbef42c0221d6f457 lrc: 2/1,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceecaa expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1787826390.841345:0:163077:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843800/0xbef42c0221d6f457 lrc: 2/1,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceecaa expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:2.0:1787826390.841354:0:163077:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200027cd1:0x4:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826390.841363:0:163077:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001f59:0x16:0x0](ffff9181d080f448) 00000080:00010000:2.0:1787826390.841367:0:163077:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001f59:0x16:0x0](ffff9181d080f448) for lock 0xbef42c0221d6f457 00000080:00010000:2.0:1787826390.841377:0:163077:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f457 from it ffffb97502bafcd8 00010000:00010000:2.0:1787826390.841381:0:163077:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843800/0xbef42c0221d6f457 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceecaa expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1787826390.841392:0:163077:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843800/0xbef42c0221d6f457 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceecaa expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00000080:00200000:2.0:1787826390.841435:0:163077:0:(llite_lib.c:2903:ll_statfs()) VFS Op:sb=lustre (ffff9181c5620000) remote_dir proj=0 00000080:00000004:2.0:1787826390.841442:0:163077:0:(obd_class.h:1090:obd_statfs()) lustre-clilmv-ffff9181c5620000: age 8391, max_age 8461 00800000:00000004:2.0:1787826390.841456:0:163077:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9181c5620000: age 8461, max_age 8461 00800000:00000004:2.0:1787826390.841471:0:163077:0:(obd_class.h:1130:obd_statfs()) lustre-MDT0000-mdc-ffff9181c5620000: use ffff9181c354d178 cache blocks 317223/353529 objects 1023644/1024000 00800000:00000004:2.0:1787826390.841475:0:163077:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9181c5620000: age 8461, max_age 8461 00800000:00000004:2.0:1787826390.841478:0:163077:0:(obd_class.h:1130:obd_statfs()) lustre-MDT0001-mdc-ffff9181c5620000: use ffff9181c3548450 cache blocks 317326/353529 objects 1023702/1024000 00000080:00000004:2.0:1787826390.841484:0:163077:0:(obd_class.h:1116:obd_statfs()) lustre-clilmv-ffff9181c5620000: update ffff9181d1a19e08 cache blocks 634549/707058 objects 2047346/2048000 00000080:00000004:2.0:1787826390.841488:0:163077:0:(llite_lib.c:2755:ll_statfs_internal()) MDC blocks 634549/707058 objects 2047346/2048000 00000080:00000004:2.0:1787826390.841492:0:163077:0:(obd_class.h:1090:obd_statfs()) lustre-clilov-ffff9181c5620000: age 8391, max_age 8461 00020000:00000004:2.0:1787826390.841510:0:163077:0:(obd_class.h:1056:obd_statfs_async()) lustre-OST0000-osc-ffff9181c5620000: age 8461, max_age 8461 00000008:00000004:2.0:1787826390.841525:0:163077:0:(osc_request.c:3417:osc_statfs_async()) lustre-OST0000-osc-ffff9181c5620000: use ffff9181c3549e08 cache blocks 901356/958279 objects 261710/262144 00020000:00000004:2.0:1787826390.841531:0:163077:0:(obd_class.h:1056:obd_statfs_async()) lustre-OST0001-osc-ffff9181c5620000: age 8461, max_age 8461 00000008:00000004:2.0:1787826390.841534:0:163077:0:(osc_request.c:3417:osc_statfs_async()) lustre-OST0001-osc-ffff9181c5620000: use ffff9181c354b7c0 cache blocks 901360/958279 objects 261727/262144 00000080:00000004:2.0:1787826390.842012:0:163077:0:(obd_class.h:1116:obd_statfs()) lustre-clilov-ffff9181c5620000: update ffff9181d1a1b7c0 cache blocks 1802716/1916558 objects 523437/524288 00000080:00000004:2.0:1787826390.842019:0:163077:0:(llite_lib.c:2766:ll_statfs_internal()) OSC blocks 1802716/1916558 objects 523437/524288 00000080:00200000:2.0:1787826390.842042:0:163077:0:(dir.c:332:ll_iterate()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448) pos/size0/4096 32bit_api 0 00000080:00200000:2.0:1787826390.842255:0:163077:0:(dir.c:332:ll_iterate()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448) pos/size18446744073709551614/4096 32bit_api 0 00000080:00200000:2.0:1787826390.842280:0:163077:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.842291:0:163077:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f-5, flags=0 00000080:00200000:2.0:1787826390.842299:0:163077:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 3 00000080:00200000:2.0:1787826390.842305:0:163077:0:(namei.c:2418:ll_unlink()) VFS Op:name=f-5, dir=[0x240001f59:0x16:0x0](ffff9181d080f448) 00010000:00010000:2.0:1787826390.842346:0:163077:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843800/0xbef42c0221d6f457 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ceecaa expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00010000:00010000:2.0:1787826390.842446:0:163077:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843c00/0xbef42c0221d6f3cb lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x1c:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x878a5eea00ceea57 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826390.842473:0:163077:0:(vvp_object.c:117:vvp_conf_set()) [0x240001f59:0x1c:0x0]: losing layout lock 00010000:00010000:2.0:1787826390.842650:0:163077:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843c00/0xbef42c0221d6f3cb lrc: 1/0,0 mode: --/PR res: [0x240001f59:0x1c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00ceea57 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826390.842660:0:163077:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843800/0xbef42c0221d6f457 lrc: 1/0,0 mode: --/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ceecaa expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00010000:00010000:2.0:1787826390.842668:0:163077:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:2.0:1787826390.842671:0:163077:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843c00/0xbef42c0221d6f3cb lrc: 0/0,0 mode: --/PR res: [0x240001f59:0x1c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00ceea57 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826390.843022:0:163077:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843800/0xbef42c0221d6f457 lrc: 0/0,0 mode: --/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ceecaa expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00000020:00100000:2.0:1787826390.843044:0:163077:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787826390.843061:0:163077:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c49e6300 x1874663043054464/t0(0) o36->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826390.843203:0:163077:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c49e6300 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043054464:192.168.203.147@tcp:36:rm.0 00000100:00100000:2.0:1787826390.843348:0:163077:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d8141e40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826390.848125:0:163077:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c49e6300 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043054464:192.168.203.147@tcp:36:rm.0 00000080:00200000:2.0:1787826390.848200:0:163077:0:(vvp_io.c:1826:vvp_io_init()) [0x240001f59:0x1c:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826390.848216:0:163077:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826390.848250:0:163077:0:(vvp_io.c:304:vvp_io_fini()) [0x240001f59:0x1c:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826390.848259:0:163077:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f59:0x1c:0x0](ffff9181c830a1c8) 00000080:00200000:2.0:1787826390.848355:0:163077:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.848367:0:163077:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f-0, flags=0 00000080:00200000:2.0:1787826390.848373:0:163077:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 3 00000080:00200000:2.0:1787826390.848378:0:163077:0:(namei.c:2418:ll_unlink()) VFS Op:name=f-0, dir=[0x240001f59:0x16:0x0](ffff9181d080f448) 00010000:00010000:2.0:1787826390.848430:0:163077:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b4200/0xbef42c0221d6f3a8 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x878a5eea00cee9a8 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826390.848455:0:163077:0:(vvp_object.c:117:vvp_conf_set()) [0x240001f59:0x17:0x0]: losing layout lock 00010000:00010000:2.0:1787826390.848529:0:163077:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b4200/0xbef42c0221d6f3a8 lrc: 1/0,0 mode: --/PR res: [0x240001f59:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cee9a8 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826390.848537:0:163077:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826390.848541:0:163077:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b4200/0xbef42c0221d6f3a8 lrc: 0/0,0 mode: --/PR res: [0x240001f59:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cee9a8 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000020:00100000:2.0:1787826390.848572:0:163077:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787826390.848582:0:163077:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c49e4380 x1874663043054720/t0(0) o36->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826390.848613:0:163077:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c49e4380 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043054720:192.168.203.147@tcp:36:rm.0 00000100:00100000:2.0:1787826390.848711:0:163077:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d8141e40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826390.854109:0:163077:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c49e4380 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043054720:192.168.203.147@tcp:36:rm.0 00000080:00200000:2.0:1787826390.854188:0:163077:0:(vvp_io.c:1826:vvp_io_init()) [0x240001f59:0x17:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826390.854205:0:163077:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826390.854240:0:163077:0:(vvp_io.c:304:vvp_io_fini()) [0x240001f59:0x17:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826390.854250:0:163077:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f59:0x17:0x0](ffff9181c83e7448) 00000080:00200000:2.0:1787826390.854347:0:163077:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.854361:0:163077:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f-6, flags=0 00000080:00200000:2.0:1787826390.854368:0:163077:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 3 00000080:00200000:2.0:1787826390.854372:0:163077:0:(namei.c:2418:ll_unlink()) VFS Op:name=f-6, dir=[0x240001f59:0x16:0x0](ffff9181d080f448) 00010000:00010000:2.0:1787826390.854421:0:163077:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842600/0xbef42c0221d6f3d2 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x878a5eea00ceea7a expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826390.854444:0:163077:0:(vvp_object.c:117:vvp_conf_set()) [0x240001f59:0x1d:0x0]: losing layout lock 00010000:00010000:2.0:1787826390.854507:0:163077:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842600/0xbef42c0221d6f3d2 lrc: 1/0,0 mode: --/PR res: [0x240001f59:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00ceea7a expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826390.854523:0:163077:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826390.854526:0:163077:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842600/0xbef42c0221d6f3d2 lrc: 0/0,0 mode: --/PR res: [0x240001f59:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00ceea7a expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000020:00100000:2.0:1787826390.854558:0:163077:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787826390.854566:0:163077:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c49e5c00 x1874663043054976/t0(0) o36->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826390.854590:0:163077:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c49e5c00 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043054976:192.168.203.147@tcp:36:rm.0 00000100:00100000:2.0:1787826390.854663:0:163077:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d8141e40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826390.859846:0:163077:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c49e5c00 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043054976:192.168.203.147@tcp:36:rm.0 00000080:00200000:2.0:1787826390.859924:0:163077:0:(vvp_io.c:1826:vvp_io_init()) [0x240001f59:0x1d:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826390.859942:0:163077:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826390.859978:0:163077:0:(vvp_io.c:304:vvp_io_fini()) [0x240001f59:0x1d:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826390.859988:0:163077:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f59:0x1d:0x0](ffff9181c830cb08) 00000080:00200000:2.0:1787826390.860106:0:163077:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.860125:0:163077:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f-14, flags=0 00000080:00200000:2.0:1787826390.860132:0:163077:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 3 00000080:00200000:2.0:1787826390.860138:0:163077:0:(namei.c:2418:ll_unlink()) VFS Op:name=f-14, dir=[0x240001f59:0x16:0x0](ffff9181d080f448) 00010000:00010000:2.0:1787826390.860191:0:163077:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842800/0xbef42c0221d6f40a lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x25:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x878a5eea00ceeb92 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826390.860215:0:163077:0:(vvp_object.c:117:vvp_conf_set()) [0x240001f59:0x25:0x0]: losing layout lock 00010000:00010000:2.0:1787826390.860274:0:163077:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842800/0xbef42c0221d6f40a lrc: 1/0,0 mode: --/PR res: [0x240001f59:0x25:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00ceeb92 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826390.860281:0:163077:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826390.860285:0:163077:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842800/0xbef42c0221d6f40a lrc: 0/0,0 mode: --/PR res: [0x240001f59:0x25:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00ceeb92 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000020:00100000:2.0:1787826390.860311:0:163077:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787826390.860319:0:163077:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c49e4a80 x1874663043055232/t0(0) o36->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826390.860343:0:163077:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c49e4a80 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043055232:192.168.203.147@tcp:36:rm.0 00000100:00100000:2.0:1787826390.860439:0:163077:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d8141e40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826390.866337:0:163077:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c49e4a80 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043055232:192.168.203.147@tcp:36:rm.0 00000080:00200000:2.0:1787826390.866422:0:163077:0:(vvp_io.c:1826:vvp_io_init()) [0x240001f59:0x25:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826390.866441:0:163077:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826390.866488:0:163077:0:(vvp_io.c:304:vvp_io_fini()) [0x240001f59:0x25:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826390.866498:0:163077:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f59:0x25:0x0](ffff9181c830f448) 00000080:00200000:2.0:1787826390.866622:0:163077:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.866638:0:163077:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f-8, flags=0 00000080:00200000:2.0:1787826390.866646:0:163077:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 3 00000080:00200000:2.0:1787826390.866652:0:163077:0:(namei.c:2418:ll_unlink()) VFS Op:name=f-8, dir=[0x240001f59:0x16:0x0](ffff9181d080f448) 00010000:00010000:2.0:1787826390.866724:0:163077:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843a00/0xbef42c0221d6f3e0 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x1f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x878a5eea00ceeac0 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826390.867193:0:163077:0:(vvp_object.c:117:vvp_conf_set()) [0x240001f59:0x1f:0x0]: losing layout lock 00010000:00010000:2.0:1787826390.867262:0:163077:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843a00/0xbef42c0221d6f3e0 lrc: 1/0,0 mode: --/PR res: [0x240001f59:0x1f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00ceeac0 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826390.867271:0:163077:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826390.867274:0:163077:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843a00/0xbef42c0221d6f3e0 lrc: 0/0,0 mode: --/PR res: [0x240001f59:0x1f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00ceeac0 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000020:00100000:2.0:1787826390.867303:0:163077:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787826390.867315:0:163077:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c49e4e00 x1874663043055488/t0(0) o36->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826390.867341:0:163077:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c49e4e00 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043055488:192.168.203.147@tcp:36:rm.0 00000100:00100000:2.0:1787826390.867441:0:163077:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d8141e40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826390.874270:0:163077:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c49e4e00 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043055488:192.168.203.147@tcp:36:rm.0 00000080:00200000:2.0:1787826390.874347:0:163077:0:(vvp_io.c:1826:vvp_io_init()) [0x240001f59:0x1f:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826390.874365:0:163077:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826390.874401:0:163077:0:(vvp_io.c:304:vvp_io_fini()) [0x240001f59:0x1f:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826390.874410:0:163077:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f59:0x1f:0x0](ffff9181c8309988) 00000080:00200000:2.0:1787826390.874530:0:163077:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.874543:0:163077:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f-4, flags=0 00000080:00200000:2.0:1787826390.874554:0:163077:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 3 00000080:00200000:2.0:1787826390.874560:0:163077:0:(namei.c:2418:ll_unlink()) VFS Op:name=f-4, dir=[0x240001f59:0x16:0x0](ffff9181d080f448) 00010000:00010000:2.0:1787826390.874607:0:163077:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842c00/0xbef42c0221d6f3c4 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x878a5eea00ceea34 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826390.874629:0:163077:0:(vvp_object.c:117:vvp_conf_set()) [0x240001f59:0x1b:0x0]: losing layout lock 00010000:00010000:2.0:1787826390.874696:0:163077:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842c00/0xbef42c0221d6f3c4 lrc: 1/0,0 mode: --/PR res: [0x240001f59:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00ceea34 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826390.874705:0:163077:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826390.874709:0:163077:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842c00/0xbef42c0221d6f3c4 lrc: 0/0,0 mode: --/PR res: [0x240001f59:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00ceea34 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000020:00100000:2.0:1787826390.874735:0:163077:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787826390.874782:0:163077:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c49e7100 x1874663043055744/t0(0) o36->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826390.874808:0:163077:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c49e7100 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043055744:192.168.203.147@tcp:36:rm.0 00000100:00100000:2.0:1787826390.875319:0:163077:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d8141e40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826390.883345:0:163077:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c49e7100 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043055744:192.168.203.147@tcp:36:rm.0 00000080:00200000:2.0:1787826390.883435:0:163077:0:(vvp_io.c:1826:vvp_io_init()) [0x240001f59:0x1b:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826390.883453:0:163077:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826390.883506:0:163077:0:(vvp_io.c:304:vvp_io_fini()) [0x240001f59:0x1b:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826390.883653:0:163077:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f59:0x1b:0x0](ffff9181c830ec08) 00000080:00200000:2.0:1787826390.884233:0:163077:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.884254:0:163077:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f-13, flags=0 00000080:00200000:2.0:1787826390.884262:0:163077:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 3 00000080:00200000:2.0:1787826390.884268:0:163077:0:(namei.c:2418:ll_unlink()) VFS Op:name=f-13, dir=[0x240001f59:0x16:0x0](ffff9181d080f448) 00010000:00010000:2.0:1787826390.884326:0:163077:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481e00/0xbef42c0221d6f403 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x24:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x878a5eea00ceeb6f expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826390.884352:0:163077:0:(vvp_object.c:117:vvp_conf_set()) [0x240001f59:0x24:0x0]: losing layout lock 00010000:00010000:2.0:1787826390.884420:0:163077:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481e00/0xbef42c0221d6f403 lrc: 1/0,0 mode: --/PR res: [0x240001f59:0x24:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00ceeb6f expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826390.884428:0:163077:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826390.884432:0:163077:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481e00/0xbef42c0221d6f403 lrc: 0/0,0 mode: --/PR res: [0x240001f59:0x24:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00ceeb6f expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000020:00100000:2.0:1787826390.884475:0:163077:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787826390.884489:0:163077:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c49e7480 x1874663043056000/t0(0) o36->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826390.884650:0:163077:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c49e7480 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043056000:192.168.203.147@tcp:36:rm.0 00000100:00100000:2.0:1787826390.885368:0:163077:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d8141e40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826390.890565:0:163077:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c49e7480 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043056000:192.168.203.147@tcp:36:rm.0 00000080:00200000:2.0:1787826390.890652:0:163077:0:(vvp_io.c:1826:vvp_io_init()) [0x240001f59:0x24:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826390.890669:0:163077:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826390.890721:0:163077:0:(vvp_io.c:304:vvp_io_fini()) [0x240001f59:0x24:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826390.890731:0:163077:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f59:0x24:0x0](ffff9181d080db88) 00000080:00200000:2.0:1787826390.890891:0:163077:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.890907:0:163077:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f-15, flags=0 00000080:00200000:2.0:1787826390.890913:0:163077:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 3 00000080:00200000:2.0:1787826390.890918:0:163077:0:(namei.c:2418:ll_unlink()) VFS Op:name=f-15, dir=[0x240001f59:0x16:0x0](ffff9181d080f448) 00010000:00010000:2.0:1787826390.890970:0:163077:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842200/0xbef42c0221d6f411 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x26:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x878a5eea00ceebb5 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826390.890994:0:163077:0:(vvp_object.c:117:vvp_conf_set()) [0x240001f59:0x26:0x0]: losing layout lock 00010000:00010000:2.0:1787826390.891050:0:163077:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842200/0xbef42c0221d6f411 lrc: 1/0,0 mode: --/PR res: [0x240001f59:0x26:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00ceebb5 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826390.891057:0:163077:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826390.891060:0:163077:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842200/0xbef42c0221d6f411 lrc: 0/0,0 mode: --/PR res: [0x240001f59:0x26:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00ceebb5 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000020:00100000:2.0:1787826390.891098:0:163077:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787826390.891109:0:163077:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c49e6d80 x1874663043056256/t0(0) o36->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826390.891136:0:163077:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c49e6d80 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043056256:192.168.203.147@tcp:36:rm.0 00000100:00100000:2.0:1787826390.891224:0:163077:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d8141e40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826390.898308:0:163077:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c49e6d80 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043056256:192.168.203.147@tcp:36:rm.0 00000080:00200000:2.0:1787826390.898389:0:163077:0:(vvp_io.c:1826:vvp_io_init()) [0x240001f59:0x26:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826390.898406:0:163077:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826390.898444:0:163077:0:(vvp_io.c:304:vvp_io_fini()) [0x240001f59:0x26:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826390.898453:0:163077:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f59:0x26:0x0](ffff9181c830e3c8) 00000080:00200000:2.0:1787826390.898582:0:163077:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.898598:0:163077:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f-11, flags=0 00000080:00200000:2.0:1787826390.898606:0:163077:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 3 00000080:00200000:2.0:1787826390.898613:0:163077:0:(namei.c:2418:ll_unlink()) VFS Op:name=f-11, dir=[0x240001f59:0x16:0x0](ffff9181d080f448) 00010000:00010000:2.0:1787826390.898670:0:163077:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481c00/0xbef42c0221d6f3f5 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x22:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x878a5eea00ceeb29 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826390.898709:0:163077:0:(vvp_object.c:117:vvp_conf_set()) [0x240001f59:0x22:0x0]: losing layout lock 00010000:00010000:2.0:1787826390.898799:0:163077:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481c00/0xbef42c0221d6f3f5 lrc: 1/0,0 mode: --/PR res: [0x240001f59:0x22:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00ceeb29 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826390.898807:0:163077:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826390.898810:0:163077:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481c00/0xbef42c0221d6f3f5 lrc: 0/0,0 mode: --/PR res: [0x240001f59:0x22:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00ceeb29 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000020:00100000:2.0:1787826390.898955:0:163077:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787826390.898966:0:163077:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181e007ca80 x1874663043056512/t0(0) o36->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826390.898995:0:163077:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043056512:192.168.203.147@tcp:36:rm.0 00000100:00100000:2.0:1787826390.899236:0:163077:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d8141e40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826390.905543:0:163077:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043056512:192.168.203.147@tcp:36:rm.0 00000080:00200000:2.0:1787826390.905637:0:163077:0:(vvp_io.c:1826:vvp_io_init()) [0x240001f59:0x22:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826390.905656:0:163077:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826390.905712:0:163077:0:(vvp_io.c:304:vvp_io_fini()) [0x240001f59:0x22:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826390.905722:0:163077:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f59:0x22:0x0](ffff9181d080a1c8) 00000080:00200000:2.0:1787826390.906220:0:163077:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.906236:0:163077:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f-9, flags=0 00000080:00200000:2.0:1787826390.906242:0:163077:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 3 00000080:00200000:2.0:1787826390.906248:0:163077:0:(namei.c:2418:ll_unlink()) VFS Op:name=f-9, dir=[0x240001f59:0x16:0x0](ffff9181d080f448) 00010000:00010000:2.0:1787826390.906301:0:163077:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843e00/0xbef42c0221d6f3e7 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x878a5eea00ceeae3 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826390.906326:0:163077:0:(vvp_object.c:117:vvp_conf_set()) [0x240001f59:0x20:0x0]: losing layout lock 00010000:00010000:2.0:1787826390.906389:0:163077:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843e00/0xbef42c0221d6f3e7 lrc: 1/0,0 mode: --/PR res: [0x240001f59:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00ceeae3 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826390.906398:0:163077:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826390.906401:0:163077:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843e00/0xbef42c0221d6f3e7 lrc: 0/0,0 mode: --/PR res: [0x240001f59:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00ceeae3 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000020:00100000:2.0:1787826390.906426:0:163077:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787826390.906437:0:163077:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181e007fb80 x1874663043056768/t0(0) o36->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826390.906475:0:163077:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043056768:192.168.203.147@tcp:36:rm.0 00000100:00100000:2.0:1787826390.906564:0:163077:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d8141e40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826390.913025:0:163077:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043056768:192.168.203.147@tcp:36:rm.0 00000080:00200000:2.0:1787826390.913172:0:163077:0:(vvp_io.c:1826:vvp_io_init()) [0x240001f59:0x20:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826390.913187:0:163077:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826390.913224:0:163077:0:(vvp_io.c:304:vvp_io_fini()) [0x240001f59:0x20:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826390.913233:0:163077:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f59:0x20:0x0](ffff9181c830ba88) 00000080:00200000:2.0:1787826390.913339:0:163077:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.913353:0:163077:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f-3, flags=0 00000080:00200000:2.0:1787826390.913360:0:163077:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 3 00000080:00200000:2.0:1787826390.913366:0:163077:0:(namei.c:2418:ll_unlink()) VFS Op:name=f-3, dir=[0x240001f59:0x16:0x0](ffff9181d080f448) 00010000:00010000:2.0:1787826390.913418:0:163077:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b5400/0xbef42c0221d6f3bd lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x1a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x878a5eea00ceea11 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826390.913443:0:163077:0:(vvp_object.c:117:vvp_conf_set()) [0x240001f59:0x1a:0x0]: losing layout lock 00010000:00010000:2.0:1787826390.916010:0:163077:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b5400/0xbef42c0221d6f3bd lrc: 1/0,0 mode: --/PR res: [0x240001f59:0x1a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00ceea11 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826390.916037:0:163077:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826390.916041:0:163077:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b5400/0xbef42c0221d6f3bd lrc: 0/0,0 mode: --/PR res: [0x240001f59:0x1a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00ceea11 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000020:00100000:2.0:1787826390.916233:0:163077:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787826390.916250:0:163077:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181e007ce00 x1874663043057024/t0(0) o36->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826390.916291:0:163077:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043057024:192.168.203.147@tcp:36:rm.0 00000100:00100000:2.0:1787826390.916405:0:163077:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d8141e40 going to sleep for 11 seconds 00000100:00100000:2.0:1787826390.922565:0:163077:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043057024:192.168.203.147@tcp:36:rm.0 00000080:00200000:2.0:1787826390.922651:0:163077:0:(vvp_io.c:1826:vvp_io_init()) [0x240001f59:0x1a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826390.922667:0:163077:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826390.923063:0:163077:0:(vvp_io.c:304:vvp_io_fini()) [0x240001f59:0x1a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826390.923074:0:163077:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f59:0x1a:0x0](ffff9181c830aa08) 00000080:00200000:2.0:1787826390.923198:0:163077:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.923214:0:163077:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f-19, flags=0 00000080:00200000:2.0:1787826390.923220:0:163077:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 3 00000080:00200000:2.0:1787826390.923226:0:163077:0:(namei.c:2418:ll_unlink()) VFS Op:name=f-19, dir=[0x240001f59:0x16:0x0](ffff9181d080f448) 00010000:00010000:2.0:1787826390.923280:0:163077:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843600/0xbef42c0221d6f42d lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x878a5eea00ceec41 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826390.923306:0:163077:0:(vvp_object.c:117:vvp_conf_set()) [0x240001f59:0x2a:0x0]: losing layout lock 00010000:00010000:2.0:1787826390.923368:0:163077:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843600/0xbef42c0221d6f42d lrc: 1/0,0 mode: --/PR res: [0x240001f59:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00ceec41 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826390.923377:0:163077:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826390.923382:0:163077:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843600/0xbef42c0221d6f42d lrc: 0/0,0 mode: --/PR res: [0x240001f59:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00ceec41 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000020:00100000:2.0:1787826390.923407:0:163077:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787826390.923419:0:163077:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181e007c700 x1874663043057280/t0(0) o36->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826390.923445:0:163077:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043057280:192.168.203.147@tcp:36:rm.0 00000100:00100000:3.0:1787826390.924382:0:163077:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d8141e40 going to sleep for 11 seconds 00000100:00100000:3.0:1787826390.929239:0:163077:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043057280:192.168.203.147@tcp:36:rm.0 00000080:00200000:3.0:1787826390.929316:0:163077:0:(vvp_io.c:1826:vvp_io_init()) [0x240001f59:0x2a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826390.929333:0:163077:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826390.929411:0:163077:0:(vvp_io.c:304:vvp_io_fini()) [0x240001f59:0x2a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826390.929421:0:163077:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f59:0x2a:0x0](ffff9181c830d348) 00000080:00200000:3.0:1787826390.929621:0:163077:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 1 00000080:00200000:3.0:1787826390.929637:0:163077:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f-18, flags=0 00000080:00200000:3.0:1787826390.929643:0:163077:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 3 00000080:00200000:3.0:1787826390.929650:0:163077:0:(namei.c:2418:ll_unlink()) VFS Op:name=f-18, dir=[0x240001f59:0x16:0x0](ffff9181d080f448) 00010000:00010000:3.0:1787826390.929979:0:163077:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f426 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x29:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x878a5eea00ceec1e expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826390.930004:0:163077:0:(vvp_object.c:117:vvp_conf_set()) [0x240001f59:0x29:0x0]: losing layout lock 00010000:00010000:3.0:1787826390.930065:0:163077:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f426 lrc: 1/0,0 mode: --/PR res: [0x240001f59:0x29:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00ceec1e expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826390.930074:0:163077:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826390.930077:0:163077:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f426 lrc: 0/0,0 mode: --/PR res: [0x240001f59:0x29:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00ceec1e expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000020:00100000:3.0:1787826390.930449:0:163077:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:3.0:1787826390.930471:0:163077:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c5637100 x1874663043057536/t0(0) o36->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826390.930500:0:163077:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043057536:192.168.203.147@tcp:36:rm.0 00000100:00100000:3.0:1787826390.930617:0:163077:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d22a2600 going to sleep for 11 seconds 00000100:00100000:3.0:1787826390.940195:0:163077:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043057536:192.168.203.147@tcp:36:rm.0 00000080:00200000:3.0:1787826390.940280:0:163077:0:(vvp_io.c:1826:vvp_io_init()) [0x240001f59:0x29:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826390.940296:0:163077:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826390.940333:0:163077:0:(vvp_io.c:304:vvp_io_fini()) [0x240001f59:0x29:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826390.940343:0:163077:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f59:0x29:0x0](ffff9181c83080c8) 00000080:00200000:3.0:1787826390.940448:0:163077:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 1 00000080:00200000:3.0:1787826390.940472:0:163077:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f-2, flags=0 00000080:00200000:3.0:1787826390.940479:0:163077:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 3 00000080:00200000:3.0:1787826390.940485:0:163077:0:(namei.c:2418:ll_unlink()) VFS Op:name=f-2, dir=[0x240001f59:0x16:0x0](ffff9181d080f448) 00010000:00010000:3.0:1787826390.940539:0:163077:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b4000/0xbef42c0221d6f3b6 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x878a5eea00cee9ee expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826390.940577:0:163077:0:(vvp_object.c:117:vvp_conf_set()) [0x240001f59:0x19:0x0]: losing layout lock 00010000:00010000:3.0:1787826390.940636:0:163077:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b4000/0xbef42c0221d6f3b6 lrc: 1/0,0 mode: --/PR res: [0x240001f59:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cee9ee expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826390.940646:0:163077:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826390.940650:0:163077:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b4000/0xbef42c0221d6f3b6 lrc: 0/0,0 mode: --/PR res: [0x240001f59:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cee9ee expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000020:00100000:3.0:1787826390.940678:0:163077:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:3.0:1787826390.940913:0:163077:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c5635500 x1874663043057792/t0(0) o36->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826390.940945:0:163077:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043057792:192.168.203.147@tcp:36:rm.0 00000100:00100000:3.0:1787826390.941039:0:163077:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d22a2600 going to sleep for 11 seconds 00000100:00100000:3.0:1787826390.947405:0:163077:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043057792:192.168.203.147@tcp:36:rm.0 00000080:00200000:3.0:1787826390.947503:0:163077:0:(vvp_io.c:1826:vvp_io_init()) [0x240001f59:0x19:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826390.947531:0:163077:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826390.947579:0:163077:0:(vvp_io.c:304:vvp_io_fini()) [0x240001f59:0x19:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826390.947589:0:163077:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f59:0x19:0x0](ffff9181c83e1988) 00000080:00200000:3.0:1787826390.947717:0:163077:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 1 00000080:00200000:3.0:1787826390.947731:0:163077:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f-1, flags=0 00000080:00200000:3.0:1787826390.947737:0:163077:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 3 00000080:00200000:3.0:1787826390.947795:0:163077:0:(namei.c:2418:ll_unlink()) VFS Op:name=f-1, dir=[0x240001f59:0x16:0x0](ffff9181d080f448) 00010000:00010000:3.0:1787826390.947851:0:163077:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b5200/0xbef42c0221d6f3af lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x878a5eea00cee9cb expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:3.0:1787826390.947875:0:163077:0:(vvp_object.c:117:vvp_conf_set()) [0x240001f59:0x18:0x0]: losing layout lock 00010000:00010000:3.0:1787826390.947936:0:163077:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b5200/0xbef42c0221d6f3af lrc: 1/0,0 mode: --/PR res: [0x240001f59:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cee9cb expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:3.0:1787826390.947945:0:163077:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826390.947948:0:163077:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e01b5200/0xbef42c0221d6f3af lrc: 0/0,0 mode: --/PR res: [0x240001f59:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cee9cb expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000020:00100000:3.0:1787826390.947976:0:163077:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:3.0:1787826390.947985:0:163077:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c5637800 x1874663043058048/t0(0) o36->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826390.948012:0:163077:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043058048:192.168.203.147@tcp:36:rm.0 00000100:00100000:3.0:1787826390.948097:0:163077:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d22a2600 going to sleep for 11 seconds 00000100:00100000:3.0:1787826390.953051:0:163077:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043058048:192.168.203.147@tcp:36:rm.0 00000080:00200000:3.0:1787826390.953142:0:163077:0:(vvp_io.c:1826:vvp_io_init()) [0x240001f59:0x18:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826390.953158:0:163077:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826390.953195:0:163077:0:(vvp_io.c:304:vvp_io_fini()) [0x240001f59:0x18:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826390.953204:0:163077:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f59:0x18:0x0](ffff9181c83e21c8) 00000080:00200000:3.0:1787826390.953306:0:163077:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 1 00000080:00200000:3.0:1787826390.953319:0:163077:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f-16, flags=0 00000080:00200000:3.0:1787826390.953324:0:163077:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 3 00000080:00200000:3.0:1787826390.953329:0:163077:0:(namei.c:2418:ll_unlink()) VFS Op:name=f-16, dir=[0x240001f59:0x16:0x0](ffff9181d080f448) 00010000:00010000:3.0:1787826390.953378:0:163077:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842000/0xbef42c0221d6f418 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x878a5eea00ceebd8 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826390.953400:0:163077:0:(vvp_object.c:117:vvp_conf_set()) [0x240001f59:0x27:0x0]: losing layout lock 00010000:00010000:3.0:1787826390.953470:0:163077:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842000/0xbef42c0221d6f418 lrc: 1/0,0 mode: --/PR res: [0x240001f59:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00ceebd8 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826390.953483:0:163077:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826390.953486:0:163077:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842000/0xbef42c0221d6f418 lrc: 0/0,0 mode: --/PR res: [0x240001f59:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00ceebd8 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000020:00100000:3.0:1787826390.953511:0:163077:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:3.0:1787826390.953520:0:163077:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c5635c00 x1874663043058304/t0(0) o36->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826390.953544:0:163077:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043058304:192.168.203.147@tcp:36:rm.0 00000100:00100000:3.0:1787826390.953640:0:163077:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d22a2600 going to sleep for 11 seconds 00000100:00100000:3.0:1787826390.960477:0:163077:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043058304:192.168.203.147@tcp:36:rm.0 00000080:00200000:3.0:1787826390.960573:0:163077:0:(vvp_io.c:1826:vvp_io_init()) [0x240001f59:0x27:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826390.960591:0:163077:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826390.960629:0:163077:0:(vvp_io.c:304:vvp_io_fini()) [0x240001f59:0x27:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826390.960639:0:163077:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f59:0x27:0x0](ffff9181c8309148) 00000080:00200000:3.0:1787826390.960967:0:163077:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 1 00000080:00200000:3.0:1787826390.960982:0:163077:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f-7, flags=0 00000080:00200000:3.0:1787826390.960988:0:163077:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 3 00000080:00200000:3.0:1787826390.960993:0:163077:0:(namei.c:2418:ll_unlink()) VFS Op:name=f-7, dir=[0x240001f59:0x16:0x0](ffff9181d080f448) 00010000:00010000:3.0:1787826390.961044:0:163077:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842400/0xbef42c0221d6f3d9 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x1e:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x878a5eea00ceea9d expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826390.961069:0:163077:0:(vvp_object.c:117:vvp_conf_set()) [0x240001f59:0x1e:0x0]: losing layout lock 00010000:00010000:3.0:1787826390.961139:0:163077:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842400/0xbef42c0221d6f3d9 lrc: 1/0,0 mode: --/PR res: [0x240001f59:0x1e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00ceea9d expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826390.961148:0:163077:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826390.961152:0:163077:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842400/0xbef42c0221d6f3d9 lrc: 0/0,0 mode: --/PR res: [0x240001f59:0x1e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00ceea9d expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000020:00100000:3.0:1787826390.961178:0:163077:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:3.0:1787826390.961189:0:163077:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c5637b80 x1874663043058560/t0(0) o36->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826390.961219:0:163077:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043058560:192.168.203.147@tcp:36:rm.0 00000100:00100000:3.0:1787826390.961301:0:163077:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d22a2600 going to sleep for 11 seconds 00000100:00100000:3.0:1787826390.973190:0:163077:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043058560:192.168.203.147@tcp:36:rm.0 00000080:00200000:3.0:1787826390.973270:0:163077:0:(vvp_io.c:1826:vvp_io_init()) [0x240001f59:0x1e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1787826390.973287:0:163077:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1787826390.973325:0:163077:0:(vvp_io.c:304:vvp_io_fini()) [0x240001f59:0x1e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826390.973335:0:163077:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f59:0x1e:0x0](ffff9181c830b248) 00000080:00200000:3.0:1787826390.973508:0:163077:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 1 00000080:00200000:3.0:1787826390.973522:0:163077:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f-12, flags=0 00000080:00200000:3.0:1787826390.973529:0:163077:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 3 00000080:00200000:3.0:1787826390.973534:0:163077:0:(namei.c:2418:ll_unlink()) VFS Op:name=f-12, dir=[0x240001f59:0x16:0x0](ffff9181d080f448) 00010000:00010000:2.0:1787826390.976946:0:163077:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f3fc lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x878a5eea00ceeb4c expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826390.976984:0:163077:0:(vvp_object.c:117:vvp_conf_set()) [0x240001f59:0x23:0x0]: losing layout lock 00010000:00010000:2.0:1787826390.977059:0:163077:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f3fc lrc: 1/0,0 mode: --/PR res: [0x240001f59:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00ceeb4c expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826390.977068:0:163077:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826390.977072:0:163077:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f3fc lrc: 0/0,0 mode: --/PR res: [0x240001f59:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00ceeb4c expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000020:00100000:2.0:1787826390.977114:0:163077:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787826390.977131:0:163077:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181e007e300 x1874663043058816/t0(0) o36->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826390.977163:0:163077:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043058816:192.168.203.147@tcp:36:rm.0 00000100:00100000:2.0:1787826390.977270:0:163077:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d81410c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826390.981721:0:163077:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043058816:192.168.203.147@tcp:36:rm.0 00000080:00200000:2.0:1787826390.981840:0:163077:0:(vvp_io.c:1826:vvp_io_init()) [0x240001f59:0x23:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826390.981857:0:163077:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826390.981897:0:163077:0:(vvp_io.c:304:vvp_io_fini()) [0x240001f59:0x23:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826390.981908:0:163077:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f59:0x23:0x0](ffff9181d0809988) 00000080:00200000:2.0:1787826390.982011:0:163077:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.982026:0:163077:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f-10, flags=0 00000080:00200000:2.0:1787826390.982033:0:163077:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 3 00000080:00200000:2.0:1787826390.982039:0:163077:0:(namei.c:2418:ll_unlink()) VFS Op:name=f-10, dir=[0x240001f59:0x16:0x0](ffff9181d080f448) 00010000:00010000:2.0:1787826390.982087:0:163077:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843400/0xbef42c0221d6f3ee lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x21:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x878a5eea00ceeb06 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826390.982111:0:163077:0:(vvp_object.c:117:vvp_conf_set()) [0x240001f59:0x21:0x0]: losing layout lock 00010000:00010000:2.0:1787826390.982169:0:163077:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843400/0xbef42c0221d6f3ee lrc: 1/0,0 mode: --/PR res: [0x240001f59:0x21:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00ceeb06 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826390.982177:0:163077:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826390.982181:0:163077:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1843400/0xbef42c0221d6f3ee lrc: 0/0,0 mode: --/PR res: [0x240001f59:0x21:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00ceeb06 expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000020:00100000:2.0:1787826390.982208:0:163077:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787826390.982218:0:163077:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181e007f100 x1874663043059072/t0(0) o36->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826390.982243:0:163077:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043059072:192.168.203.147@tcp:36:rm.0 00000100:00100000:2.0:1787826390.983050:0:163077:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d81410c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826390.989109:0:163077:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043059072:192.168.203.147@tcp:36:rm.0 00000080:00200000:2.0:1787826390.989188:0:163077:0:(vvp_io.c:1826:vvp_io_init()) [0x240001f59:0x21:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826390.989206:0:163077:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826390.989241:0:163077:0:(vvp_io.c:304:vvp_io_fini()) [0x240001f59:0x21:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826390.989251:0:163077:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f59:0x21:0x0](ffff9181c830db88) 00000080:00200000:2.0:1787826390.989358:0:163077:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.989372:0:163077:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f-17, flags=0 00000080:00200000:2.0:1787826390.989379:0:163077:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448), inode mode 41ed mask 3 00000080:00200000:2.0:1787826390.989384:0:163077:0:(namei.c:2418:ll_unlink()) VFS Op:name=f-17, dir=[0x240001f59:0x16:0x0](ffff9181d080f448) 00010000:00010000:2.0:1787826390.989437:0:163077:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0512a00/0xbef42c0221d6f41f lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x878a5eea00ceebfb expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826390.989474:0:163077:0:(vvp_object.c:117:vvp_conf_set()) [0x240001f59:0x28:0x0]: losing layout lock 00010000:00010000:2.0:1787826390.989536:0:163077:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0512a00/0xbef42c0221d6f41f lrc: 1/0,0 mode: --/PR res: [0x240001f59:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00ceebfb expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826390.989545:0:163077:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1787826390.989549:0:163077:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0512a00/0xbef42c0221d6f41f lrc: 0/0,0 mode: --/PR res: [0x240001f59:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00ceebfb expref: -99 pid: 163071 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000020:00100000:2.0:1787826390.989588:0:163077:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1787826390.989598:0:163077:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181e007f800 x1874663043059328/t0(0) o36->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 488/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826390.989625:0:163077:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043059328:192.168.203.147@tcp:36:rm.0 00000100:00100000:2.0:1787826390.989738:0:163077:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d81410c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826390.992732:0:163077:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043059328:192.168.203.147@tcp:36:rm.0 00000080:00200000:2.0:1787826390.992848:0:163077:0:(vvp_io.c:1826:vvp_io_init()) [0x240001f59:0x28:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1787826390.992866:0:163077:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1787826390.992905:0:163077:0:(vvp_io.c:304:vvp_io_fini()) [0x240001f59:0x28:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1787826390.992915:0:163077:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f59:0x28:0x0](ffff9181c83f2a08) 00000080:00200008:2.0:1787826390.993022:0:163077:0:(file.c:479:ll_file_release()) START file remote_dir:[0x240001f59:0x16:0x0](ffff9181d080f448), flags 704000 00010000:00010000:2.0:1787826390.993058:0:163077:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842e00/0xbef42c0221d6f45e lrc: 2/0,0 mode: CR/CR res: [0x240001f59:0x16:0x0].0x0 bits 0x5/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceecb8 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:2.0:1787826390.993108:0:163077:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x4:0x0](ffff9181d0809148), inode mode 41ed mask 1 00000080:00200000:2.0:1787826390.993130:0:163077:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=remote_dir, dir=[0x200027cd1:0x4:0x0](ffff9181d0809148), flags=0 00000080:00200000:2.0:1787826390.993135:0:163077:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=remote_dir, dir=[0x200027cd1:0x4:0x0](ffff9181d0809148), intent=getattr 00000002:00010000:2.0:1787826390.993167:0:163077:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: remote_dir,[0x0:0x0:0x0]) in obj [0x200027cd1:0x4:0x0], intent: getattr flags 01 00010000:00010000:2.0:1787826390.993237:0:163077:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843600/0xbef42c0221d6f465 lrc: 3/1,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x10000000000000 pid: 163077 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.993246:0:163077:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843600/0xbef42c0221d6f465 lrc: 3/1,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x0 pid: 163077 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826390.993257:0:163077:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843600/0xbef42c0221d6f465 lrc: 3/1,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x0 pid: 163077 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826390.993280:0:163077:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ed80 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043059456:192.168.203.147@tcp:101:rm.0 00000100:00100000:2.0:1787826390.993376:0:163077:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d81410c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826391.002991:0:163077:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ed80 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043059456:192.168.203.147@tcp:101:rm.0 00010000:00010000:2.0:1787826391.003028:0:163077:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843600/0xbef42c0221d6f465 lrc: 4/1,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceef0b expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826391.003050:0:163077:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843600/0xbef42c0221d6f465 lrc: 4/1,0 mode: --/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceef0b expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826391.003059:0:163077:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843600/0xbef42c0221d6f465 lrc: 4/1,0 mode: --/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceef0b expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826391.003073:0:163077:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843600/0xbef42c0221d6f465 lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceef0b expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826391.003224:0:163077:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843600/0xbef42c0221d6f465 lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceef0b expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826391.003240:0:163077:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843600/0xbef42c0221d6f465 lrc: 5/2,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceef0b expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826391.003248:0:163077:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843600/0xbef42c0221d6f465 lrc: 5/2,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceef0b expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826391.003256:0:163077:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843600/0xbef42c0221d6f465 lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceef0b expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787826391.003270:0:163077:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181e007ed80 x1874663043059456/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 584/584 e 0 to 0 dl 1787826406 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787826391.003285:0:163077:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843600/0xbef42c0221d6f465 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceef0b expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826391.003298:0:163077:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4600/0xbef42c0221d6f39a lrc: 2/1,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee970 expref: -99 pid: 163065 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826391.003400:0:163077:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4600/0xbef42c0221d6f39a lrc: 2/1,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee970 expref: -99 pid: 163065 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826391.003416:0:163077:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843600/0xbef42c0221d6f465 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceef0b expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826391.003424:0:163077:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843600/0xbef42c0221d6f465 lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000400000000 nid: local remote: 0x878a5eea00ceef0b expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826391.003432:0:163077:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843600/0xbef42c0221d6f465 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000400000000 nid: local remote: 0x878a5eea00ceef0b expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826391.003470:0:163077:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000002:00010000:0.0:1787826391.003492:0:163077:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x0:0x0:0x0]) in obj [0x240001f59:0x16:0x0], intent: getattr flags 01 00010000:00010000:3.0:1787826391.003504:0:159733:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826391.003512:0:159733:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843600/0xbef42c0221d6f465 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x400000000 nid: local remote: 0x878a5eea00ceef0b expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826391.003522:0:159733:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181d1843600 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:3.0:1787826391.003528:0:159733:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843600/0xbef42c0221d6f465 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT gid 0 flags: 0x400000000 nid: local remote: 0x878a5eea00ceef0b expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826391.003536:0:159733:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843600/0xbef42c0221d6f465 lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT gid 0 flags: 0x400000000 nid: local remote: 0x878a5eea00ceef0b expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826391.003550:0:159733:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843600/0xbef42c0221d6f465 lrc: 3/0,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ceef0b expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826391.003570:0:159733:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4600/0xbef42c0221d6f39a lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee970 expref: -99 pid: 163065 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826391.003633:0:159733:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843600/0xbef42c0221d6f465 lrc: 2/0,0 mode: --/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ceef0b expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826391.003640:0:159733:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826391.003666:0:163077:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0512c00/0xbef42c0221d6f46c lrc: 3/1,0 mode: --/CR res: [0x240001f59:0x16:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 163077 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826391.003671:0:159733:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843600/0xbef42c0221d6f465 lrc: 1/0,0 mode: --/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ceef0b expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826391.003679:0:159733:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843600/0xbef42c0221d6f465 lrc: 0/0,0 mode: --/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ceef0b expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826391.003705:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826391.003707:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826391.003708:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826391.003807:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663043059584:192.168.203.147@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1787826391.005271:0:163077:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0512c00/0xbef42c0221d6f46c lrc: 3/1,0 mode: --/CR res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 163077 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826391.005301:0:163077:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0512c00/0xbef42c0221d6f46c lrc: 3/1,0 mode: --/CR res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 163077 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826391.005334:0:163077:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043059712:192.168.203.147@tcp:101:rm.0 00000100:00100000:0.0:1787826391.005410:0:163077:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d01a56c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826391.008712:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663043059584:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1787826391.010190:0:163077:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043059712:192.168.203.147@tcp:101:rm.0 00010000:00010000:0.0:1787826391.010224:0:163077:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0512c00/0xbef42c0221d6f46c lrc: 4/1,0 mode: --/CR res: [0x240001f59:0x16:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceef19 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826391.010238:0:163077:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0512c00/0xbef42c0221d6f46c lrc: 4/1,0 mode: --/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceef19 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826391.010253:0:163077:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0512c00/0xbef42c0221d6f46c lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceef19 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826391.010266:0:163077:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0512c00/0xbef42c0221d6f46c lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceef19 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826391.010281:0:163077:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0512c00/0xbef42c0221d6f46c lrc: 5/2,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceef19 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826391.010290:0:163077:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0512c00/0xbef42c0221d6f46c lrc: 5/2,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceef19 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826391.010297:0:163077:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0512c00/0xbef42c0221d6f46c lrc: 4/1,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceef19 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1787826391.010311:0:163077:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c55a5880 x1874663043059712/t0(0) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826407 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1787826391.010327:0:163077:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0512c00/0xbef42c0221d6f46c lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceef19 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826391.010342:0:163077:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181c77f3000 type 13 mode 4 res 9663684441/22 (0 0) 00000080:00200000:0.0:1787826391.010363:0:163077:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200027cd1:0x4:0x0] suppgids 0 -1: rc 0 00000080:00200000:0.0:1787826391.010402:0:163077:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x240001f59:0x16:0x0], updating i_size 4096 00000080:00200000:0.0:1787826391.010405:0:163077:0:(namei.c:147:ll_iget()) got inode: [0x240001f59:0x16:0x0](ffff9181d080f448): rc = 0 00000080:00010000:0.0:1787826391.010412:0:163077:0:(llite_internal.h:1981:ll_set_lock_data()) setting l_data to inode [0x240001f59:0x16:0x0](ffff9181d080f448) for remote lock 0xbef42c0221d6f46c 00000080:00010000:0.0:1787826391.010417:0:163077:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x240001f59:0x16:0x0](ffff9181d080f448) for lock 0xbef42c0221d6f39a 00000080:00010000:0.0:1787826391.010429:0:163077:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x240001f59:0x16:0x0](ffff9181d080f448) 00000080:00010000:0.0:1787826391.010433:0:163077:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f39a from it ffffb97502bafe00 00010000:00010000:0.0:1787826391.010437:0:163077:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4600/0xbef42c0221d6f39a lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee970 expref: -99 pid: 163065 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826391.010447:0:163077:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4600/0xbef42c0221d6f39a lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee970 expref: -99 pid: 163065 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00010000:0.0:1787826391.010477:0:163077:0:(dcache.c:164:ll_intent_drop_lock()) releasing remote lock with cookie 0xbef42c0221d6f46c from it ffffb97502bafe00 00010000:00010000:0.0:1787826391.010481:0:163077:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0512c00/0xbef42c0221d6f46c lrc: 3/1,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceef19 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826391.010488:0:163077:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0512c00/0xbef42c0221d6f46c lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceef19 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1787826391.010513:0:163077:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x4:0x0](ffff9181d0809148), inode mode 41ed mask 3 00000080:00200000:0.0:1787826391.010522:0:163077:0:(namei.c:2328:ll_rmdir()) VFS Op:name=remote_dir, dir=[0x200027cd1:0x4:0x0](ffff9181d0809148) 00010000:00010000:0.0:1787826391.010556:0:163077:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4600/0xbef42c0221d6f39a lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00cee970 expref: -99 pid: 163065 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787826391.010610:0:163077:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4600/0xbef42c0221d6f39a lrc: 1/0,0 mode: --/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cee970 expref: -99 pid: 163065 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787826391.010617:0:163077:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826391.010659:0:163077:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b4600/0xbef42c0221d6f39a lrc: 0/0,0 mode: --/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cee970 expref: -99 pid: 163065 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000100:00100000:0.0:1787826391.010693:0:2390:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1787826391.010882:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043059840:192.168.203.147@tcp:103:rm.0 00010000:00010000:0.0:1787826391.011012:0:163077:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0512c00/0xbef42c0221d6f46c lrc: 2/0,0 mode: PR/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ceef19 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826391.011031:0:163077:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842e00/0xbef42c0221d6f45e lrc: 2/0,0 mode: CR/CR res: [0x240001f59:0x16:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ceecb8 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000002:00100000:0.0:1787826391.011067:0:163077:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181c49e6680 x1874663043054080/t21474836527(21474836527) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 0 dl 1787826406 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'rm.0' uid:0 gid:0 projid:0 00000020:00100000:0.0:1787826391.011077:0:163077:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:0.0:1787826391.011083:0:163077:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c55a7480 x1874663043060096/t0(0) o35->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:0.0:1787826391.011098:0:163077:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043060096:192.168.203.147@tcp:35:rm.0 00000100:00100000:0.0:1787826391.011124:0:163077:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d01a56c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826391.019822:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043059840:192.168.203.147@tcp:103:rm.0 00000100:00100000:0.0:1787826391.019977:0:163077:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043060096:192.168.203.147@tcp:35:rm.0 00000002:00100000:0.0:1787826391.020021:0:163077:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181c49e6680 x1874663043054080/t21474836527(21474836527) o101->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 0 dl 1787826406 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'rm.0' uid:0 gid:0 projid:0 00010000:00010000:0.0:1787826391.020134:0:163077:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0512c00/0xbef42c0221d6f46c lrc: 1/0,0 mode: --/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ceef19 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826391.020145:0:163077:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842e00/0xbef42c0221d6f45e lrc: 1/0,0 mode: --/CR res: [0x240001f59:0x16:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ceecb8 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787826391.020152:0:163077:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:0.0:1787826391.020155:0:163077:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181e0512c00/0xbef42c0221d6f46c lrc: 0/0,0 mode: --/PR res: [0x240001f59:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ceef19 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826391.020176:0:163077:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0001-mdc-ffff9181c5620000 lock: ffff9181d1842e00/0xbef42c0221d6f45e lrc: 0/0,0 mode: --/CR res: [0x240001f59:0x16:0x0].0x0 bits 0x5/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ceecb8 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000020:00100000:0.0:1787826391.020201:0:163077:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787826391.020209:0:163077:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c55a7480 x1874663043060352/t0(0) o36->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 496/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826391.020238:0:163077:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043060352:192.168.203.147@tcp:36:rm.0 00000100:00100000:0.0:1787826391.020324:0:163077:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d01a56c0 going to sleep for 11 seconds 00000100:00100000:1.0:1787826391.023348:0:2382:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826391.023501:0:159722:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:0.0:1787826391.023536:0:159722:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049435136 00000100:00100000:0.0:1787826391.023548:0:159722:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049435136 00000100:00100000:0.0:1787826391.023618:0:159722:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 6 00000100:00100000:0.0:1787826391.023629:0:159722:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c343e450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:c0d7758a-4501-4a89-8ec1-407fef4f0c7f+5:187909:x1874663049435136:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826391.023652:0:159722:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5600/0xbef42c0221d6f442 lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceec79 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:0.0:1787826391.024113:0:159722:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826391.024123:0:159722:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c343e450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:c0d7758a-4501-4a89-8ec1-407fef4f0c7f+5:187909:x1874663049435136:12345-192.168.203.147@tcp:104: Request processed in 494us (785us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826391.024132:0:159722:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 6 00010000:00010000:2.0:1787826391.026453:0:159732:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1787826391.026479:0:159732:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5600/0xbef42c0221d6f442 lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00ceec79 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:2.0:1787826391.026497:0:159732:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b5600 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:2.0:1787826391.026503:0:159732:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5600/0xbef42c0221d6f442 lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x13/0x13 rrc: 4 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00ceec79 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:2.0:1787826391.026561:0:159732:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5600/0xbef42c0221d6f442 lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x13/0x13 rrc: 4 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00ceec79 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:2.0:1787826391.026574:0:159732:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5600/0xbef42c0221d6f442 lrc: 3/0,0 mode: PR/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x13/0x13 rrc: 4 type: IBT gid 0 flags: 0x408400000000 nid: local remote: 0x878a5eea00ceec79 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:2.0:1787826391.026614:0:159732:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f346 lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x11/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee572 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1787826391.026684:0:159732:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5600/0xbef42c0221d6f442 lrc: 2/0,0 mode: --/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x13/0x13 rrc: 4 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ceec79 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:2.0:1787826391.026691:0:159732:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:3.0:1787826391.027235:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663043060480:192.168.203.147@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1787826391.027383:0:159732:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5600/0xbef42c0221d6f442 lrc: 1/0,0 mode: --/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x13/0x13 rrc: 4 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ceec79 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:2.0:1787826391.027394:0:159732:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e01b5600/0xbef42c0221d6f442 lrc: 0/0,0 mode: --/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x13/0x13 rrc: 4 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00ceec79 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:2.0:1787826391.027414:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826391.027416:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826391.027417:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826391.034866:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663043060480:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1787826391.050044:0:163077:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043060352:192.168.203.147@tcp:36:rm.0 00000080:00200000:0.0:1787826391.050119:0:163077:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x240001f59:0x16:0x0](ffff9181d080f448) 00000080:00200008:0.0:1787826391.050202:0:163077:0:(file.c:479:ll_file_release()) START file d80b.replay-single:[0x200027cd1:0x4:0x0](ffff9181d0809148), flags 704000 00010000:00010000:0.0:1787826391.050258:0:163077:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843200/0xbef42c0221d6f450 lrc: 2/0,0 mode: CR/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceec95 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1787826391.050308:0:163077:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:0.0:1787826391.050329:0:163077:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826391.050341:0:163077:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 34 lru_type: 1 00010000:00010000:0.0:1787826391.050350:0:163077:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 34 lru_type: 1 00000080:00200000:0.0:1787826391.050357:0:163077:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826391.050363:0:163077:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:0.0:1787826391.050367:0:163077:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f393 00000080:00010000:0.0:1787826391.050375:0:163077:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f393 from it ffffb97502bafc10 00010000:00010000:0.0:1787826391.050379:0:163077:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 34 lru_type: 1 00010000:00010000:0.0:1787826391.050388:0:163077:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 35 lru_type: 1 00000080:00200000:0.0:1787826391.050404:0:163077:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826391.050416:0:163077:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80b.replay-single, flags=0 00000080:00200000:0.0:1787826391.050422:0:163077:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:0.0:1787826391.050428:0:163077:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826391.050434:0:163077:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 35 lru_type: 1 00010000:00010000:0.0:1787826391.050442:0:163077:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 35 lru_type: 1 00000080:00200000:0.0:1787826391.050453:0:163077:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826391.050467:0:163077:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:0.0:1787826391.050470:0:163077:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f393 00000080:00010000:0.0:1787826391.050474:0:163077:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f393 from it ffffb97502bafd88 00010000:00010000:0.0:1787826391.050477:0:163077:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 35 lru_type: 1 00010000:00010000:0.0:1787826391.050485:0:163077:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 36 lru_type: 1 00000080:00200000:0.0:1787826391.050492:0:163077:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 3 00000080:00200000:0.0:1787826391.050498:0:163077:0:(namei.c:2328:ll_rmdir()) VFS Op:name=d80b.replay-single, dir=[0x200000007:0x1:0x0](ffff9181d080cb08) 00010000:00010000:0.0:1787826391.050520:0:163077:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 36 lru_type: 1 00010000:00010000:0.0:1787826391.050548:0:163077:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843200/0xbef42c0221d6f450 lrc: 2/0,0 mode: CR/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ceec95 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000002:00100000:0.0:1787826391.050605:0:163077:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181c49e4000 x1874663043053696/t352187318297(352187318297) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 0 dl 1787826406 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'rm.0' uid:0 gid:0 projid:0 00000020:00100000:0.0:1787826391.050618:0:163077:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:0.0:1787826391.050623:0:163077:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c55a4000 x1874663043060736/t0(0) o35->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:0.0:1787826391.050650:0:163077:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4000 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043060736:192.168.203.147@tcp:35:rm.0 00000100:00100000:0.0:1787826391.050821:0:163077:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d01a56c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826391.054613:0:163077:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4000 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043060736:192.168.203.147@tcp:35:rm.0 00000002:00100000:0.0:1787826391.054680:0:163077:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181c49e4000 x1874663043053696/t352187318297(352187318297) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 0 dl 1787826406 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'rm.0' uid:0 gid:0 projid:0 00010000:00010000:0.0:1787826391.054813:0:163077:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f346 lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x11/0x0 rrc: 4 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00cee572 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787826391.054867:0:163077:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843200/0xbef42c0221d6f450 lrc: 1/0,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ceec95 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787826391.054876:0:163077:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f346 lrc: 1/0,0 mode: --/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cee572 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787826391.054884:0:163077:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 36 lru_type: 1 00010000:00010000:0.0:1787826391.054891:0:163077:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 3 locks packed 00010000:00010000:0.0:1787826391.054894:0:163077:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843200/0xbef42c0221d6f450 lrc: 0/0,0 mode: --/CR res: [0x200027cd1:0x4:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ceec95 expref: -99 pid: 163077 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1787826391.054907:0:163077:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f346 lrc: 0/0,0 mode: --/PR res: [0x200027cd1:0x4:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cee572 expref: -99 pid: 162075 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1787826391.054923:0:163077:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842a00/0xbef42c0221d6f393 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cee946 expref: -99 pid: 162580 timeout: 0 lvb_type: 0 lru_score: 36 lru_type: 1 00000020:00100000:0.0:1787826391.054945:0:163077:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787826391.054953:0:163077:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c55a4000 x1874663043060992/t0(0) o36->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 512/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826391.054984:0:163077:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4000 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043060992:192.168.203.147@tcp:36:rm.0 00000100:00100000:0.0:1787826391.055073:0:163077:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d01a56c0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826391.066141:0:163077:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4000 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163077:1874663043060992:192.168.203.147@tcp:36:rm.0 00000080:00200000:0.0:1787826391.066203:0:163077:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200027cd1:0x4:0x0](ffff9181d0809148) 00010000:00010000:3.0:1787826391.650835:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826392.674896:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826393.132899:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:f40147eb-9b53-47d4-b77c-99ceedd9664c:2390:1874663043061120:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826393.133303:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663043061248:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826393.133393:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043061376:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826393.133519:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663043061504:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826393.133599:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043061632:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826393.145802:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043061376:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826393.145828:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:f40147eb-9b53-47d4-b77c-99ceedd9664c:2390:1874663043061120:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826393.145888:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663043061248:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826393.157861:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043061632:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826393.157917:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663043061504:192.168.203.147@tcp:400:kworker.0 00010000:00010000:0.0:1787826393.711166:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826394.720915:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826395.755628:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826396.769793:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826397.793290:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826398.244248:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:f40147eb-9b53-47d4-b77c-99ceedd9664c:2392:1874663043061760:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826398.244341:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663043062016:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826398.244369:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663043062272:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826398.244400:0:2392:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1787826398.244417:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663043061888:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826398.244442:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663043062144:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826398.254009:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663043061888:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826398.254025:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663043062144:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826398.254033:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:f40147eb-9b53-47d4-b77c-99ceedd9664c:2392:1874663043061760:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826398.254041:0:2392:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff9181c5620000: committing for last_committed 21474836549 gen 1 00000100:00100000:2.0:1787826398.254198:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663043062016:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826398.254211:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663043062272:192.168.203.147@tcp:400:kworker.0 00000001:02000400:2.0:1787826398.592116:0:163644:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == replay-single test 80c: DNE: create remote dir, drop update rep from MDT1, fail MDT[0,1] ========================================================== 06:26:37 (1787826397) 00000080:00200000:0.0:1787826398.683450:0:163648:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:0.0:1787826398.683604:0:163648:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826398.683667:0:163648:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f473 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 163648 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826398.683680:0:163648:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f473 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163648 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826398.683690:0:163648:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f473 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163648 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826398.683715:0:163648:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163648:1874663043062400:192.168.203.147@tcp:101:lfs.0 00000100:00100000:0.0:1787826398.684531:0:163648:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d01a56c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826398.696825:0:163648:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163648:1874663043062400:192.168.203.147@tcp:101:lfs.0 00010000:00010000:2.0:1787826398.696863:0:163648:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f473 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceef7b expref: -99 pid: 163648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826398.696881:0:163648:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f473 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceef7b expref: -99 pid: 163648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826398.696897:0:163648:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f473 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceef7b expref: -99 pid: 163648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826398.696907:0:163648:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f473 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceef7b expref: -99 pid: 163648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826398.696918:0:163648:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f473 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceef7b expref: -99 pid: 163648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826398.696926:0:163648:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f473 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceef7b expref: -99 pid: 163648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826398.696934:0:163648:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f473 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceef7b expref: -99 pid: 163648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787826398.696945:0:163648:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c55a7b80 x1874663043062400/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826414 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1787826398.696958:0:163648:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f473 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceef7b expref: -99 pid: 163648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826398.696971:0:163648:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d1a77c00 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:2.0:1787826398.696978:0:163648:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:2.0:1787826398.697005:0:163648:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 69632 00000080:00010000:2.0:1787826398.697011:0:163648:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f473 00000080:00010000:2.0:1787826398.697016:0:163648:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:2.0:1787826398.697019:0:163648:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f473 from it ffffb97501d2bb58 00010000:00010000:2.0:1787826398.697023:0:163648:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f473 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceef7b expref: -99 pid: 163648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826398.697031:0:163648:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f473 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceef7b expref: -99 pid: 163648 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826398.697054:0:163648:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:2.0:1787826398.697076:0:163648:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=d80c.replay-single, dir=[0x200000007:0x1:0x0](ffff9181d080cb08), flags=13 00000080:00200000:2.0:1787826398.697081:0:163648:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=d80c.replay-single, dir=[0x200000007:0x1:0x0](ffff9181d080cb08), intent=getattr 00000002:00010000:2.0:1787826398.697101:0:163648:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: d80c.replay-single,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 01 00010000:00010000:2.0:1787826398.697140:0:163648:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843600/0xbef42c0221d6f47a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 163648 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826398.697147:0:163648:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843600/0xbef42c0221d6f47a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 163648 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826398.697155:0:163648:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843600/0xbef42c0221d6f47a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 163648 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826398.697173:0:163648:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163648:1874663043062528:192.168.203.147@tcp:101:lfs.0 00000100:00100000:2.0:1787826398.697240:0:163648:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5c94600 going to sleep for 11 seconds 00000100:00100000:3.0:1787826398.714798:0:163648:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163648:1874663043062528:192.168.203.147@tcp:101:lfs.0 00010000:00010000:3.0:1787826398.714820:0:163648:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843600/0xbef42c0221d6f47a lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 163648 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826398.714832:0:163648:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843600/0xbef42c0221d6f47a lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 163648 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826398.714839:0:163648:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843600/0xbef42c0221d6f47a lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 163648 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826398.714855:0:163648:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843600/0xbef42c0221d6f47a lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 163648 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826398.714863:0:163648:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843600/0xbef42c0221d6f47a lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 163648 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826398.714869:0:163648:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181d1843600 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:3.0:1787826398.714873:0:163648:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843600/0xbef42c0221d6f47a lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 163648 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826398.714879:0:163648:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843600/0xbef42c0221d6f47a lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 163648 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826398.714886:0:163648:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843600/0xbef42c0221d6f47a lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x20e400000000 pid: 163648 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826398.714894:0:163648:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826398.714900:0:163648:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843600/0xbef42c0221d6f47a lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 163648 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826398.714907:0:163648:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843600/0xbef42c0221d6f47a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 163648 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826398.714925:0:163648:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181e007ea00 x1874663043062528/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 592/584 e 0 to 0 dl 1787826414 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:3.0:1787826398.714939:0:163648:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00010000:00010000:3.0:1787826398.714968:0:163648:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f473 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceef7b expref: -99 pid: 163648 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826398.714980:0:163648:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f473 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceef7b expref: -99 pid: 163648 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00010000:3.0:1787826398.714988:0:163648:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f473 from it ffffb97501d2b930 00010000:00010000:3.0:1787826398.714992:0:163648:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f473 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceef7b expref: -99 pid: 163648 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826398.715000:0:163648:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f473 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceef7b expref: -99 pid: 163648 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:3.0:1787826398.715080:0:163648:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:3.0:1787826398.715103:0:163648:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826398.715110:0:163648:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f473 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceef7b expref: -99 pid: 163648 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826398.715118:0:163648:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f473 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceef7b expref: -99 pid: 163648 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:3.0:1787826398.715124:0:163648:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826398.715129:0:163648:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:3.0:1787826398.715132:0:163648:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f473 00000080:00010000:3.0:1787826398.715136:0:163648:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f473 from it ffffb97501d2bbe8 00010000:00010000:3.0:1787826398.715139:0:163648:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f473 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceef7b expref: -99 pid: 163648 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826398.715146:0:163648:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f473 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceef7b expref: -99 pid: 163648 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:3.0:1787826398.715165:0:163648:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826398.715173:0:163648:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80c.replay-single, flags=16392 00000080:00200008:3.0:1787826398.715195:0:163648:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181d080cb08), request_mask 0, flags 0, foreign 0 00000080:00200000:3.0:1787826398.715200:0:163648:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:3.0:1787826398.715207:0:163648:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826398.715213:0:163648:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f473 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceef7b expref: -99 pid: 163648 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:3.0:1787826398.715221:0:163648:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f473 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceef7b expref: -99 pid: 163648 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:3.0:1787826398.715227:0:163648:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826398.715230:0:163648:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:3.0:1787826398.715233:0:163648:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f473 00000080:00010000:3.0:1787826398.715236:0:163648:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f473 from it ffffb97501d2bcb8 00010000:00010000:3.0:1787826398.715239:0:163648:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f473 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceef7b expref: -99 pid: 163648 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:3.0:1787826398.715247:0:163648:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f473 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceef7b expref: -99 pid: 163648 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00000080:00200008:3.0:1787826398.715771:0:163648:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181d080cb08), request_mask 0, flags 0, foreign 0 00000080:00200000:3.0:1787826398.715777:0:163648:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:3.0:1787826398.715787:0:163648:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826398.715795:0:163648:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f473 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceef7b expref: -99 pid: 163648 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:3.0:1787826398.715805:0:163648:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f473 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceef7b expref: -99 pid: 163648 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00000080:00200000:3.0:1787826398.715811:0:163648:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826398.715815:0:163648:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:3.0:1787826398.715818:0:163648:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f473 00000080:00010000:3.0:1787826398.715821:0:163648:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f473 from it ffffb97501d2bcb8 00010000:00010000:3.0:1787826398.715824:0:163648:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f473 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceef7b expref: -99 pid: 163648 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:3.0:1787826398.715830:0:163648:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f473 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceef7b expref: -99 pid: 163648 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00000080:00200000:3.0:1787826398.715934:0:163648:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:3.0:1787826398.715942:0:163648:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826398.715950:0:163648:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f473 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceef7b expref: -99 pid: 163648 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00010000:00010000:3.0:1787826398.715959:0:163648:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f473 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceef7b expref: -99 pid: 163648 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00000080:00200000:3.0:1787826398.715969:0:163648:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826398.715973:0:163648:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:3.0:1787826398.715976:0:163648:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f473 00000080:00010000:3.0:1787826398.715980:0:163648:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f473 from it ffffb97501d2bbb0 00010000:00010000:3.0:1787826398.715983:0:163648:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f473 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceef7b expref: -99 pid: 163648 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00010000:00010000:3.0:1787826398.715990:0:163648:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f473 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceef7b expref: -99 pid: 163648 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00000080:00200000:3.0:1787826398.715998:0:163648:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 44 00000080:00200008:3.0:1787826398.716004:0:163648:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9181d080cb08), flags 300000 00000080:00200080:3.0:1787826398.716021:0:163648:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c03066f0 arg=7ffd0c368170 00000080:00200000:3.0:1787826398.716035:0:163648:0:(dir.c:451:ll_dir_setdirstripe()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) name=d80c.replay-single stripe_offset=0 stripe_count=1, hash_type=0 00010000:00010000:3.0:1787826398.716059:0:163648:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f473 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ceef7b expref: -99 pid: 163648 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00010000:00010000:3.0:1787826398.716109:0:163648:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f473 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ceef7b expref: -99 pid: 163648 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00010000:00010000:3.0:1787826398.716116:0:163648:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826398.716118:0:163648:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181e0513200/0xbef42c0221d6f473 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ceef7b expref: -99 pid: 163648 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00000020:00100000:3.0:1787826398.716145:0:163648:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:3.0:1787826398.716154:0:163648:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c5636300 x1874663043062784/t0(0) o36->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 568/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826398.716178:0:163648:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636300 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163648:1874663043062784:192.168.203.147@tcp:36:lfs.0 00000100:00100000:3.0:1787826398.716255:0:163648:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d225f6c0 going to sleep for 11 seconds 00000100:00100000:2.0:1787826398.722385:0:163648:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636300 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163648:1874663043062784:192.168.203.147@tcp:36:lfs.0 00000080:00200000:2.0:1787826398.722438:0:163648:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200027cd1:0x7:0x0](ffff9181c830db88) 00000080:00200000:2.0:1787826398.722445:0:163648:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200027cd1:0x7:0x0], updating i_size 4096 00000080:00200008:2.0:1787826398.722611:0:163648:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9181d080cb08), flags 300000 00010000:00010000:3.0:1787826398.818566:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826399.842312:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200000:2.0:1787826399.912101:0:163667:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:2.0:1787826399.912138:0:163667:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826399.912271:0:163667:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 163667 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826399.912283:0:163667:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163667 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826399.912294:0:163667:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163667 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826399.912319:0:163667:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163667:1874663043062912:192.168.203.147@tcp:101:lfs.0 00000100:00100000:2.0:1787826399.912418:0:163667:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5c94300 going to sleep for 11 seconds 00000100:00100000:3.0:1787826399.915593:0:163667:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163667:1874663043062912:192.168.203.147@tcp:101:lfs.0 00010000:00010000:3.0:1787826399.915622:0:163667:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceefb3 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826399.915634:0:163667:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceefb3 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826399.915645:0:163667:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceefb3 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826399.915658:0:163667:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceefb3 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826399.915671:0:163667:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceefb3 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826399.915679:0:163667:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceefb3 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826399.915687:0:163667:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceefb3 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826399.915705:0:163667:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181e007d500 x1874663043062912/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826415 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826399.915720:0:163667:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceefb3 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826399.915735:0:163667:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d1a77c00 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:3.0:1787826399.915741:0:163667:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:3.0:1787826399.915790:0:163667:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 69632 00000080:00010000:3.0:1787826399.915797:0:163667:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f481 00000080:00010000:3.0:1787826399.915803:0:163667:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:3.0:1787826399.915806:0:163667:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f481 from it ffffb9750149bb58 00010000:00010000:3.0:1787826399.915810:0:163667:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceefb3 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826399.915820:0:163667:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceefb3 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826399.915844:0:163667:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826399.915874:0:163667:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=d80c.replay-single, dir=[0x200000007:0x1:0x0](ffff9181d080cb08), flags=29 00000080:00200000:3.0:1787826399.915880:0:163667:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=d80c.replay-single, dir=[0x200000007:0x1:0x0](ffff9181d080cb08), intent=0 00000002:00010000:3.0:1787826399.915903:0:163667:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: d80c.replay-single,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: lookup flags 0 00010000:00010000:3.0:1787826399.915945:0:163667:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480000/0xbef42c0221d6f488 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 163667 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826399.915953:0:163667:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480000/0xbef42c0221d6f488 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 163667 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826399.915967:0:163667:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480000/0xbef42c0221d6f488 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 163667 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826399.916001:0:163667:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163667:1874663043063040:192.168.203.147@tcp:101:lfs.0 00000100:00100000:3.0:1787826399.916071:0:163667:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d225fa80 going to sleep for 11 seconds 00000100:00100000:3.0:1787826399.927680:0:163667:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163667:1874663043063040:192.168.203.147@tcp:101:lfs.0 00010000:00010000:3.0:1787826399.927711:0:163667:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480000/0xbef42c0221d6f488 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceefcf expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826399.927733:0:163667:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480000/0xbef42c0221d6f488 lrc: 4/1,0 mode: --/PR res: [0x200027cd1:0x7:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceefcf expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826399.927757:0:163667:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480000/0xbef42c0221d6f488 lrc: 4/1,0 mode: --/PR res: [0x200027cd1:0x7:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceefcf expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826399.927769:0:163667:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480000/0xbef42c0221d6f488 lrc: 4/1,0 mode: PR/PR res: [0x200027cd1:0x7:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceefcf expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826399.927781:0:163667:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480000/0xbef42c0221d6f488 lrc: 4/1,0 mode: PR/PR res: [0x200027cd1:0x7:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceefcf expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826399.927794:0:163667:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480000/0xbef42c0221d6f488 lrc: 5/2,0 mode: PR/PR res: [0x200027cd1:0x7:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceefcf expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826399.927803:0:163667:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480000/0xbef42c0221d6f488 lrc: 5/2,0 mode: PR/PR res: [0x200027cd1:0x7:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceefcf expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826399.927816:0:163667:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480000/0xbef42c0221d6f488 lrc: 4/1,0 mode: PR/PR res: [0x200027cd1:0x7:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceefcf expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826399.927828:0:163667:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x10 disposition=0xb, status=0 req@ffff9181c5634a80 x1874663043063040/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 592/688 e 0 to 0 dl 1787826415 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826399.927842:0:163667:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480000/0xbef42c0221d6f488 lrc: 3/1,0 mode: PR/PR res: [0x200027cd1:0x7:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceefcf expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826399.927853:0:163667:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d1a77c00 type 13 mode 4 res 8590097617/7 (0 0) 00000080:00200000:3.0:1787826399.927871:0:163667:0:(namei.c:952:ll_intent_lock()) intent lock 16 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:3.0:1787826399.927901:0:163667:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200027cd1:0x7:0x0], updating i_size 4096 00000080:00200000:3.0:1787826399.927905:0:163667:0:(namei.c:147:ll_iget()) got inode: [0x200027cd1:0x7:0x0](ffff9181c830db88): rc = 0 00000080:00010000:3.0:1787826399.927910:0:163667:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200027cd1:0x7:0x0](ffff9181c830db88) for lock 0xbef42c0221d6f488 00000080:00010000:3.0:1787826399.927922:0:163667:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200027cd1:0x7:0x0](ffff9181c830db88) 00000080:00010000:3.0:1787826399.927926:0:163667:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f488 from it ffffb9750149ba28 00010000:00010000:3.0:1787826399.927930:0:163667:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480000/0xbef42c0221d6f488 lrc: 3/1,0 mode: PR/PR res: [0x200027cd1:0x7:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceefcf expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826399.927939:0:163667:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480000/0xbef42c0221d6f488 lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x7:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceefcf expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826399.927968:0:163667:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x7:0x0](ffff9181c830db88), inode mode 41ed mask 1 00000080:00200000:3.0:1787826399.927982:0:163667:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=remote_dir, dir=[0x200027cd1:0x7:0x0](ffff9181c830db88), flags=13 00000080:00200000:3.0:1787826399.927987:0:163667:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=remote_dir, dir=[0x200027cd1:0x7:0x0](ffff9181c830db88), intent=getattr 00000002:00010000:3.0:1787826399.928007:0:163667:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: remote_dir,[0x0:0x0:0x0]) in obj [0x200027cd1:0x7:0x0], intent: getattr flags 01 00010000:00010000:3.0:1787826399.928063:0:163667:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481800/0xbef42c0221d6f48f lrc: 3/1,0 mode: --/CR res: [0x200027cd1:0x7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 163667 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826399.928071:0:163667:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481800/0xbef42c0221d6f48f lrc: 3/1,0 mode: --/CR res: [0x200027cd1:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 163667 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826399.928080:0:163667:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481800/0xbef42c0221d6f48f lrc: 3/1,0 mode: --/CR res: [0x200027cd1:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 163667 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826399.928101:0:163667:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163667:1874663043063168:192.168.203.147@tcp:101:lfs.0 00000100:00100000:3.0:1787826399.928176:0:163667:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d225fa80 going to sleep for 11 seconds 00000100:00100000:2.0:1787826399.945434:0:163667:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163667:1874663043063168:192.168.203.147@tcp:101:lfs.0 00010000:00010000:2.0:1787826399.945472:0:163667:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481800/0xbef42c0221d6f48f lrc: 4/1,0 mode: --/CR res: [0x200027cd1:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 163667 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826399.945488:0:163667:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481800/0xbef42c0221d6f48f lrc: 4/1,0 mode: --/CR res: [0x200027cd1:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 163667 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826399.945496:0:163667:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481800/0xbef42c0221d6f48f lrc: 4/1,0 mode: --/CR res: [0x200027cd1:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 163667 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826399.945504:0:163667:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481800/0xbef42c0221d6f48f lrc: 3/0,0 mode: --/CR res: [0x200027cd1:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 163667 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826399.945512:0:163667:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481800/0xbef42c0221d6f48f lrc: 4/0,0 mode: --/CR res: [0x200027cd1:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 163667 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826399.945520:0:163667:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3481800 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:2.0:1787826399.945525:0:163667:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481800/0xbef42c0221d6f48f lrc: 4/0,0 mode: --/CR res: [0x200027cd1:0x7:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 163667 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826399.945532:0:163667:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481800/0xbef42c0221d6f48f lrc: 4/0,0 mode: --/CR res: [0x200027cd1:0x7:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 163667 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826399.945541:0:163667:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481800/0xbef42c0221d6f48f lrc: 5/0,0 mode: --/CR res: [0x200027cd1:0x7:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x20e400000000 pid: 163667 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826399.945549:0:163667:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787826399.945556:0:163667:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481800/0xbef42c0221d6f48f lrc: 3/0,0 mode: --/CR res: [0x200027cd1:0x7:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 163667 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826399.945563:0:163667:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481800/0xbef42c0221d6f48f lrc: 0/0,0 mode: --/CR res: [0x200027cd1:0x7:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 163667 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1787826399.945581:0:163667:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181c5634a80 x1874663043063168/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 584/584 e 0 to 0 dl 1787826415 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1787826399.945597:0:163667:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200027cd1:0x7:0x0] suppgids 0 -1: rc 0 00010000:00010000:2.0:1787826399.945625:0:163667:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d1a77c00 type 13 mode 30 res 8590097617/7 (0 0) 00000080:00200000:2.0:1787826399.945693:0:163667:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:2.0:1787826399.945707:0:163667:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826399.945717:0:163667:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceefb3 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826399.945729:0:163667:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceefb3 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826399.945736:0:163667:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826399.945768:0:163667:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:2.0:1787826399.945772:0:163667:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f481 00000080:00010000:2.0:1787826399.945778:0:163667:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f481 from it ffffb9750149bbe8 00010000:00010000:2.0:1787826399.945781:0:163667:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceefb3 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826399.945795:0:163667:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceefb3 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1787826399.945810:0:163667:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:2.0:1787826399.945817:0:163667:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80c.replay-single, flags=16408 00000080:00200000:2.0:1787826399.945823:0:163667:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x7:0x0](ffff9181c830db88), inode mode 41ed mask 1 00000080:00200000:2.0:1787826399.945832:0:163667:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=remote_dir, dir=[0x200027cd1:0x7:0x0](ffff9181c830db88), flags=16392 00000080:00200000:2.0:1787826399.945836:0:163667:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=remote_dir, dir=[0x200027cd1:0x7:0x0](ffff9181c830db88), intent=getattr 00000002:00010000:2.0:1787826399.945848:0:163667:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: remote_dir,[0x0:0x0:0x0]) in obj [0x200027cd1:0x7:0x0], intent: getattr flags 01 00010000:00010000:2.0:1787826399.945887:0:163667:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842000/0xbef42c0221d6f496 lrc: 3/1,0 mode: --/CR res: [0x200027cd1:0x7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 163667 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826399.945894:0:163667:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842000/0xbef42c0221d6f496 lrc: 3/1,0 mode: --/CR res: [0x200027cd1:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 163667 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826399.945903:0:163667:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842000/0xbef42c0221d6f496 lrc: 3/1,0 mode: --/CR res: [0x200027cd1:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 163667 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826399.945922:0:163667:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163667:1874663043063296:192.168.203.147@tcp:101:lfs.0 00000100:00100000:2.0:1787826399.946011:0:163667:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5c94b40 going to sleep for 11 seconds 00000100:00100000:3.0:1787826399.960639:0:163667:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163667:1874663043063296:192.168.203.147@tcp:101:lfs.0 00010000:00010000:3.0:1787826399.960662:0:163667:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842000/0xbef42c0221d6f496 lrc: 4/1,0 mode: --/CR res: [0x200027cd1:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 163667 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826399.960677:0:163667:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842000/0xbef42c0221d6f496 lrc: 4/1,0 mode: --/CR res: [0x200027cd1:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 163667 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826399.960685:0:163667:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842000/0xbef42c0221d6f496 lrc: 4/1,0 mode: --/CR res: [0x200027cd1:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 163667 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826399.960693:0:163667:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842000/0xbef42c0221d6f496 lrc: 3/0,0 mode: --/CR res: [0x200027cd1:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 163667 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826399.960701:0:163667:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842000/0xbef42c0221d6f496 lrc: 4/0,0 mode: --/CR res: [0x200027cd1:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 163667 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826399.960708:0:163667:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181d1842000 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:3.0:1787826399.960712:0:163667:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842000/0xbef42c0221d6f496 lrc: 4/0,0 mode: --/CR res: [0x200027cd1:0x7:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 163667 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826399.960719:0:163667:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842000/0xbef42c0221d6f496 lrc: 4/0,0 mode: --/CR res: [0x200027cd1:0x7:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 163667 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826399.960726:0:163667:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842000/0xbef42c0221d6f496 lrc: 5/0,0 mode: --/CR res: [0x200027cd1:0x7:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x20e400000000 pid: 163667 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826399.960733:0:163667:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826399.960739:0:163667:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842000/0xbef42c0221d6f496 lrc: 3/0,0 mode: --/CR res: [0x200027cd1:0x7:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 163667 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826399.960842:0:163667:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1842000/0xbef42c0221d6f496 lrc: 0/0,0 mode: --/CR res: [0x200027cd1:0x7:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 163667 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826399.960859:0:163667:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9181e007d180 x1874663043063296/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 584/584 e 0 to 0 dl 1787826415 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:3.0:1787826399.960873:0:163667:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200027cd1:0x7:0x0] suppgids 0 -1: rc 0 00010000:00010000:3.0:1787826399.960901:0:163667:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d1a77c00 type 13 mode 30 res 8590097617/7 (0 0) 00000080:00200000:3.0:1787826399.960971:0:163667:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:3.0:1787826399.960987:0:163667:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826399.960996:0:163667:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceefb3 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826399.961007:0:163667:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceefb3 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:3.0:1787826399.961013:0:163667:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826399.961018:0:163667:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:3.0:1787826399.961021:0:163667:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f481 00000080:00010000:3.0:1787826399.961027:0:163667:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f481 from it ffffb9750149bb58 00010000:00010000:3.0:1787826399.961030:0:163667:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceefb3 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826399.961043:0:163667:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceefb3 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:3.0:1787826399.961056:0:163667:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826399.961063:0:163667:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80c.replay-single, flags=13 00000080:00200008:3.0:1787826399.961073:0:163667:0:(file.c:6296:ll_getattr_dentry()) START file d80c.replay-single:[0x200027cd1:0x7:0x0](ffff9181c830db88), request_mask 0, flags 0, foreign 0 00000080:00200000:3.0:1787826399.961077:0:163667:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200027cd1:0x7:0x0](ffff9181c830db88),name=d80c.replay-single 00000002:00010000:3.0:1787826399.961089:0:163667:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200027cd1:0x7:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826399.961094:0:163667:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d1a77c00 type 13 mode 30 res 8590097617/7 (0 0) 00010000:00010000:3.0:1787826399.961129:0:163667:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f49d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 163667 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826399.961137:0:163667:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f49d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 163667 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826399.961146:0:163667:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f49d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 163667 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826399.961165:0:163667:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163667:1874663043063424:192.168.203.147@tcp:101:lfs.0 00000100:00100000:3.0:1787826399.961238:0:163667:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d225f300 going to sleep for 11 seconds 00000100:00100000:3.0:1787826399.971202:0:163667:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163667:1874663043063424:192.168.203.147@tcp:101:lfs.0 00010000:00010000:3.0:1787826399.971230:0:163667:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f49d lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef007 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826399.971246:0:163667:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f49d lrc: 4/1,0 mode: --/PR res: [0x200027cd1:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef007 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826399.971259:0:163667:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f49d lrc: 4/1,0 mode: --/PR res: [0x200027cd1:0x7:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef007 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826399.971269:0:163667:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f49d lrc: 4/1,0 mode: PR/PR res: [0x200027cd1:0x7:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cef007 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826399.971279:0:163667:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f49d lrc: 4/1,0 mode: PR/PR res: [0x200027cd1:0x7:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef007 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826399.971290:0:163667:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f49d lrc: 5/2,0 mode: PR/PR res: [0x200027cd1:0x7:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cef007 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826399.971298:0:163667:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f49d lrc: 5/2,0 mode: PR/PR res: [0x200027cd1:0x7:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cef007 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826399.971305:0:163667:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f49d lrc: 4/1,0 mode: PR/PR res: [0x200027cd1:0x7:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cef007 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826399.971316:0:163667:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c5634000 x1874663043063424/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826415 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826399.971331:0:163667:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f49d lrc: 3/1,0 mode: PR/PR res: [0x200027cd1:0x7:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef007 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826399.971341:0:163667:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d1a77c00 type 13 mode 4 res 8590097617/7 (0 0) 00000080:00200000:3.0:1787826399.971347:0:163667:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:3.0:1787826399.971372:0:163667:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200027cd1:0x7:0x0], updating i_size 4096 00000080:00010000:3.0:1787826399.971378:0:163667:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200027cd1:0x7:0x0](ffff9181c830db88) for lock 0xbef42c0221d6f49d 00000080:00010000:3.0:1787826399.971385:0:163667:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200027cd1:0x7:0x0](ffff9181c830db88) 00000080:00010000:3.0:1787826399.971388:0:163667:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f49d from it ffffb9750149bcb8 00010000:00010000:3.0:1787826399.971392:0:163667:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f49d lrc: 3/1,0 mode: PR/PR res: [0x200027cd1:0x7:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cef007 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826399.971416:0:163667:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f49d lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x7:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef007 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:3.0:1787826399.971967:0:163667:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181d080cb08), request_mask 0, flags 0, foreign 0 00000080:00200000:3.0:1787826399.971973:0:163667:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:3.0:1787826399.971987:0:163667:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826399.971997:0:163667:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceefb3 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:3.0:1787826399.972008:0:163667:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceefb3 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:3.0:1787826399.972015:0:163667:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826399.972019:0:163667:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:3.0:1787826399.972022:0:163667:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f481 00000080:00010000:3.0:1787826399.972026:0:163667:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f481 from it ffffb9750149bcb8 00010000:00010000:3.0:1787826399.972029:0:163667:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceefb3 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:3.0:1787826399.972037:0:163667:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceefb3 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00000080:00200000:3.0:1787826399.972139:0:163667:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:3.0:1787826399.972147:0:163667:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826399.972154:0:163667:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceefb3 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:3.0:1787826399.972163:0:163667:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceefb3 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00000080:00200000:3.0:1787826399.972170:0:163667:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826399.972174:0:163667:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:3.0:1787826399.972177:0:163667:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f481 00000080:00010000:3.0:1787826399.972181:0:163667:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f481 from it ffffb9750149bbf8 00010000:00010000:3.0:1787826399.972184:0:163667:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceefb3 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:3.0:1787826399.972192:0:163667:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceefb3 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00000080:00200000:3.0:1787826399.972201:0:163667:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826399.972207:0:163667:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d80c.replay-single, flags=267 00000080:00200000:3.0:1787826399.972211:0:163667:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200027cd1:0x7:0x0](ffff9181c830db88), inode mode 41ed mask 44 00000080:00200008:3.0:1787826399.972217:0:163667:0:(file.c:1090:ll_file_open()) START file d80c.replay-single:[0x200027cd1:0x7:0x0](ffff9181c830db88), flags 300000 00000002:00010000:3.0:1787826399.972230:0:163667:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200027cd1:0x7:0x0]) in obj [0x200000007:0x1:0x0], intent: open flags 0240300001 00010000:00010000:3.0:1787826399.972273:0:163667:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4a4 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 163667 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826399.972280:0:163667:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4a4 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 163667 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:3.0:1787826399.972291:0:163667:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:3.0:1787826399.972298:0:163667:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c5634000 x1874663043063680/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/131856 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:3.0:1787826399.972308:0:163667:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4a4 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 163667 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826399.972326:0:163667:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163667:1874663043063680:192.168.203.147@tcp:101:lfs.0 00000100:00100000:3.0:1787826399.972493:0:163667:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d225f300 going to sleep for 11 seconds 00000100:00100000:3.0:1787826399.983811:0:163667:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163667:1874663043063680:192.168.203.147@tcp:101:lfs.0 00010000:00010000:3.0:1787826399.983836:0:163667:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4a4 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 163667 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826399.983851:0:163667:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4a4 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 163667 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826399.983858:0:163667:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4a4 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 163667 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826399.983866:0:163667:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4a4 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 163667 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826399.983875:0:163667:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4a4 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 163667 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826399.983882:0:163667:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3481000 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:3.0:1787826399.983887:0:163667:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4a4 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x206400000000 pid: 163667 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826399.983898:0:163667:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4a4 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x206400000000 pid: 163667 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826399.983905:0:163667:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4a4 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x20e400000000 pid: 163667 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826399.983911:0:163667:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826399.983917:0:163667:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4a4 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 163667 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826399.983922:0:163667:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4a4 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 163667 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826399.983939:0:163667:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x1 disposition=0x2b, status=0 req@ffff9181c5634000 x1874663043063680/t352187318304(352187318304) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826415 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00100000:3.0:1787826399.983951:0:163667:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c5634000 x1874663043063680/t352187318304(352187318304) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 648/664 e 0 to 0 dl 1787826415 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:3.0:1787826399.983966:0:163667:0:(namei.c:952:ll_intent_lock()) intent lock 1 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:3.0:1787826399.983993:0:163667:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200027cd1:0x7:0x0], updating i_size 4096 00000002:00100000:3.0:1787826399.984011:0:163667:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9181c5634000 x1874663043063680/t352187318304(352187318304) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 0 dl 1787826415 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200080:3.0:1787826399.984079:0:163667:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200027cd1:0x7:0x0](ffff9181c830db88) cmd=c03066f0 arg=7ffcb1c31960 00000080:00200000:3.0:1787826399.984094:0:163667:0:(dir.c:451:ll_dir_setdirstripe()) VFS Op:inode=[0x200027cd1:0x7:0x0](ffff9181c830db88) name=remote_dir stripe_offset=1 stripe_count=0, hash_type=0 00000020:00100000:3.0:1787826399.984149:0:163667:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0001-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:3.0:1787826399.984156:0:163667:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c5636680 x1874663043063936/t0(0) o36->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 560/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826399.984176:0:163667:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636680 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163667:1874663043063936:192.168.203.147@tcp:36:lfs.0 00000100:00100000:3.0:1787826399.984255:0:163667:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d225f300 going to sleep for 11 seconds 00000100:00100000:0.0:1787826399.999953:0:2381:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.147@tcp (source: 12345-192.168.203.147@tcp) 00000100:00100000:0.0:1787826399.999991:0:159722:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1874663049442560 00000100:00100000:0.0:1787826400.000009:0:159722:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1874663049442560 00000100:00100000:0.0:1787826400.000077:0:159722:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.147@tcp, seq: 7 00000100:00100000:0.0:1787826400.000087:0:159722:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9181c55a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:c0d7758a-4501-4a89-8ec1-407fef4f0c7f+5:187910:x1874663049442560:12345-192.168.203.147@tcp:104: 00010000:00010000:0.0:1787826400.000104:0:159722:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f49d lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x7:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cef007 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1787826400.000178:0:159722:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1787826400.000186:0:159722:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9181c55a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:c0d7758a-4501-4a89-8ec1-407fef4f0c7f+5:187910:x1874663049442560:12345-192.168.203.147@tcp:104: Request processed in 101us (244us total) trans 0 rc 0/0 00000100:00100000:0.0:1787826400.000193:0:159722:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.147@tcp, seq: 7 00010000:00010000:3.0:1787826400.009441:0:159732:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787826400.009458:0:159732:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f49d lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x7:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x878a5eea00cef007 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826400.009473:0:159732:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3480800 already unused, calling callback (ffffffffc1a233c0) 00010000:00010000:3.0:1787826400.009478:0:159732:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f49d lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x7:0x0].0x0 bits 0x13/0x13 rrc: 3 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00cef007 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826400.009488:0:159732:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f49d lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x7:0x0].0x0 bits 0x13/0x13 rrc: 3 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x878a5eea00cef007 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826400.009499:0:159732:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f49d lrc: 3/0,0 mode: PR/PR res: [0x200027cd1:0x7:0x0].0x0 bits 0x13/0x13 rrc: 3 type: IBT gid 0 flags: 0x408400000000 nid: local remote: 0x878a5eea00cef007 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826400.009529:0:159732:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480000/0xbef42c0221d6f488 lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x7:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceefcf expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826400.009574:0:159732:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f49d lrc: 2/0,0 mode: --/PR res: [0x200027cd1:0x7:0x0].0x0 bits 0x13/0x13 rrc: 3 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cef007 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826400.009581:0:159732:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1787826400.009603:0:159732:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f49d lrc: 1/0,0 mode: --/PR res: [0x200027cd1:0x7:0x0].0x0 bits 0x13/0x13 rrc: 3 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cef007 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826400.009610:0:159732:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f49d lrc: 0/0,0 mode: --/PR res: [0x200027cd1:0x7:0x0].0x0 bits 0x13/0x13 rrc: 3 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x878a5eea00cef007 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826400.009621:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826400.009624:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826400.009626:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826400.009921:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663043064064:192.168.203.147@tcp:103:ldlm_bl.0 00000100:00100000:2.0:1787826400.019023:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663043064064:192.168.203.147@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1787826400.868176:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200008:0.0:1787826400.888571:0:163676:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181d080cb08), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1787826400.888589:0:163676:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:0.0:1787826400.888621:0:163676:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826400.888645:0:163676:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceefb3 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00010000:00010000:0.0:1787826400.888665:0:163676:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceefb3 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00000080:00200000:0.0:1787826400.888674:0:163676:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826400.888684:0:163676:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:0.0:1787826400.888688:0:163676:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f481 00000080:00010000:0.0:1787826400.888698:0:163676:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f481 from it ffffb975015ebcb8 00010000:00010000:0.0:1787826400.888702:0:163676:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceefb3 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00010000:00010000:0.0:1787826400.888712:0:163676:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceefb3 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00000080:00200008:0.0:1787826400.889531:0:163676:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181d080cb08), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1787826400.889540:0:163676:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:0.0:1787826400.889553:0:163676:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826400.889565:0:163676:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceefb3 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00010000:00010000:0.0:1787826400.889584:0:163676:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceefb3 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00000080:00200000:0.0:1787826400.889591:0:163676:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826400.889598:0:163676:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:0.0:1787826400.889601:0:163676:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f481 00000080:00010000:0.0:1787826400.889606:0:163676:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f481 from it ffffb975015ebcb8 00010000:00010000:0.0:1787826400.889610:0:163676:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceefb3 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00010000:00010000:0.0:1787826400.889617:0:163676:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceefb3 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 0 00000080:00200000:0.0:1787826400.889791:0:163676:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:0.0:1787826400.889801:0:163676:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826400.889808:0:163676:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceefb3 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 0 00010000:00010000:0.0:1787826400.889818:0:163676:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceefb3 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 0 00000080:00200000:0.0:1787826400.889824:0:163676:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826400.889829:0:163676:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:0.0:1787826400.889831:0:163676:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f481 00000080:00010000:0.0:1787826400.889836:0:163676:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f481 from it ffffb975015ebbb0 00010000:00010000:0.0:1787826400.889839:0:163676:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceefb3 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 0 00010000:00010000:0.0:1787826400.889847:0:163676:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceefb3 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 0 00000080:00200000:0.0:1787826400.889855:0:163676:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 44 00000080:00200008:0.0:1787826400.889864:0:163676:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9181d080cb08), flags 100000 00000080:00200080:0.0:1787826400.896508:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:0.0:1787826400.896551:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181d1a2dc00 uarg=0000000000000000 00800000:00000004:0.0:1787826400.896564:0:163676:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9181c5620000: age 8461, max_age 8471 00000100:00100000:0.0:1787826400.896637:0:163676:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163676:1874663043064192:192.168.203.147@tcp:41:lfs.0 00000100:00100000:0.0:1787826400.896778:0:163676:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d01a53c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826400.902910:0:163676:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163676:1874663043064192:192.168.203.147@tcp:41:lfs.0 00800000:00000004:3.0:1787826400.902951:0:163676:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0000-mdc-ffff9181c5620000: update ffff9181c354d178 cache blocks 317214/353529 objects 1023642/1024000 00000080:00200080:3.0:1787826400.903069:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.903092:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00800000:00000004:3.0:1787826400.903102:0:163676:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9181c5620000: age 8461, max_age 8471 00000100:00100000:3.0:1787826400.903150:0:163676:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635880 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163676:1874663043064320:192.168.203.147@tcp:41:lfs.0 00000100:00100000:3.0:1787826400.903254:0:163676:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d225f0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826400.909873:0:163676:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff9181c5620000: committing for last_committed 21474836550 gen 1 00000100:00100000:3.0:1787826400.909898:0:163676:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635880 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163676:1874663043064320:192.168.203.147@tcp:41:lfs.0 00800000:00000004:3.0:1787826400.909932:0:163676:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0001-mdc-ffff9181c5620000: update ffff9181c3548450 cache blocks 317314/353529 objects 1023701/1024000 00000080:00200080:3.0:1787826400.910035:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910067:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910078:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910083:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910088:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910092:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910096:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910100:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910105:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910109:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910113:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910117:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910122:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910127:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910131:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910135:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910140:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910144:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910148:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910152:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910156:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910159:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910163:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910167:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910171:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910175:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910179:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910183:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910187:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910191:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910195:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910199:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910203:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910206:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910210:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910214:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910218:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910222:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910226:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910230:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910234:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910238:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910242:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910246:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910251:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910255:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910259:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910263:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910267:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910271:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910275:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910279:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910283:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910287:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910291:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910295:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910299:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910303:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910307:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910310:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910313:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910316:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910321:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910325:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910328:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910331:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910336:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910339:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910343:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910346:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910350:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910354:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910358:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910362:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910365:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910369:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910373:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910377:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910381:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910384:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910400:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910405:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910410:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910414:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910418:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910421:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910425:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910430:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910434:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910438:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910454:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910460:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910464:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910467:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910471:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910474:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910478:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910482:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910490:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910494:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910497:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910501:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910505:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910508:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910512:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910515:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910519:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910522:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910526:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910529:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910533:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910537:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910542:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910545:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910550:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910553:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910557:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910560:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910564:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910568:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910572:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910576:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910579:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910583:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910587:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00800000:00000080:3.0:1787826400.910590:0:163676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.910594:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.910602:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00020000:00000004:3.0:1787826400.910621:0:163676:0:(obd_class.h:1090:obd_statfs()) lustre-OST0000-osc-ffff9181c5620000: age 8461, max_age 8471 00000100:00100000:3.0:1787826400.910670:0:163676:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635880 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163676:1874663043064448:192.168.203.147@tcp:13:lfs.0 00000100:00100000:3.0:1787826400.910822:0:163676:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d225f0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826400.914620:0:163676:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635880 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163676:1874663043064448:192.168.203.147@tcp:13:lfs.0 00020000:00000004:3.0:1787826400.914662:0:163676:0:(obd_class.h:1116:obd_statfs()) lustre-OST0000-osc-ffff9181c5620000: update ffff9181c3549e08 cache blocks 901356/958279 objects 261711/262144 00000080:00200080:3.0:1787826400.914828:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.914854:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00020000:00000004:3.0:1787826400.914864:0:163676:0:(obd_class.h:1090:obd_statfs()) lustre-OST0001-osc-ffff9181c5620000: age 8461, max_age 8471 00000100:00100000:3.0:1787826400.914911:0:163676:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635880 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163676:1874663043064576:192.168.203.147@tcp:13:lfs.0 00000100:00100000:3.0:1787826400.915003:0:163676:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d225f0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826400.919263:0:163676:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635880 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163676:1874663043064576:192.168.203.147@tcp:13:lfs.0 00020000:00000004:3.0:1787826400.919300:0:163676:0:(obd_class.h:1116:obd_statfs()) lustre-OST0001-osc-ffff9181c5620000: update ffff9181c354b7c0 cache blocks 901360/958279 objects 261728/262144 00000080:00200080:3.0:1787826400.919382:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.919413:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.919422:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.919426:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.919430:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.919434:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.919437:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.919441:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.919445:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.919448:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.919452:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.919455:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.919458:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.919461:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.919464:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.919467:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.919470:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.919473:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.919475:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.919478:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.919485:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.919488:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.919491:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.919494:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.919497:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.919500:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.919504:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.919507:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.919510:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.919512:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.919515:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.919518:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.919520:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.919523:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.919526:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.919528:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.919531:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.919534:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.919537:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.919540:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.919543:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.919546:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.919549:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.919552:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.919555:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.919559:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.919562:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.919565:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.919568:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.919572:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.919575:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.919578:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.919581:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.919584:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.919587:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.919591:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.919594:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.919597:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.919600:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.919604:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.919607:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.919611:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.919614:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.919618:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.919623:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.919626:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.919629:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.919633:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.919636:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.919640:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.919644:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.919647:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.919651:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.919655:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.919659:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.919662:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.919666:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.919670:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.919673:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.919676:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.919680:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.919684:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.919688:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.919691:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.919695:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.919698:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.919702:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.919706:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.919710:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.919713:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.919716:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.919719:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.919722:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.919725:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.919729:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.919733:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.919737:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.919741:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.920240:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.920248:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.920252:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.920255:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.920259:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.920262:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.920265:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.920269:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.920273:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.920276:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.920280:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.920284:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.920287:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.920291:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.920295:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.920298:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.920301:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.920304:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.920307:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.920311:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.920314:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.920318:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.920322:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.920325:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.920329:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.920332:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200080:3.0:1787826400.920335:0:163676:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe4083c700 00020000:00000080:3.0:1787826400.920338:0:163676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c440ac00 uarg=0000000000000000 00000080:00200008:3.0:1787826400.920356:0:163676:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9181d080cb08), flags 100000 00000080:00200000:0.0:1787826400.994472:0:163683:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:0.0:1787826400.994514:0:163683:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826400.994538:0:163683:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceefb3 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 0 00010000:00010000:0.0:1787826400.994558:0:163683:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceefb3 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 0 00000080:00200000:0.0:1787826400.994567:0:163683:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826400.994577:0:163683:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:0.0:1787826400.994581:0:163683:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f481 00000080:00010000:0.0:1787826400.994590:0:163683:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f481 from it ffffb975015ebbd0 00010000:00010000:0.0:1787826400.994595:0:163683:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceefb3 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 0 00010000:00010000:0.0:1787826400.994606:0:163683:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceefb3 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00000080:00200000:0.0:1787826400.994627:0:163683:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:0.0:1787826400.994644:0:163683:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=fsa-oleg347-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9181d080cb08), flags=1536 00000080:00200000:0.0:1787826400.994648:0:163683:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:0.0:1787826400.994654:0:163683:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826400.994661:0:163683:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceefb3 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00010000:00010000:0.0:1787826400.994670:0:163683:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceefb3 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00000080:00200000:0.0:1787826400.994677:0:163683:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826400.994682:0:163683:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:0.0:1787826400.994684:0:163683:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f481 00000080:00010000:0.0:1787826400.994688:0:163683:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f481 from it ffffb975015ebcd0 00010000:00010000:0.0:1787826400.994691:0:163683:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceefb3 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00010000:00010000:0.0:1787826400.994700:0:163683:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceefb3 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00000080:00200000:0.0:1787826400.994709:0:163683:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 3 00000080:00200000:0.0:1787826400.994715:0:163683:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:0.0:1787826400.994721:0:163683:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1787826400.994727:0:163683:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceefb3 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00010000:00010000:0.0:1787826400.994915:0:163683:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceefb3 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00000080:00200000:0.0:1787826400.994925:0:163683:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1787826400.994930:0:163683:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:0.0:1787826400.994933:0:163683:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f481 00000080:00010000:0.0:1787826400.994937:0:163683:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f481 from it ffffb975015ebdb0 00010000:00010000:0.0:1787826400.994941:0:163683:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00ceefb3 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00010000:00010000:0.0:1787826400.994949:0:163683:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00ceefb3 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 11 lru_type: 1 00000080:00200000:0.0:1787826400.994959:0:163683:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 3 00000080:00200000:0.0:1787826400.994964:0:163683:0:(namei.c:2129:ll_create_nd()) VFS Op:name=fsa-oleg347-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9181d080cb08), flags=33188, excl=1 00000080:00200000:0.0:1787826400.994969:0:163683:0:(namei.c:2084:ll_mknod()) VFS Op:name=fsa-oleg347-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9181d080cb08) mode 100644 dev 0 00010000:00010000:0.0:1787826400.995203:0:163683:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ceefb3 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 11 lru_type: 1 00010000:00010000:0.0:1787826400.995272:0:163683:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ceefb3 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 11 lru_type: 1 00010000:00010000:0.0:1787826400.995279:0:163683:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1787826400.995282:0:163683:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181d1843000/0xbef42c0221d6f481 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ceefb3 expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 11 lru_type: 1 00000020:00100000:0.0:1787826400.995312:0:163683:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1787826400.995329:0:163683:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c55a6a00 x1874663043064832/t0(0) o36->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 528/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826400.995357:0:163683:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6a00 pname:cluuid:pid:xid:nid:opc:job mcreate:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163683:1874663043064832:192.168.203.147@tcp:36:mcreate.0 00000100:00100000:0.0:1787826400.995645:0:163683:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d01a5cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1787826401.000194:0:163683:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6a00 pname:cluuid:pid:xid:nid:opc:job mcreate:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163683:1874663043064832:192.168.203.147@tcp:36:mcreate.0 00000080:00200000:0.0:1787826401.000262:0:163683:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200027cd1:0x8:0x0](ffff9181c83f4b08) 00000080:00200000:0.0:1787826401.000315:0:163683:0:(namei.c:2137:ll_create_nd()) VFS Op:name=fsa-oleg347-client.virtnet, unhashed 1 00000080:00200000:3.0:1787826401.024475:0:163686:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:3.0:1787826401.024516:0:163686:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826401.024599:0:163686:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4ab lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 163686 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826401.024613:0:163686:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4ab lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163686 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826401.024626:0:163686:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4ab lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163686 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826401.024659:0:163686:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635880 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163686:1874663043064960:192.168.203.147@tcp:101:rm.0 00000100:00100000:3.0:1787826401.025268:0:163686:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d225f0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826401.028599:0:163686:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635880 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163686:1874663043064960:192.168.203.147@tcp:101:rm.0 00010000:00010000:3.0:1787826401.029616:0:163686:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4ab lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef0cb expref: -99 pid: 163686 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826401.029633:0:163686:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4ab lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef0cb expref: -99 pid: 163686 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826401.029647:0:163686:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4ab lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cef0cb expref: -99 pid: 163686 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826401.029661:0:163686:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4ab lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef0cb expref: -99 pid: 163686 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826401.029676:0:163686:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4ab lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cef0cb expref: -99 pid: 163686 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826401.029685:0:163686:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4ab lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cef0cb expref: -99 pid: 163686 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826401.029692:0:163686:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4ab lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cef0cb expref: -99 pid: 163686 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826401.029708:0:163686:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c5635880 x1874663043064960/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826417 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826401.029724:0:163686:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4ab lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef0cb expref: -99 pid: 163686 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826401.029739:0:163686:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d1a77c00 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:3.0:1787826401.030159:0:163686:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:3.0:1787826401.030205:0:163686:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 69632 00000080:00010000:3.0:1787826401.030214:0:163686:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f4ab 00000080:00010000:3.0:1787826401.030222:0:163686:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:3.0:1787826401.030226:0:163686:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f4ab from it ffffb97501d73b58 00010000:00010000:3.0:1787826401.030231:0:163686:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4ab lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cef0cb expref: -99 pid: 163686 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826401.030248:0:163686:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4ab lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef0cb expref: -99 pid: 163686 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826401.030280:0:163686:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826401.030306:0:163686:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=fsa-oleg347-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9181d080cb08), flags=8 00000080:00200000:3.0:1787826401.030312:0:163686:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=fsa-oleg347-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9181d080cb08), intent=getattr 00000002:00010000:3.0:1787826401.030342:0:163686:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: fsa-oleg347-client.virtnet,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 01 00010000:00010000:3.0:1787826401.030404:0:163686:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b2 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 163686 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826401.030413:0:163686:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b2 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 163686 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826401.030424:0:163686:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b2 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 163686 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826401.030645:0:163686:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635880 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163686:1874663043065088:192.168.203.147@tcp:101:rm.0 00000100:00100000:3.0:1787826401.030973:0:163686:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d225f0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826401.034503:0:163686:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635880 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163686:1874663043065088:192.168.203.147@tcp:101:rm.0 00010000:00010000:3.0:1787826401.034541:0:163686:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b2 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef0e7 expref: -99 pid: 163686 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826401.034569:0:163686:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b2 lrc: 4/1,0 mode: --/PR res: [0x200027cd1:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef0e7 expref: -99 pid: 163686 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826401.034578:0:163686:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b2 lrc: 4/1,0 mode: --/PR res: [0x200027cd1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef0e7 expref: -99 pid: 163686 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826401.034592:0:163686:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b2 lrc: 4/1,0 mode: PR/PR res: [0x200027cd1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cef0e7 expref: -99 pid: 163686 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826401.034605:0:163686:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b2 lrc: 4/1,0 mode: PR/PR res: [0x200027cd1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef0e7 expref: -99 pid: 163686 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826401.034631:0:163686:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b2 lrc: 5/2,0 mode: PR/PR res: [0x200027cd1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cef0e7 expref: -99 pid: 163686 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826401.034640:0:163686:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b2 lrc: 5/2,0 mode: PR/PR res: [0x200027cd1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cef0e7 expref: -99 pid: 163686 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826401.034647:0:163686:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b2 lrc: 4/1,0 mode: PR/PR res: [0x200027cd1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cef0e7 expref: -99 pid: 163686 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826401.034660:0:163686:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c5635880 x1874663043065088/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 600/584 e 0 to 0 dl 1787826417 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826401.034674:0:163686:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b2 lrc: 3/1,0 mode: PR/PR res: [0x200027cd1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef0e7 expref: -99 pid: 163686 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826401.034684:0:163686:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d1a77c00 type 13 mode 4 res 8590097617/8 (0 0) 00000080:00200000:3.0:1787826401.034692:0:163686:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:3.0:1787826401.034719:0:163686:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200027cd1:0x8:0x0], updating i_size 0 00000080:00200000:3.0:1787826401.034723:0:163686:0:(namei.c:147:ll_iget()) got inode: [0x200027cd1:0x8:0x0](ffff9181c83f4b08): rc = 0 00000080:00200000:3.0:1787826401.034769:0:163686:0:(file.c:6843:ll_layout_conf()) [0x200027cd1:0x8:0x0]: layout version change: 4294967294 -> 4294967295 00000080:00010000:3.0:1787826401.034777:0:163686:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200027cd1:0x8:0x0](ffff9181c83f4b08) for lock 0xbef42c0221d6f4b2 00000080:00010000:3.0:1787826401.034792:0:163686:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200027cd1:0x8:0x0](ffff9181c83f4b08) 00000080:00010000:3.0:1787826401.034796:0:163686:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f4b2 from it ffffb97501d73b20 00010000:00010000:3.0:1787826401.034799:0:163686:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b2 lrc: 3/1,0 mode: PR/PR res: [0x200027cd1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00cef0e7 expref: -99 pid: 163686 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826401.034808:0:163686:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b2 lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00cef0e7 expref: -99 pid: 163686 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:3.0:1787826401.035265:0:163686:0:(file.c:6296:ll_getattr_dentry()) START file fsa-oleg347-client.virtnet:[0x200027cd1:0x8:0x0](ffff9181c83f4b08), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1787826401.035275:0:163686:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200027cd1:0x8:0x0](ffff9181c83f4b08),name=fsa-oleg347-client.virtnet 00000002:00010000:3.0:1787826401.035305:0:163686:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200027cd1:0x8:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826401.035316:0:163686:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b2 lrc: 2/1,0 mode: PR/PR res: [0x200027cd1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00cef0e7 expref: -99 pid: 163686 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826401.035330:0:163686:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b2 lrc: 2/1,0 mode: PR/PR res: [0x200027cd1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00cef0e7 expref: -99 pid: 163686 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826401.035338:0:163686:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826401.035344:0:163686:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200027cd1:0x8:0x0](ffff9181c83f4b08) 00000080:00010000:3.0:1787826401.035348:0:163686:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200027cd1:0x8:0x0](ffff9181c83f4b08) for lock 0xbef42c0221d6f4b2 00000080:00010000:3.0:1787826401.035352:0:163686:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f4b2 from it ffffb97501d73cb8 00010000:00010000:3.0:1787826401.035355:0:163686:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b2 lrc: 3/1,0 mode: PR/PR res: [0x200027cd1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00cef0e7 expref: -99 pid: 163686 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826401.035364:0:163686:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b2 lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00cef0e7 expref: -99 pid: 163686 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:3.0:1787826401.035448:0:163686:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:3.0:1787826401.035457:0:163686:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826401.035464:0:163686:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4ab lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef0cb expref: -99 pid: 163686 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826401.035473:0:163686:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4ab lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef0cb expref: -99 pid: 163686 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826401.035479:0:163686:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826401.035484:0:163686:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:3.0:1787826401.035486:0:163686:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f4ab 00000080:00010000:3.0:1787826401.035490:0:163686:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f4ab from it ffffb97501d73c10 00010000:00010000:3.0:1787826401.035494:0:163686:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4ab lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cef0cb expref: -99 pid: 163686 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826401.035502:0:163686:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4ab lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef0cb expref: -99 pid: 163686 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:3.0:1787826401.035511:0:163686:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826401.035521:0:163686:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=fsa-oleg347-client.virtnet, flags=0 00000080:00200000:3.0:1787826401.035526:0:163686:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:3.0:1787826401.035533:0:163686:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826401.035538:0:163686:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4ab lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef0cb expref: -99 pid: 163686 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826401.035547:0:163686:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4ab lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef0cb expref: -99 pid: 163686 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:3.0:1787826401.035555:0:163686:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826401.035558:0:163686:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:3.0:1787826401.035562:0:163686:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f4ab 00000080:00010000:3.0:1787826401.035565:0:163686:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f4ab from it ffffb97501d73d68 00010000:00010000:3.0:1787826401.035569:0:163686:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4ab lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cef0cb expref: -99 pid: 163686 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826401.035576:0:163686:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4ab lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef0cb expref: -99 pid: 163686 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:3.0:1787826401.035585:0:163686:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 3 00000080:00200000:3.0:1787826401.035591:0:163686:0:(namei.c:2418:ll_unlink()) VFS Op:name=fsa-oleg347-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9181d080cb08) 00010000:00010000:3.0:1787826401.038374:0:163686:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4ab lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00cef0cb expref: -99 pid: 163686 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:3.0:1787826401.038576:0:163686:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b2 lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x878a5eea00cef0e7 expref: -99 pid: 163686 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:3.0:1787826401.038594:0:163686:0:(vvp_object.c:117:vvp_conf_set()) [0x200027cd1:0x8:0x0]: losing layout lock 00010000:00010000:3.0:1787826401.038659:0:163686:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b2 lrc: 1/0,0 mode: --/PR res: [0x200027cd1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cef0e7 expref: -99 pid: 163686 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826401.038667:0:163686:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4ab lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cef0cb expref: -99 pid: 163686 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:3.0:1787826401.038673:0:163686:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:3.0:1787826401.038677:0:163686:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b2 lrc: 0/0,0 mode: --/PR res: [0x200027cd1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cef0e7 expref: -99 pid: 163686 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826401.038766:0:163686:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4ab lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cef0cb expref: -99 pid: 163686 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000020:00100000:3.0:1787826401.038798:0:163686:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:3.0:1787826401.038813:0:163686:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c5635880 x1874663043065344/t0(0) o36->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 512/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826401.038844:0:163686:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635880 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163686:1874663043065344:192.168.203.147@tcp:36:rm.0 00000100:00100000:3.0:1787826401.038955:0:163686:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d225f0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826401.045137:0:163686:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635880 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163686:1874663043065344:192.168.203.147@tcp:36:rm.0 00000080:00200000:3.0:1787826401.045235:0:163686:0:(vvp_io.c:1826:vvp_io_init()) [0x200027cd1:0x8:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:3.0:1787826401.045249:0:163686:0:(vvp_io.c:304:vvp_io_fini()) [0x200027cd1:0x8:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826401.045269:0:163686:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200027cd1:0x8:0x0](ffff9181c83f4b08) 00010000:00010000:3.0:1787826401.888962:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826402.913015:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826403.361810:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:f40147eb-9b53-47d4-b77c-99ceedd9664c:2389:1874663043065472:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826403.361941:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663043065728:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826403.361974:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663043065984:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826403.362016:0:2389:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1787826403.362022:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663043065600:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826403.362045:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663043065856:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826403.368654:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663043065600:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826403.368678:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663043065856:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826403.368691:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:f40147eb-9b53-47d4-b77c-99ceedd9664c:2389:1874663043065472:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826403.368704:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663043065728:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826403.368716:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663043065984:192.168.203.147@tcp:400:kworker.0 00010000:00010000:3.0:1787826403.939428:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826404.960893:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826405.985172:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826407.010823:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826408.034406:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826408.486599:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663043066240:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826408.486705:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663043066496:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826408.486771:0:2391:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:2.0:1787826408.486779:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:f40147eb-9b53-47d4-b77c-99ceedd9664c:2391:1874663043066112:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826408.486811:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663043066368:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826408.486830:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663043066624:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826408.490146:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663043066496:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826408.492504:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663043066368:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826408.492531:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663043066240:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826408.500644:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:f40147eb-9b53-47d4-b77c-99ceedd9664c:2391:1874663043066112:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826408.500672:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663043066624:192.168.203.147@tcp:400:kworker.0 00000001:02000400:2.0:1787826408.528971:0:163759:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: local REPLAY BARRIER on lustre-MDT0000 00010000:00010000:3.0:1787826409.057531:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200008:3.0:1787826409.687051:0:163768:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181d080cb08), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1787826409.687070:0:163768:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:3.0:1787826409.687093:0:163768:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826409.687157:0:163768:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b9 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 163768 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826409.687167:0:163768:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b9 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163768 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826409.687180:0:163768:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b9 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163768 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826409.687200:0:163768:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635180 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163768:1874663043066752:192.168.203.147@tcp:101:lfs.0 00000100:00100000:3.0:1787826409.687274:0:163768:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d225f6c0 going to sleep for 50 seconds 00000100:00100000:3.0:1787826409.691511:0:163768:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635180 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163768:1874663043066752:192.168.203.147@tcp:101:lfs.0 00010000:00010000:3.0:1787826409.691538:0:163768:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b9 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef111 expref: -99 pid: 163768 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826409.691549:0:163768:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b9 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef111 expref: -99 pid: 163768 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826409.691560:0:163768:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b9 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cef111 expref: -99 pid: 163768 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826409.691571:0:163768:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b9 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef111 expref: -99 pid: 163768 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826409.691584:0:163768:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b9 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cef111 expref: -99 pid: 163768 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826409.691591:0:163768:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b9 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cef111 expref: -99 pid: 163768 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826409.691598:0:163768:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b9 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cef111 expref: -99 pid: 163768 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826409.691616:0:163768:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c5635180 x1874663043066752/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826464 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826409.691630:0:163768:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef111 expref: -99 pid: 163768 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826409.691642:0:163768:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d1a77c00 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:3.0:1787826409.691648:0:163768:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:3.0:1787826409.692630:0:163768:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 69632 00000080:00010000:3.0:1787826409.692643:0:163768:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f4b9 00000080:00010000:3.0:1787826409.692650:0:163768:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:3.0:1787826409.692654:0:163768:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f4b9 from it ffffb97501ce3cb8 00010000:00010000:3.0:1787826409.692658:0:163768:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cef111 expref: -99 pid: 163768 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826409.692668:0:163768:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b9 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef111 expref: -99 pid: 163768 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:2.0:1787826409.693437:0:163768:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9181d080cb08), request_mask 0, flags 0, foreign 0 00000080:00200000:2.0:1787826409.693446:0:163768:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:2.0:1787826409.693467:0:163768:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826409.693481:0:163768:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b9 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef111 expref: -99 pid: 163768 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826409.693494:0:163768:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b9 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef111 expref: -99 pid: 163768 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1787826409.693501:0:163768:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826409.693506:0:163768:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:2.0:1787826409.693509:0:163768:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f4b9 00000080:00010000:2.0:1787826409.693514:0:163768:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f4b9 from it ffffb97501ce3cb8 00010000:00010000:2.0:1787826409.693517:0:163768:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cef111 expref: -99 pid: 163768 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1787826409.693524:0:163768:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b9 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef111 expref: -99 pid: 163768 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1787826409.693656:0:163768:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:2.0:1787826409.693665:0:163768:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1787826409.693683:0:163768:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b9 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef111 expref: -99 pid: 163768 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787826409.693691:0:163768:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b9 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef111 expref: -99 pid: 163768 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1787826409.693697:0:163768:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1787826409.693703:0:163768:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:2.0:1787826409.693706:0:163768:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f4b9 00000080:00010000:2.0:1787826409.693710:0:163768:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f4b9 from it ffffb97501ce3bb0 00010000:00010000:2.0:1787826409.693714:0:163768:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cef111 expref: -99 pid: 163768 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1787826409.693723:0:163768:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b9 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef111 expref: -99 pid: 163768 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:2.0:1787826409.693731:0:163768:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 44 00000080:00200008:2.0:1787826409.693789:0:163768:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9181d080cb08), flags 100000 00000080:00200080:2.0:1787826409.693868:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:2.0:1787826409.693883:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0c00 uarg=0000000000000000 00800000:00000004:2.0:1787826409.693889:0:163768:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9181c5620000: age 8472, max_age 8480 00000100:00100000:2.0:1787826409.693931:0:163768:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163768:1874663043066880:192.168.203.147@tcp:41:lfs.0 00000100:00100000:2.0:1787826409.694265:0:163768:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5c94000 going to sleep for 50 seconds 00000100:00100000:2.0:1787826409.699116:0:163768:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163768:1874663043066880:192.168.203.147@tcp:41:lfs.0 00800000:00000004:2.0:1787826409.699146:0:163768:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0000-mdc-ffff9181c5620000: update ffff9181c354d178 cache blocks 317214/353529 objects 1023642/1024000 00000080:00200080:2.0:1787826409.699269:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:2.0:1787826409.699288:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df0c00 uarg=0000000000000000 00800000:00000004:2.0:1787826409.699295:0:163768:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9181c5620000: age 8472, max_age 8480 00000100:00100000:2.0:1787826409.699341:0:163768:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163768:1874663043067008:192.168.203.147@tcp:41:lfs.0 00000100:00100000:2.0:1787826409.699426:0:163768:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5c94000 going to sleep for 50 seconds 00000100:00100000:3.0:1787826409.702793:0:163768:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163768:1874663043067008:192.168.203.147@tcp:41:lfs.0 00800000:00000004:3.0:1787826409.702829:0:163768:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0001-mdc-ffff9181c5620000: update ffff9181c3548450 cache blocks 317314/353529 objects 1023701/1024000 00000080:00200080:3.0:1787826409.702907:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.702930:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.702939:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.702943:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.702948:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.702965:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.702972:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.702977:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.702983:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.702987:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.702993:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.702997:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703003:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703007:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703013:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703017:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703022:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703026:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703030:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703266:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703274:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703278:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703283:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703287:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703291:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703296:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703301:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703305:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703309:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703312:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703316:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703319:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703323:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703326:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703339:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703343:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703346:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703349:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703353:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703356:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703360:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703364:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703368:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703372:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703375:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703390:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703396:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703400:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703404:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703408:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703411:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703415:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703419:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703422:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703427:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703431:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703436:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703439:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703442:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703445:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703449:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703453:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703456:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703459:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703463:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703466:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703470:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703474:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703478:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703482:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703486:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703491:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703495:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703498:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703502:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703506:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703510:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703514:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703517:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703522:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703525:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703530:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703534:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703537:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703541:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703545:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703549:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703552:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703555:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703558:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703562:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703565:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703569:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703572:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703576:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703580:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703584:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703588:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703592:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703596:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703600:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703603:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703607:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703610:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703614:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703618:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703623:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703627:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703631:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703635:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703639:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703643:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703647:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703651:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703656:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703659:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703663:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703667:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703672:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703676:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703679:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703683:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703687:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703691:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703695:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00800000:00000080:3.0:1787826409.703710:0:163768:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.703716:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:3.0:1787826409.703722:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00020000:00000004:3.0:1787826409.703732:0:163768:0:(obd_class.h:1090:obd_statfs()) lustre-OST0000-osc-ffff9181c5620000: age 8472, max_age 8480 00000100:00100000:3.0:1787826409.703833:0:163768:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635180 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163768:1874663043067136:192.168.203.147@tcp:13:lfs.0 00000100:00100000:3.0:1787826409.703943:0:163768:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d225f6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826409.721397:0:163768:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635180 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163768:1874663043067136:192.168.203.147@tcp:13:lfs.0 00020000:00000004:3.0:1787826409.721433:0:163768:0:(obd_class.h:1116:obd_statfs()) lustre-OST0000-osc-ffff9181c5620000: update ffff9181c3549e08 cache blocks 901356/958279 objects 261720/262144 00000080:00200080:3.0:1787826409.721521:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:3.0:1787826409.721542:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00020000:00000004:3.0:1787826409.721551:0:163768:0:(obd_class.h:1090:obd_statfs()) lustre-OST0001-osc-ffff9181c5620000: age 8472, max_age 8480 00000100:00100000:3.0:1787826409.721594:0:163768:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635180 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163768:1874663043067264:192.168.203.147@tcp:13:lfs.0 00000100:00100000:3.0:1787826409.721684:0:163768:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d225f6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1787826409.728524:0:163768:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635180 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163768:1874663043067264:192.168.203.147@tcp:13:lfs.0 00020000:00000004:3.0:1787826409.728559:0:163768:0:(obd_class.h:1116:obd_statfs()) lustre-OST0001-osc-ffff9181c5620000: update ffff9181c354b7c0 cache blocks 901360/958279 objects 261737/262144 00000080:00200080:3.0:1787826409.728642:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:3.0:1787826409.728663:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.728670:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:3.0:1787826409.728674:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.728678:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:3.0:1787826409.728681:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.728685:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:3.0:1787826409.728688:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.728691:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:3.0:1787826409.728694:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.728706:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:3.0:1787826409.728710:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.728713:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:3.0:1787826409.728716:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.728719:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:3.0:1787826409.728724:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.728727:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:3.0:1787826409.728731:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.728734:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:3.0:1787826409.728737:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.728741:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:3.0:1787826409.728783:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.728787:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:3.0:1787826409.728790:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.728794:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:3.0:1787826409.728797:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.728800:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:3.0:1787826409.728803:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.728807:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:3.0:1787826409.728810:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.728813:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:3.0:1787826409.728817:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.728821:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:3.0:1787826409.728825:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.728829:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:3.0:1787826409.728833:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.728836:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:3.0:1787826409.728839:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.728843:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:3.0:1787826409.728846:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.728850:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:3.0:1787826409.728855:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.728859:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:3.0:1787826409.728862:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.728867:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:3.0:1787826409.728869:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.728874:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:3.0:1787826409.728877:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.728881:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:3.0:1787826409.728884:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.728888:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:3.0:1787826409.728891:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.728895:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:3.0:1787826409.728898:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.728902:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:3.0:1787826409.728905:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.728909:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:3.0:1787826409.728912:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.728916:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:3.0:1787826409.728919:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.728923:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:3.0:1787826409.728927:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.728931:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:3.0:1787826409.728935:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.728939:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:3.0:1787826409.728944:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.728948:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:3.0:1787826409.728952:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.728956:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:3.0:1787826409.728959:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.728962:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:3.0:1787826409.728966:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.728969:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:3.0:1787826409.728973:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.728977:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:3.0:1787826409.728980:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.728984:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:3.0:1787826409.728988:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.728992:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:3.0:1787826409.728996:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.729000:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:3.0:1787826409.729003:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.729008:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:3.0:1787826409.729011:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.729015:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:3.0:1787826409.729018:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.729022:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:3.0:1787826409.729025:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c4408800 uarg=0000000000000000 00000080:00200080:3.0:1787826409.729029:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:2.0:1787826409.729350:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2c00 uarg=0000000000000000 00000080:00200080:2.0:1787826409.729390:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:2.0:1787826409.729402:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2c00 uarg=0000000000000000 00000080:00200080:2.0:1787826409.729408:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:2.0:1787826409.729412:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2c00 uarg=0000000000000000 00000080:00200080:2.0:1787826409.729416:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:2.0:1787826409.729419:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2c00 uarg=0000000000000000 00000080:00200080:2.0:1787826409.729425:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:2.0:1787826409.729430:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2c00 uarg=0000000000000000 00000080:00200080:2.0:1787826409.729435:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:2.0:1787826409.729439:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2c00 uarg=0000000000000000 00000080:00200080:2.0:1787826409.729444:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:2.0:1787826409.729449:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2c00 uarg=0000000000000000 00000080:00200080:2.0:1787826409.729456:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:2.0:1787826409.729461:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2c00 uarg=0000000000000000 00000080:00200080:2.0:1787826409.729465:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:2.0:1787826409.729470:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2c00 uarg=0000000000000000 00000080:00200080:2.0:1787826409.729474:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:2.0:1787826409.729479:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2c00 uarg=0000000000000000 00000080:00200080:2.0:1787826409.729482:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:2.0:1787826409.729485:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2c00 uarg=0000000000000000 00000080:00200080:2.0:1787826409.729489:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:2.0:1787826409.729493:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2c00 uarg=0000000000000000 00000080:00200080:2.0:1787826409.729498:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:2.0:1787826409.729502:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2c00 uarg=0000000000000000 00000080:00200080:2.0:1787826409.729507:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:2.0:1787826409.729511:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2c00 uarg=0000000000000000 00000080:00200080:2.0:1787826409.729516:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:2.0:1787826409.729520:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2c00 uarg=0000000000000000 00000080:00200080:2.0:1787826409.729525:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:2.0:1787826409.729529:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2c00 uarg=0000000000000000 00000080:00200080:2.0:1787826409.729534:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:2.0:1787826409.729538:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2c00 uarg=0000000000000000 00000080:00200080:2.0:1787826409.729542:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:2.0:1787826409.729547:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2c00 uarg=0000000000000000 00000080:00200080:2.0:1787826409.729551:0:163768:0:(dir.c:2001:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08) cmd=c00866a4 arg=7ffe0dea2350 00020000:00000080:2.0:1787826409.729554:0:163768:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9181c5620000: cmd=c00866a4 len=592 karg=ffff9181c3df2c00 uarg=0000000000000000 00000080:00200008:2.0:1787826409.729572:0:163768:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9181d080cb08), flags 100000 00000080:00200000:1.0:1787826409.794192:0:163775:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:1.0:1787826409.794226:0:163775:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826409.794245:0:163775:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b9 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef111 expref: -99 pid: 163768 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787826409.794267:0:163775:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b9 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef111 expref: -99 pid: 163768 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1787826409.794276:0:163775:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826409.794285:0:163775:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:1.0:1787826409.794289:0:163775:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f4b9 00000080:00010000:1.0:1787826409.794296:0:163775:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f4b9 from it ffffb97501d5bbd0 00010000:00010000:1.0:1787826409.794299:0:163775:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cef111 expref: -99 pid: 163768 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1787826409.794309:0:163775:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b9 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef111 expref: -99 pid: 163768 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00000080:00200000:1.0:1787826409.794325:0:163775:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:1.0:1787826409.794351:0:163775:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=fsa-oleg347-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9181d080cb08), flags=1536 00000080:00200000:1.0:1787826409.794354:0:163775:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:1.0:1787826409.794360:0:163775:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826409.794367:0:163775:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b9 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef111 expref: -99 pid: 163768 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:1.0:1787826409.794375:0:163775:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b9 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef111 expref: -99 pid: 163768 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00000080:00200000:1.0:1787826409.794381:0:163775:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826409.794384:0:163775:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:1.0:1787826409.794387:0:163775:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f4b9 00000080:00010000:1.0:1787826409.794390:0:163775:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f4b9 from it ffffb97501d5bcd0 00010000:00010000:1.0:1787826409.794393:0:163775:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cef111 expref: -99 pid: 163768 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:1.0:1787826409.794400:0:163775:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b9 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef111 expref: -99 pid: 163768 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00000080:00200000:1.0:1787826409.794408:0:163775:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 3 00000080:00200000:1.0:1787826409.794413:0:163775:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:1.0:1787826409.794419:0:163775:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1787826409.794425:0:163775:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b9 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef111 expref: -99 pid: 163768 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00010000:00010000:1.0:1787826409.794432:0:163775:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b9 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef111 expref: -99 pid: 163768 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00000080:00200000:1.0:1787826409.794438:0:163775:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1787826409.794441:0:163775:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:1.0:1787826409.794444:0:163775:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f4b9 00000080:00010000:1.0:1787826409.794447:0:163775:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f4b9 from it ffffb97501d5bdb0 00010000:00010000:1.0:1787826409.794449:0:163775:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cef111 expref: -99 pid: 163768 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00010000:00010000:1.0:1787826409.794456:0:163775:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b9 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef111 expref: -99 pid: 163768 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00000080:00200000:1.0:1787826409.794463:0:163775:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 3 00000080:00200000:1.0:1787826409.794468:0:163775:0:(namei.c:2129:ll_create_nd()) VFS Op:name=fsa-oleg347-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9181d080cb08), flags=33188, excl=1 00000080:00200000:1.0:1787826409.794472:0:163775:0:(namei.c:2084:ll_mknod()) VFS Op:name=fsa-oleg347-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9181d080cb08) mode 100644 dev 0 00010000:00010000:1.0:1787826409.794495:0:163775:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b9 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00cef111 expref: -99 pid: 163768 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00010000:00010000:1.0:1787826409.794544:0:163775:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b9 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cef111 expref: -99 pid: 163768 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00010000:00010000:1.0:1787826409.794551:0:163775:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1787826409.794554:0:163775:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480800/0xbef42c0221d6f4b9 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cef111 expref: -99 pid: 163768 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00000020:00100000:1.0:1787826409.794579:0:163775:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1787826409.794593:0:163775:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c35b1880 x1874663043067520/t0(0) o36->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 528/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826409.794619:0:163775:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1880 pname:cluuid:pid:xid:nid:opc:job mcreate:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163775:1874663043067520:192.168.203.147@tcp:36:mcreate.0 00000100:00100000:1.0:1787826409.795972:0:163775:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215d0c0 going to sleep for 50 seconds 00000100:00100000:1.0:1787826409.812240:0:163775:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1880 pname:cluuid:pid:xid:nid:opc:job mcreate:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163775:1874663043067520:192.168.203.147@tcp:36:mcreate.0 00000080:00200000:1.0:1787826409.812293:0:163775:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200027cd1:0x9:0x0](ffff9181c83e4b08) 00000080:00200000:1.0:1787826409.812343:0:163775:0:(namei.c:2137:ll_create_nd()) VFS Op:name=fsa-oleg347-client.virtnet, unhashed 1 00000080:00200000:3.0:1787826409.864133:0:163778:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:3.0:1787826409.864168:0:163778:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826409.866388:0:163778:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4c0 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 163778 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826409.866409:0:163778:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4c0 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163778 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826409.866425:0:163778:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4c0 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 163778 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826409.866453:0:163778:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635180 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163778:1874663043067648:192.168.203.147@tcp:101:rm.0 00000100:00100000:3.0:1787826409.866555:0:163778:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d225f6c0 going to sleep for 50 seconds 00000100:00100000:3.0:1787826409.875270:0:163778:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635180 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163778:1874663043067648:192.168.203.147@tcp:101:rm.0 00010000:00010000:3.0:1787826409.875298:0:163778:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4c0 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef12d expref: -99 pid: 163778 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826409.875310:0:163778:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4c0 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef12d expref: -99 pid: 163778 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826409.875322:0:163778:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4c0 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cef12d expref: -99 pid: 163778 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826409.875341:0:163778:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4c0 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef12d expref: -99 pid: 163778 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826409.875354:0:163778:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4c0 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cef12d expref: -99 pid: 163778 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826409.875362:0:163778:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4c0 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cef12d expref: -99 pid: 163778 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826409.875372:0:163778:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4c0 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cef12d expref: -99 pid: 163778 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826409.875383:0:163778:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c5635180 x1874663043067648/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 576/688 e 0 to 0 dl 1787826464 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826409.875396:0:163778:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4c0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef12d expref: -99 pid: 163778 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826409.875408:0:163778:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d1a77c00 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:3.0:1787826409.875415:0:163778:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:3.0:1787826409.875442:0:163778:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 69632 00000080:00010000:3.0:1787826409.875448:0:163778:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f4c0 00000080:00010000:3.0:1787826409.875452:0:163778:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:3.0:1787826409.875455:0:163778:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f4c0 from it ffffb97501d83b58 00010000:00010000:3.0:1787826409.875458:0:163778:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4c0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cef12d expref: -99 pid: 163778 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826409.875467:0:163778:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4c0 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef12d expref: -99 pid: 163778 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826409.875498:0:163778:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826409.875515:0:163778:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=fsa-oleg347-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9181d080cb08), flags=8 00000080:00200000:3.0:1787826409.875519:0:163778:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=fsa-oleg347-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9181d080cb08), intent=getattr 00000002:00010000:3.0:1787826409.875538:0:163778:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: fsa-oleg347-client.virtnet,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 01 00010000:00010000:3.0:1787826409.875572:0:163778:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481c00/0xbef42c0221d6f4c7 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 163778 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826409.875579:0:163778:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481c00/0xbef42c0221d6f4c7 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 163778 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826409.875587:0:163778:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481c00/0xbef42c0221d6f4c7 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 163778 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826409.875604:0:163778:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635180 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163778:1874663043067776:192.168.203.147@tcp:101:rm.0 00000100:00100000:3.0:1787826409.875674:0:163778:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d225f6c0 going to sleep for 50 seconds 00000100:00100000:3.0:1787826409.880694:0:163778:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635180 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163778:1874663043067776:192.168.203.147@tcp:101:rm.0 00010000:00010000:3.0:1787826409.880725:0:163778:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481c00/0xbef42c0221d6f4c7 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef149 expref: -99 pid: 163778 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826409.880770:0:163778:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481c00/0xbef42c0221d6f4c7 lrc: 4/1,0 mode: --/PR res: [0x200027cd1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef149 expref: -99 pid: 163778 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826409.880780:0:163778:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481c00/0xbef42c0221d6f4c7 lrc: 4/1,0 mode: --/PR res: [0x200027cd1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef149 expref: -99 pid: 163778 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826409.880791:0:163778:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481c00/0xbef42c0221d6f4c7 lrc: 4/1,0 mode: PR/PR res: [0x200027cd1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cef149 expref: -99 pid: 163778 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826409.880802:0:163778:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481c00/0xbef42c0221d6f4c7 lrc: 4/1,0 mode: PR/PR res: [0x200027cd1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef149 expref: -99 pid: 163778 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826409.880815:0:163778:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481c00/0xbef42c0221d6f4c7 lrc: 5/2,0 mode: PR/PR res: [0x200027cd1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cef149 expref: -99 pid: 163778 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826409.880829:0:163778:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481c00/0xbef42c0221d6f4c7 lrc: 5/2,0 mode: PR/PR res: [0x200027cd1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cef149 expref: -99 pid: 163778 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826409.880837:0:163778:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481c00/0xbef42c0221d6f4c7 lrc: 4/1,0 mode: PR/PR res: [0x200027cd1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cef149 expref: -99 pid: 163778 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1787826409.880848:0:163778:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9181c5635180 x1874663043067776/t0(0) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 600/584 e 0 to 0 dl 1787826464 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1787826409.880860:0:163778:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481c00/0xbef42c0221d6f4c7 lrc: 3/1,0 mode: PR/PR res: [0x200027cd1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef149 expref: -99 pid: 163778 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826409.880871:0:163778:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9181d1a77c00 type 13 mode 4 res 8590097617/9 (0 0) 00000080:00200000:3.0:1787826409.880878:0:163778:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:3.0:1787826409.880900:0:163778:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200027cd1:0x9:0x0], updating i_size 0 00000080:00200000:3.0:1787826409.880903:0:163778:0:(namei.c:147:ll_iget()) got inode: [0x200027cd1:0x9:0x0](ffff9181c83e4b08): rc = 0 00000080:00200000:3.0:1787826409.880919:0:163778:0:(file.c:6843:ll_layout_conf()) [0x200027cd1:0x9:0x0]: layout version change: 4294967294 -> 4294967295 00000080:00010000:3.0:1787826409.880925:0:163778:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200027cd1:0x9:0x0](ffff9181c83e4b08) for lock 0xbef42c0221d6f4c7 00000080:00010000:3.0:1787826409.880937:0:163778:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200027cd1:0x9:0x0](ffff9181c83e4b08) 00000080:00010000:3.0:1787826409.880940:0:163778:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f4c7 from it ffffb97501d83b20 00010000:00010000:3.0:1787826409.880943:0:163778:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481c00/0xbef42c0221d6f4c7 lrc: 3/1,0 mode: PR/PR res: [0x200027cd1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00cef149 expref: -99 pid: 163778 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826409.880952:0:163778:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481c00/0xbef42c0221d6f4c7 lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00cef149 expref: -99 pid: 163778 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:3.0:1787826409.880988:0:163778:0:(file.c:6296:ll_getattr_dentry()) START file fsa-oleg347-client.virtnet:[0x200027cd1:0x9:0x0](ffff9181c83e4b08), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1787826409.880995:0:163778:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200027cd1:0x9:0x0](ffff9181c83e4b08),name=fsa-oleg347-client.virtnet 00000002:00010000:3.0:1787826409.881016:0:163778:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200027cd1:0x9:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826409.881026:0:163778:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481c00/0xbef42c0221d6f4c7 lrc: 2/1,0 mode: PR/PR res: [0x200027cd1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00cef149 expref: -99 pid: 163778 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826409.881036:0:163778:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481c00/0xbef42c0221d6f4c7 lrc: 2/1,0 mode: PR/PR res: [0x200027cd1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00cef149 expref: -99 pid: 163778 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826409.881042:0:163778:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826409.881047:0:163778:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200027cd1:0x9:0x0](ffff9181c83e4b08) 00000080:00010000:3.0:1787826409.881050:0:163778:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200027cd1:0x9:0x0](ffff9181c83e4b08) for lock 0xbef42c0221d6f4c7 00000080:00010000:3.0:1787826409.881137:0:163778:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f4c7 from it ffffb97501d83cb8 00010000:00010000:3.0:1787826409.881141:0:163778:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481c00/0xbef42c0221d6f4c7 lrc: 3/1,0 mode: PR/PR res: [0x200027cd1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x878a5eea00cef149 expref: -99 pid: 163778 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826409.881149:0:163778:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481c00/0xbef42c0221d6f4c7 lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x878a5eea00cef149 expref: -99 pid: 163778 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:3.0:1787826409.881203:0:163778:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:3.0:1787826409.881210:0:163778:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826409.881217:0:163778:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4c0 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef12d expref: -99 pid: 163778 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826409.881225:0:163778:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4c0 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef12d expref: -99 pid: 163778 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1787826409.881231:0:163778:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826409.881235:0:163778:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:3.0:1787826409.881238:0:163778:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f4c0 00000080:00010000:3.0:1787826409.881242:0:163778:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f4c0 from it ffffb97501d83c10 00010000:00010000:3.0:1787826409.881244:0:163778:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4c0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cef12d expref: -99 pid: 163778 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1787826409.881250:0:163778:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4c0 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef12d expref: -99 pid: 163778 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:3.0:1787826409.881258:0:163778:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 1 00000080:00200000:3.0:1787826409.881267:0:163778:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=fsa-oleg347-client.virtnet, flags=0 00000080:00200000:3.0:1787826409.881272:0:163778:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08),name=/ 00000002:00010000:3.0:1787826409.881277:0:163778:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1787826409.881282:0:163778:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4c0 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef12d expref: -99 pid: 163778 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826409.881289:0:163778:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4c0 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef12d expref: -99 pid: 163778 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:3.0:1787826409.881295:0:163778:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1787826409.881298:0:163778:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) 00000080:00010000:3.0:1787826409.881301:0:163778:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9181d080cb08) for lock 0xbef42c0221d6f4c0 00000080:00010000:3.0:1787826409.881304:0:163778:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xbef42c0221d6f4c0 from it ffffb97501d83d68 00010000:00010000:3.0:1787826409.881306:0:163778:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4c0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x878a5eea00cef12d expref: -99 pid: 163778 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826409.881313:0:163778:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4c0 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x878a5eea00cef12d expref: -99 pid: 163778 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:3.0:1787826409.881325:0:163778:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9181d080cb08), inode mode 41ed mask 3 00000080:00200000:3.0:1787826409.881341:0:163778:0:(namei.c:2418:ll_unlink()) VFS Op:name=fsa-oleg347-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9181d080cb08) 00010000:00010000:3.0:1787826409.881362:0:163778:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4c0 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00cef12d expref: -99 pid: 163778 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:3.0:1787826409.881384:0:163778:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481c00/0xbef42c0221d6f4c7 lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x878a5eea00cef149 expref: -99 pid: 163778 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:3.0:1787826409.881393:0:163778:0:(vvp_object.c:117:vvp_conf_set()) [0x200027cd1:0x9:0x0]: losing layout lock 00010000:00010000:3.0:1787826409.881427:0:163778:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481c00/0xbef42c0221d6f4c7 lrc: 1/0,0 mode: --/PR res: [0x200027cd1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cef149 expref: -99 pid: 163778 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826409.881435:0:163778:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4c0 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cef12d expref: -99 pid: 163778 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:3.0:1787826409.881442:0:163778:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:3.0:1787826409.881445:0:163778:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481c00/0xbef42c0221d6f4c7 lrc: 0/0,0 mode: --/PR res: [0x200027cd1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x878a5eea00cef149 expref: -99 pid: 163778 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787826409.881459:0:163778:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3481000/0xbef42c0221d6f4c0 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00cef12d expref: -99 pid: 163778 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000020:00100000:3.0:1787826409.881478:0:163778:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:3.0:1787826409.881485:0:163778:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c5635180 x1874663043068032/t0(0) o36->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 512/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826409.881507:0:163778:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635180 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163778:1874663043068032:192.168.203.147@tcp:36:rm.0 00000100:00100000:3.0:1787826409.881575:0:163778:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d225f6c0 going to sleep for 50 seconds 00000100:00100000:3.0:1787826409.891698:0:163778:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635180 pname:cluuid:pid:xid:nid:opc:job rm:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163778:1874663043068032:192.168.203.147@tcp:36:rm.0 00000080:00200000:3.0:1787826409.891782:0:163778:0:(vvp_io.c:1826:vvp_io_init()) [0x200027cd1:0x9:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:3.0:1787826409.891793:0:163778:0:(vvp_io.c:304:vvp_io_fini()) [0x200027cd1:0x9:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1787826409.891801:0:163778:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200027cd1:0x9:0x0](ffff9181c83e4b08) 00010000:00010000:3.0:1787826410.085938:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826411.041189:0:163667:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d225f300 going to sleep for 24 seconds 00010000:00010000:3.0:1787826411.113911:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826412.129035:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826413.155016:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826413.600957:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:f40147eb-9b53-47d4-b77c-99ceedd9664c:2391:1874663043068160:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826413.601053:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663043068416:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826413.601081:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663043068672:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826413.601110:0:2391:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1787826413.601116:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663043068288:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826413.601137:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663043068544:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826413.623421:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663043068288:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826413.623445:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663043068544:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826413.623457:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:f40147eb-9b53-47d4-b77c-99ceedd9664c:2391:1874663043068160:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826413.623470:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663043068416:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826413.623484:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663043068672:192.168.203.147@tcp:400:kworker.0 00010000:00010000:3.0:1787826414.177256:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826415.202069:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826416.225313:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826416.282921:0:163667:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636680 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163667:1874663043063936:192.168.203.147@tcp:36:lfs.0 00000080:00200000:3.0:1787826416.282976:0:163667:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x240001f59:0x2b:0x0](ffff9181d0809148) 00000080:00200000:3.0:1787826416.282985:0:163667:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x240001f59:0x2b:0x0], updating i_size 4096 00000080:00200008:3.0:1787826416.283019:0:163667:0:(file.c:479:ll_file_release()) START file d80c.replay-single:[0x200027cd1:0x7:0x0](ffff9181c830db88), flags 300000 00000002:00100000:3.0:1787826416.283078:0:163667:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9181c5634000 x1874663043063680/t352187318304(352187318304) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 0 dl 1787826415 ref 1 fl Complete:RPQU/604/ffffffff rc 301/-1 job:'lfs.0' uid:0 gid:0 projid:0 00000020:00100000:3.0:1787826416.283090:0:163667:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9181c5620000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:3.0:1787826416.283096:0:163667:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9181c5634a80 x1874663043068928/t0(0) o35->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:3.0:1787826416.283117:0:163667:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163667:1874663043068928:192.168.203.147@tcp:35:lfs.0 00000100:00100000:3.0:1787826416.283195:0:163667:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d225f300 going to sleep for 11 seconds 00000100:00100000:3.0:1787826416.290862:0:163667:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job lfs:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:163667:1874663043068928:192.168.203.147@tcp:35:lfs.0 00000002:00100000:3.0:1787826416.290901:0:163667:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9181c5634000 x1874663043063680/t352187318304(352187318304) o101->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/664 e 0 to 0 dl 1787826415 ref 1 fl Complete:RQU/604/ffffffff rc 301/-1 job:'lfs.0' uid:0 gid:0 projid:0 00000001:02000400:1.0:1787826417.096231:0:163854:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: local REPLAY BARRIER on lustre-MDT0001 00010000:00010000:3.0:1787826417.249411:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826418.280168:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826418.721408:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:f40147eb-9b53-47d4-b77c-99ceedd9664c:2392:1874663043069056:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826418.721521:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663043069312:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826418.721550:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663043069568:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826418.721583:0:2392:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1787826418.721590:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663043069184:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826418.721612:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663043069440:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826418.731179:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:f40147eb-9b53-47d4-b77c-99ceedd9664c:2392:1874663043069056:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826418.731848:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663043069184:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826418.731868:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663043069312:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826418.739415:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663043069440:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826418.739438:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663043069568:192.168.203.147@tcp:400:kworker.0 00010000:00010000:0.0:1787826419.302047:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826420.321741:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826421.347311:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826422.370187:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826423.393080:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826423.841285:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:f40147eb-9b53-47d4-b77c-99ceedd9664c:2391:1874663043069696:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826423.844725:0:2391:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787826423.844767:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663043069824:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826423.845430:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663043070080:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826423.845483:0:2392:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1787826423.845490:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663043069952:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826423.845518:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663043070208:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826423.855208:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663043070208:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826423.855244:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663043070080:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826423.863082:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663043069952:192.168.203.147@tcp:400:kworker.0 00000100:00080000:2.0:1787826423.863274:0:2391:0:(recover.c:205:ptlrpc_request_handle_notconn()) import lustre-MDT0000-mdc-ffff9181c5620000 of lustre-MDT0000_UUID@192.168.203.147@tcp abruptly disconnected: reconnecting 00000100:00080000:2.0:1787826423.863293:0:2391:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from FULL to DISCONN 00000100:02000400:2.0:1787826423.863298:0:2391:0:(import.c:182:ptlrpc_set_import_discon()) lustre-MDT0000-mdc-ffff9181c5620000: Connection to lustre-MDT0000 (at 192.168.203.147@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00080000:2.0:1787826423.863306:0:2391:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826423.863312:0:2391:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8374 00000100:00080000:2.0:1787826423.863324:0:2391:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826423.863338:0:2391:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826423.863374:0:2391:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c700 x1874663043070336/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826423.863408:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663043069824:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826423.863783:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043070336:192.168.203.147@tcp:38:ptlrpcd_01_00.0 00000100:00080000:1.0:1787826423.867512:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787826423.867530:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-19) 00000100:00100000:1.0:1787826423.867536:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043070336:192.168.203.147@tcp:38:ptlrpcd_01_00.0 00010000:00010000:3.0:1787826424.417080:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826425.441105:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826426.466989:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826427.490567:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826428.513205:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826428.961224:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:f40147eb-9b53-47d4-b77c-99ceedd9664c:2392:1874663043070464:192.168.203.147@tcp:400:kworker.0 00000100:00080000:3.0:1787826428.961456:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826428.961463:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787826428.961467:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826428.961475:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8495 00000100:00080000:3.0:1787826428.961484:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 5s 00000100:00080000:3.0:1787826428.961503:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826428.961519:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826428.961542:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043070592/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826428.961619:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043070592:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826428.967085:0:2392:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1787826428.967115:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663043070720:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826428.970560:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663043070976:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826428.970820:0:2391:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1787826428.970829:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663043070848:192.168.203.147@tcp:400:kworker.0 00000100:00080000:2.0:1787826428.979365:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826428.979384:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-19) 00000100:00100000:2.0:1787826428.979390:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043070592:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787826428.979523:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663043070720:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826428.986145:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663043070976:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826428.986188:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663043070848:192.168.203.147@tcp:400:kworker.0 00010000:00010000:0.0:1787826429.536940:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826430.561698:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826431.587437:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826432.609314:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826433.632992:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787826434.082423:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826434.082430:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787826434.082434:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826434.082446:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8500 00000100:00080000:3.0:1787826434.082459:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 10s 00000100:00080000:3.0:1787826434.082467:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826434.082480:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826434.082499:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635f80 x1874663043071232/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826434.082543:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc3800 lustre-OST0000_UUID: changing import state from FULL to CONNECTING 00000100:00080000:3.0:1787826434.082546:0:160087:0:(import.c:2101:ptlrpc_disconnect_and_idle_import()) lustre-OST0000-osc-ffff9181c5620000: disconnect after 25s idle 00000100:00080000:3.0:1787826434.082563:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc6000 lustre-OST0001_UUID: changing import state from FULL to CONNECTING 00000100:00080000:3.0:1787826434.082566:0:160087:0:(import.c:2101:ptlrpc_disconnect_and_idle_import()) lustre-OST0001-osc-ffff9181c5620000: disconnect after 25s idle 00000100:00100000:3.0:1787826434.082679:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:f40147eb-9b53-47d4-b77c-99ceedd9664c:2392:1874663043071104:192.168.203.147@tcp:400:kworker.0 00000100:00100000:3.0:1787826434.082822:0:2392:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1787826434.082829:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663043071360:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826434.083276:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043071232:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787826434.083319:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043071488:192.168.203.147@tcp:9:kworker.0 00000100:00100000:0.0:1787826434.083341:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043071616:192.168.203.147@tcp:9:kworker.0 00000100:00080000:1.0:1787826434.087698:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787826434.087714:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-19) 00000100:00100000:1.0:1787826434.087720:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043071232:192.168.203.147@tcp:38:kworker.0 00000100:00080000:1.0:1787826434.087770:0:2388:0:(import.c:2011:ptlrpc_disconnect_idle_interpret()) @@@ inflight=1, refcount=3: rc = 0 req@ffff9181c5634e00 x1874663043071488/t0(0) o9->lustre-OST0000-osc-ffff9181c5620000@192.168.203.147@tcp:28/4 lens 224/224 e 0 to 0 dl 1787826444 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787826434.087780:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc3800 lustre-OST0000_UUID: changing import state from CONNECTING to IDLE 00000100:00100000:1.0:1787826434.087784:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043071488:192.168.203.147@tcp:9:kworker.0 00000100:00080000:1.0:1787826434.087800:0:2388:0:(import.c:2011:ptlrpc_disconnect_idle_interpret()) @@@ inflight=1, refcount=3: rc = 0 req@ffff9181c5636a00 x1874663043071616/t0(0) o9->lustre-OST0001-osc-ffff9181c5620000@192.168.203.147@tcp:28/4 lens 224/224 e 0 to 0 dl 1787826444 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787826434.087807:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc6000 lustre-OST0001_UUID: changing import state from CONNECTING to IDLE 00000100:00100000:1.0:1787826434.087825:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043071616:192.168.203.147@tcp:9:kworker.0 00000100:00100000:3.0:1787826434.090836:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2392:1874663043071360:192.168.203.147@tcp:400:kworker.0 00010000:00010000:3.0:1787826434.656987:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826435.682204:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826436.705039:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826437.729940:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826438.752990:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000400:3.0:1787826439.137303:0:2391:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787826423/real 1787826423] req@ffff9181c5635500 x1874663043069696/t0(0) o400->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 224/224 e 0 to 1 dl 1787826439 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826439.137344:0:2391:0:(client.c:2547:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff9181c5635500 x1874663043069696/t0(0) o400->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 224/224 e 0 to 1 dl 1787826439 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826439.137355:0:2391:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from FULL to DISCONN 00000100:02020000:3.0:1787826439.137360:0:2391:0:(import.c:186:ptlrpc_set_import_discon()) MGC192.168.203.147@tcp: Connection to MGS (at 192.168.203.147@tcp) was lost; in progress operations using this service will fail 00000100:00080000:3.0:1787826439.137363:0:2391:0:(import.c:192:ptlrpc_set_import_discon()) import MGS@192.168.203.147@tcp for MGC192.168.203.147@tcp not replayable, auto-deactivating 00000100:00080000:3.0:1787826439.137366:0:2391:0:(import.c:138:ptlrpc_deactivate_import_nolock()) setting import MGS INVALID 00000100:00100000:3.0:1787826439.137371:0:2391:0:(client.c:3589:ptlrpc_abort_inflight()) @@@ inflight req@ffff9181c5635500 x1874663043069696/t0(0) o400->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 224/224 e 0 to 1 dl 1787826439 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826439.137380:0:2391:0:(client.c:3589:ptlrpc_abort_inflight()) @@@ inflight req@ffff9181c5634700 x1874663043070464/t0(0) o400->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 224/224 e 0 to 0 dl 1787826444 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826439.137390:0:2391:0:(client.c:3589:ptlrpc_abort_inflight()) @@@ inflight req@ffff9181c5634380 x1874663043071104/t0(0) o400->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 224/224 e 0 to 0 dl 1787826450 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 10000000:01000000:3.0:1787826439.137400:0:2391:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808001 00000100:00080000:3.0:1787826439.137404:0:2391:0:(pinger.c:193:ptlrpc_pinger_ir_down()) IR down 10000000:01000000:3.0:1787826439.137405:0:2391:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808002 00000100:00080000:3.0:1787826439.137407:0:2391:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00100000:3.0:1787826439.137448:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:f40147eb-9b53-47d4-b77c-99ceedd9664c:2391:1874663043069696:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826439.137460:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:f40147eb-9b53-47d4-b77c-99ceedd9664c:2392:1874663043071104:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826439.137470:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:f40147eb-9b53-47d4-b77c-99ceedd9664c:2392:1874663043070464:192.168.203.147@tcp:400:kworker.0 00000100:00080000:0.0:1787826439.137483:0:138695:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826439.137490:0:138695:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787826439.137494:0:138695:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826439.137503:0:138695:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8374 00000100:00080000:0.0:1787826439.137526:0:138695:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787826439.137568:0:138695:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6680 x1874663043071744/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787826439.137604:0:138695:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826439.137608:0:138695:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787826439.137611:0:138695:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826439.137615:0:138695:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8505 00000100:00080000:0.0:1787826439.137621:0:138695:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 15s 00000100:00080000:0.0:1787826439.137625:0:138695:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787826439.137636:0:138695:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826439.137660:0:138695:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7b80 x1874663043071872/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826439.137713:0:2389:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1787826439.137727:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663043072000:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826439.137819:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043071744:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787826439.137873:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043071872:192.168.203.147@tcp:38:kworker.0 00000100:00080000:2.0:1787826439.142320:0:2388:0:(import.c:1132:ptlrpc_connect_interpret()) MGC192.168.203.147@tcp: connect to target with instance 0 10000000:01000000:2.0:1787826439.142339:0:2388:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808005 00000100:02000400:2.0:1787826439.142348:0:2388:0:(import.c:1284:ptlrpc_connect_interpret()) Evicted from MGS (at 192.168.203.147@tcp) after server handle changed from 0x878a5eea00cedc3b to 0x878a5eea00cef2ae 00000100:00080000:2.0:1787826439.142358:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to EVICTED 00000100:00080000:2.0:1787826439.142364:0:2388:0:(import.c:1674:ptlrpc_import_recovery_state_machine()) evicted from MGS@192.168.203.147@tcp; invalidating 00000100:00100000:1.0:1787826439.142970:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2389:1874663043072000:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826439.143447:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043071744:192.168.203.147@tcp:250:kworker.0 00010000:00080000:1.0:1787826439.143477:0:2388:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffff9181c55a7b80 x1874663043071872/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/416 e 0 to 0 dl 1787826464 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787826439.143493:0:2388:0:(import.c:1132:ptlrpc_connect_interpret()) lustre-MDT0000-mdc-ffff9181c5620000: connect to target with instance 83 00000100:00080000:1.0:1787826439.143508:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to REPLAY 00000100:00080000:1.0:1787826439.143514:0:2388:0:(import.c:1700:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1787826439.143523:0:2388:0:(client.c:3125:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9181c5620000: committing for last_committed 352187318284 gen 1 00000100:00100000:1.0:1787826439.143532:0:2388:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff9181c55a6300 x1874663043029632/t352187318286(352187318286) o36->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 528/448 e 0 to 0 dl 1787826344 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787826439.143544:0:2388:0:(recover.c:53:ptlrpc_replay_next()) import ffff9181c5bc0000 from lustre-MDT0000_UUID committed 352187318284 last 0 00000100:00080000:1.0:1787826439.143551:0:2388:0:(client.c:3553:ptlrpc_replay_req()) @@@ REPLAY req@ffff9181c55a6300 x1874663043029632/t352187318286(352187318286) o36->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 528/448 e 0 to 0 dl 1787826344 ref 1 fl New:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787826439.143562:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043071872:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787826439.143592:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043029632:192.168.203.147@tcp:36:mcreate.0 00000100:00080000:2.0:1787826439.143699:0:164121:0:(import.c:1598:ptlrpc_invalidate_import_thread()) thread invalidate import MGC192.168.203.147@tcp to MGS@192.168.203.147@tcp 00000100:00100000:2.0:1787826439.143703:0:164121:0:(import.c:323:ptlrpc_invalidate_import()) Sleeping 20s for inflight to error out 10000000:01000000:2.0:1787826439.143706:0:164121:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808003 00010000:00010000:2.0:1787826439.143720:0:164121:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f235 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x110e400000000 nid: local remote: 0x878a5eea00cedc57 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1787826439.143730:0:164121:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f235 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x110f400000000 nid: local remote: 0x878a5eea00cedc57 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787826439.143738:0:164121:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:2.0:1787826439.143853:0:164121:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787826439.143863:0:164121:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f235 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x590f400000000 nid: local remote: 0x878a5eea00cedc57 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826439.143876:0:164121:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b5e00/0xbef42c0221d6f243 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0x878a5eea00cedc65 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1787826439.143883:0:164121:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5e00/0xbef42c0221d6f243 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0x878a5eea00cedc65 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787826439.143888:0:164121:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:2.0:1787826439.145029:0:164121:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787826439.145040:0:164121:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b5e00/0xbef42c0221d6f243 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0x878a5eea00cedc65 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826439.145062:0:164121:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f38c lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0x878a5eea00cee7f6 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1787826439.145070:0:164121:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f38c lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0x878a5eea00cee7f6 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787826439.145076:0:164121:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:2.0:1787826439.145088:0:164121:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787826439.145092:0:164121:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f38c lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0x878a5eea00cee7f6 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826439.145103:0:164121:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f23c lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0x878a5eea00cedc5e expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1787826439.145115:0:164121:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f23c lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0x878a5eea00cedc5e expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787826439.145121:0:164121:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:2.0:1787826439.145142:0:164121:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787826439.145145:0:164121:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f23c lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0x878a5eea00cedc5e expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787826439.145164:0:164121:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from EVICTED to RECOVER 00000100:00080000:2.0:1787826439.145169:0:164121:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from RECOVER to FULL 10000000:01000000:2.0:1787826439.145172:0:164121:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808004 00000100:00080000:2.0:1787826439.145180:0:164121:0:(pinger.c:186:ptlrpc_pinger_ir_up()) IR up 00000100:02000000:2.0:1787826439.145185:0:164121:0:(import.c:1743:ptlrpc_import_recovery_state_machine()) MGC192.168.203.147@tcp: Connection restored to 192.168.203.147@tcp (at 192.168.203.147@tcp) 00010000:00010000:3.0:1787826439.776915:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787826440.416857:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:1.0:1787826440.416864:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff9181c5620000 from 341 10000000:01000000:1.0:1787826440.416869:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:1.0:1787826440.416912:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f4ce lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826440.416923:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f4ce lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826440.416934:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f4ce lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826440.416961:0:159735:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043072128:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00100000:1.0:1787826440.417050:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215dc00 going to sleep for 11 seconds 00010000:00010000:3.0:1787826440.800891:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826441.824914:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826442.849195:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826443.873309:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787826444.321104:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level REPLAY/5 force 0 force_next 1 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826444.321111:0:111835:0:(pinger.c:247:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: REPLAY) 00000100:00100000:1.0:1787826444.321149:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043072384:192.168.203.147@tcp:400:kworker.0 00000100:00100000:0.0:1787826444.321214:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:f40147eb-9b53-47d4-b77c-99ceedd9664c:2389:1874663043072256:192.168.203.147@tcp:400:kworker.0 00010000:00010000:3.0:1787826444.896979:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826445.920929:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826446.945189:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826447.968939:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826448.993246:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787826449.440981:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level REPLAY/5 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826449.440991:0:111835:0:(pinger.c:247:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: REPLAY) 00000100:00100000:0.0:1787826449.441048:0:2389:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:f40147eb-9b53-47d4-b77c-99ceedd9664c:2389:1874663043072512:192.168.203.147@tcp:400:kworker.0 00000100:00100000:1.0:1787826449.441063:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043072640:192.168.203.147@tcp:400:kworker.0 00010000:00010000:3.0:1787826450.017200:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826451.040935:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826451.488928:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215dc00 going to sleep for 1 seconds 00010000:00010000:3.0:1787826452.064854:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826452.512836:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215dc00 going to sleep for 1 seconds 00010000:00010000:3.0:1787826453.088916:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826453.536847:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215dc00 going to sleep for 1 seconds 00010000:00010000:3.0:1787826454.112893:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826454.560894:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215dc00 going to sleep for 1 seconds 00000100:00080000:2.0:1787826454.561007:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level REPLAY/5 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826454.561013:0:111835:0:(pinger.c:247:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: REPLAY) 00000100:00100000:3.0:1787826454.561062:0:2392:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:f40147eb-9b53-47d4-b77c-99ceedd9664c:2392:1874663043072768:192.168.203.147@tcp:400:kworker.0 00000100:00100000:2.0:1787826454.561072:0:2391:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663043072896:192.168.203.147@tcp:400:kworker.0 00010000:00010000:3.0:1787826455.136942:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1787826455.584977:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215dc00 going to sleep for 1 seconds 00010000:00010000:3.0:1787826456.160915:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000400:2.0:1787826456.608872:0:159735:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787826440/real 1787826440] req@ffff9181c35b2d80 x1874663043072128/t0(0) o101->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 328/344 e 0 to 1 dl 1787826456 ref 2 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826456.608909:0:159735:0:(client.c:2547:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff9181c35b2d80 x1874663043072128/t0(0) o101->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 328/344 e 0 to 1 dl 1787826456 ref 2 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826456.608919:0:159735:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from FULL to DISCONN 00000100:02020000:2.0:1787826456.608924:0:159735:0:(import.c:186:ptlrpc_set_import_discon()) MGC192.168.203.147@tcp: Connection to MGS (at 192.168.203.147@tcp) was lost; in progress operations using this service will fail 00000100:00080000:2.0:1787826456.608927:0:159735:0:(import.c:192:ptlrpc_set_import_discon()) import MGS@192.168.203.147@tcp for MGC192.168.203.147@tcp not replayable, auto-deactivating 00000100:00080000:2.0:1787826456.608930:0:159735:0:(import.c:138:ptlrpc_deactivate_import_nolock()) setting import MGS INVALID 00000100:00100000:2.0:1787826456.608935:0:159735:0:(client.c:3589:ptlrpc_abort_inflight()) @@@ inflight req@ffff9181c35b2d80 x1874663043072128/t0(0) o101->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 328/344 e 0 to 1 dl 1787826456 ref 2 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826456.608944:0:159735:0:(client.c:3589:ptlrpc_abort_inflight()) @@@ inflight req@ffff9181c35b0a80 x1874663043072256/t0(0) o400->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 224/224 e 0 to 0 dl 1787826460 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826456.608964:0:159735:0:(client.c:3589:ptlrpc_abort_inflight()) @@@ inflight req@ffff9181c35b3480 x1874663043072512/t0(0) o400->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 224/224 e 0 to 0 dl 1787826465 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826456.608974:0:159735:0:(client.c:3589:ptlrpc_abort_inflight()) @@@ inflight req@ffff9181e007f800 x1874663043072768/t0(0) o400->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 224/224 e 0 to 0 dl 1787826470 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 10000000:01000000:2.0:1787826456.608994:0:159735:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808001 00000100:00080000:2.0:1787826456.608997:0:159735:0:(pinger.c:193:ptlrpc_pinger_ir_down()) IR down 10000000:01000000:2.0:1787826456.608999:0:159735:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808002 00000100:00080000:2.0:1787826456.609011:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00100000:2.0:1787826456.609030:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215dc00 going to sleep for 0 seconds 00000100:00100000:2.0:1787826456.609040:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043072128:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:2.0:1787826456.609058:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f4ce lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826456.609067:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:f40147eb-9b53-47d4-b77c-99ceedd9664c:2389:1874663043072512:192.168.203.147@tcp:400:kworker.0 00010000:00010000:2.0:1787826456.609071:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f4ce lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787826456.609073:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826456.609083:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787826456.609089:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00010000:00010000:2.0:1787826456.609089:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f4ce lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826456.609101:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f4ce lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787826456.609105:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8510 00000100:00100000:0.0:1787826456.609107:0:2389:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:f40147eb-9b53-47d4-b77c-99ceedd9664c:2389:1874663043072256:192.168.203.147@tcp:400:kworker.0 00010000:00010000:2.0:1787826456.609120:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f4ce lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787826456.609123:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:2.0:1787826456.609136:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4c00 already unused, calling callback (ffffffffc1273c20) 00000100:00100000:3.0:1787826456.609139:0:2392:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:f40147eb-9b53-47d4-b77c-99ceedd9664c:2392:1874663043072768:192.168.203.147@tcp:400:kworker.0 10000000:00010000:2.0:1787826456.609141:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f4ce lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826456.609154:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f4ce lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826456.609157:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0700 x1874663043073024/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 10000000:00010000:2.0:1787826456.609167:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f4ce lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787826456.609176:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:2.0:1787826456.609180:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787826456.609189:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f4ce lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787826456.609191:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level REPLAY/5 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826456.609197:0:111835:0:(pinger.c:247:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: REPLAY) 00000100:00100000:3.0:1787826456.609209:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043073024:192.168.203.147@tcp:250:kworker.0 00010000:00010000:2.0:1787826456.609211:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f4ce lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787826456.609239:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -5 00000100:00100000:1.0:1787826456.609250:0:2390:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1787826456.609260:0:2390:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043073152:192.168.203.147@tcp:400:kworker.0 10000000:01000000:2.0:1787826456.609271:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-cliir' failed (-5). 10000000:00000400:2.0:1787826456.609276:0:159735:0:(mgc_request.c:1900:mgc_process_log()) MGC192.168.203.147@tcp: IR log lustre-cliir failed, not fatal: rc = -5 10000000:01000000:2.0:1787826456.613304:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:2.0:1787826456.613311:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff9181c5620000 from 52 10000000:01000000:2.0:1787826456.613316:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:2.0:1787826456.613357:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1842a00/0xbef42c0221d6f4d5 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826456.613369:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181d1842a00/0xbef42c0221d6f4d5 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826456.613385:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181d1842a00/0xbef42c0221d6f4d5 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826456.613404:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5c94e40 going to sleep for 0 seconds 00000100:00100000:2.0:1787826456.613413:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043073280:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:2.0:1787826456.613425:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181d1842a00/0xbef42c0221d6f4d5 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826456.613433:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181d1842a00/0xbef42c0221d6f4d5 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826456.613440:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1842a00/0xbef42c0221d6f4d5 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826456.613446:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181d1842a00/0xbef42c0221d6f4d5 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826456.613454:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181d1842a00/0xbef42c0221d6f4d5 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826456.613460:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181d1842a00 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:2.0:1787826456.613462:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181d1842a00/0xbef42c0221d6f4d5 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826456.613469:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181d1842a00/0xbef42c0221d6f4d5 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1787826456.613475:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181d1842a00/0xbef42c0221d6f4d5 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787826456.613481:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:2.0:1787826456.613484:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787826456.613489:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181d1842a00/0xbef42c0221d6f4d5 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826456.613496:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181d1842a00/0xbef42c0221d6f4d5 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787826456.613513:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:2.0:1787826456.613516:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:CONNECTING 00010000:00010000:3.0:1787826457.185060:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826458.208975:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826459.234038:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826460.257165:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000400:0.0:1787826460.705270:0:2390:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787826444/real 1787826444] req@ffff9181c35b2a00 x1874663043072384/t0(0) o400->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 224/224 e 0 to 1 dl 1787826460 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826460.705308:0:2390:0:(client.c:2547:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff9181c35b2a00 x1874663043072384/t0(0) o400->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 224/224 e 0 to 1 dl 1787826460 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826460.705319:0:2390:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from FULL to DISCONN 00000100:02000400:0.0:1787826460.705324:0:2390:0:(import.c:182:ptlrpc_set_import_discon()) lustre-MDT0001-mdc-ffff9181c5620000: Connection to lustre-MDT0001 (at 192.168.203.147@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00080000:0.0:1787826460.705332:0:2390:0:(import.c:418:ptlrpc_pinger_force()) lustre-MDT0001_UUID: waking up pinger s:DISCONN 00000100:00100000:0.0:1787826460.705358:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043072384:192.168.203.147@tcp:400:kworker.0 00000100:00080000:2.0:1787826460.705487:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level CONNECTING/4 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826460.705493:0:111835:0:(pinger.c:247:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1787826460.705496:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level REPLAY/5 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826460.705500:0:111835:0:(pinger.c:247:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: REPLAY) 00000100:00080000:2.0:1787826460.705503:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826460.705506:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787826460.705510:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826460.705517:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8444 00000100:00080000:2.0:1787826460.705533:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826460.705547:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826460.705583:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f100 x1874663043073408/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826460.705677:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043073408:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787826461.280932:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787826461.728915:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level CONNECTING/4 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826461.728928:0:160087:0:(pinger.c:247:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1787826461.728931:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level REPLAY/5 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826461.728934:0:160087:0:(pinger.c:247:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: REPLAY) 00000100:00080000:1.0:1787826461.728937:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826461.728940:0:160087:0:(pinger.c:247:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1787826462.304978:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826463.328920:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826464.352926:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826465.377064:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787826465.760932:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level CONNECTING/4 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826465.760953:0:111835:0:(pinger.c:247:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1787826465.760955:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level REPLAY/5 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826465.760958:0:111835:0:(pinger.c:247:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: REPLAY) 00000100:00080000:2.0:1787826465.760960:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826465.760962:0:111835:0:(pinger.c:247:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00000400:1.0:1787826465.824870:0:2390:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787826449/real 1787826449] req@ffff9181c35b3800 x1874663043072640/t0(0) o400->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 224/224 e 0 to 1 dl 1787826465 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787826465.824905:0:2390:0:(client.c:2547:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff9181c35b3800 x1874663043072640/t0(0) o400->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 224/224 e 0 to 1 dl 1787826465 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787826465.824914:0:2390:0:(import.c:214:ptlrpc_set_import_discon()) mdc: import ffff9181c5bc2800 already not connected (conn 8, was 7): CONNECTING 00000100:00100000:1.0:1787826465.824924:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043072640:192.168.203.147@tcp:400:kworker.0 00010000:00010000:3.0:1787826466.401693:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826466.848911:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787826456/real 1787826456] req@ffff9181c35b0700 x1874663043073024/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826466 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826466.848956:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0700 x1874663043073024/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826466 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826466.848972:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826466.849003:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826466.849021:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043073024:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787826466.849045:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826466.849051:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787826466.849053:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826466.849059:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8528 00000100:00080000:2.0:1787826466.849070:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 5s 00000100:00080000:2.0:1787826466.849077:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787826466.849104:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d880 x1874663043073536/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787826466.849125:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level REPLAY/5 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826466.849129:0:111835:0:(pinger.c:247:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: REPLAY) 00000100:00080000:2.0:1787826466.849131:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826466.849134:0:111835:0:(pinger.c:247:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1787826466.849160:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043073536:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1787826467.425000:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826468.449250:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826469.473413:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000400:3.0:1787826469.920890:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787826439/real 1787826439] req@ffff9181c55a6300 x1874663043029632/t352187318286(352187318286) o36->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 528/448 e 0 to 1 dl 1787826469 ref 2 fl Rpc:XQr/204/ffffffff rc 0/-1 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826469.920941:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=REPLAY (now=REPLAY) req@ffff9181c55a6300 x1874663043029632/t352187318286(352187318286) o36->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 528/448 e 0 to 1 dl 1787826469 ref 2 fl Rpc:XQU/204/ffffffff rc 0/-1 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00020000:3.0:1787826469.920960:0:2388:0:(client.c:3398:ptlrpc_replay_interpret()) @@@ request replay timed out req@ffff9181c55a6300 x1874663043029632/t352187318286(352187318286) o36->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 528/448 e 0 to 1 dl 1787826469 ref 2 fl Interpret:EXQU/204/ffffffff rc -110/-1 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826469.931659:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from REPLAY to CONNECTING 00000100:00080000:3.0:1787826469.931674:0:2388:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8510 00000100:00080000:3.0:1787826469.931690:0:2388:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826469.931703:0:2388:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826469.931740:0:2388:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043073664/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826469.931785:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043029632:192.168.203.147@tcp:36:mcreate.0 00000100:00100000:3.0:1787826469.931806:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043073664:192.168.203.147@tcp:38:ptlrpcd_rcv.0 00010000:00010000:3.0:1787826470.497110:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826470.944849:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787826460/real 1787826460] req@ffff9181e007f100 x1874663043073408/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826470 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826470.944880:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f100 x1874663043073408/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826470 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826470.944896:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826470.944934:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00000400:2.0:1787826470.944941:0:2391:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787826454/real 1787826454] req@ffff9181e007f480 x1874663043072896/t0(0) o400->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 224/224 e 0 to 1 dl 1787826470 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826470.944942:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043073408:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826470.944969:0:2391:0:(client.c:2547:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff9181e007f480 x1874663043072896/t0(0) o400->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 224/224 e 0 to 1 dl 1787826470 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826470.944978:0:2391:0:(import.c:214:ptlrpc_set_import_discon()) mdc: import ffff9181c5bc2800 already not connected (conn 8, was 7): DISCONN 00000100:00100000:2.0:1787826470.944988:0:2391:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2391:1874663043072896:192.168.203.147@tcp:400:kworker.0 00000100:00080000:1.0:1787826470.945052:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826470.945057:0:111835:0:(pinger.c:247:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1787826470.945060:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826470.945063:0:111835:0:(pinger.c:247:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1787826470.945065:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826470.945068:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787826470.945071:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826470.945077:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8532 00000100:00080000:1.0:1787826470.945087:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 5s 00000100:00080000:1.0:1787826470.945093:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787826470.945104:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787826470.945133:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3800 x1874663043073792/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826470.945189:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043073792:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787826471.520983:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787826471.969122:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826471.969134:0:111835:0:(pinger.c:247:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1787826471.969137:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826471.969143:0:111835:0:(pinger.c:247:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1787826471.969146:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826471.969149:0:111835:0:(pinger.c:247:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1787826472.544902:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000400:0.0:1787826472.992866:0:2390:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787826456/real 1787826456] req@ffff9181c35b1f80 x1874663043073152/t0(0) o400->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 224/224 e 0 to 1 dl 1787826472 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826472.992916:0:2390:0:(client.c:2547:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff9181c35b1f80 x1874663043073152/t0(0) o400->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 224/224 e 0 to 1 dl 1787826472 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826472.992927:0:2390:0:(import.c:214:ptlrpc_set_import_discon()) mdc: import ffff9181c5bc2800 already not connected (conn 9, was 7): CONNECTING 00000100:00100000:0.0:1787826472.992937:0:2390:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2390:1874663043073152:192.168.203.147@tcp:400:kworker.0 00010000:00010000:3.0:1787826473.569155:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826474.592905:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826475.616949:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787826476.000924:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826476.000935:0:111835:0:(pinger.c:247:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1787826476.000938:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826476.000942:0:111835:0:(pinger.c:247:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1787826476.000944:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826476.000947:0:111835:0:(pinger.c:247:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1787826476.641045:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787826477.089012:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-client' failed (-108). 10000000:00020000:1.0:1787826477.089021:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:1.0:1787826477.097771:0:159735:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:1.0:1787826477.097778:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff9181c5620000 from 7 10000000:01000000:1.0:1787826477.097783:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00010000:00010000:1.0:1787826477.097846:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5a00/0xbef42c0221d6f4dc lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826477.097856:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b5a00/0xbef42c0221d6f4dc lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826477.097888:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b5a00/0xbef42c0221d6f4dc lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826477.097976:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215dd80 going to sleep for 0 seconds 00000100:00100000:1.0:1787826477.097992:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043073920:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787826477.098014:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5a00/0xbef42c0221d6f4dc lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826477.098023:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b5a00/0xbef42c0221d6f4dc lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826477.098031:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5a00/0xbef42c0221d6f4dc lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826477.098045:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b5a00/0xbef42c0221d6f4dc lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826477.098053:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b5a00/0xbef42c0221d6f4dc lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826477.098060:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b5a00 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787826477.098064:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5a00/0xbef42c0221d6f4dc lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826477.098073:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b5a00/0xbef42c0221d6f4dc lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787826477.098080:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5a00/0xbef42c0221d6f4dc lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826477.098085:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:1.0:1787826477.098089:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787826477.098097:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b5a00/0xbef42c0221d6f4dc lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826477.098103:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b5a00/0xbef42c0221d6f4dc lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826477.098132:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787826477.098135:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:CONNECTING 00010000:00010000:3.0:1787826477.664926:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826478.689495:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826479.137005:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787826469/real 1787826469] req@ffff9181c5635500 x1874663043073664/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826479 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'ptlrpcd_rcv.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826479.137062:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043073664/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826479 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'ptlrpcd_rcv.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826479.137088:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826479.137114:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826479.137120:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043073664:192.168.203.147@tcp:38:ptlrpcd_rcv.0 00000100:00080000:2.0:1787826479.137424:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level CONNECTING/4 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826479.137432:0:111835:0:(pinger.c:247:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1787826479.137434:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826479.137437:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787826479.137439:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826479.137447:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8541 00000100:00080000:2.0:1787826479.137455:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 5s 00000100:00080000:2.0:1787826479.137461:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826479.137470:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826479.137496:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f480 x1874663043074048/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787826479.137522:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826479.137526:0:111835:0:(pinger.c:247:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1787826479.137578:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043074048:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787826479.712964:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826480.736969:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826481.184964:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787826466/real 1787826466] req@ffff9181e007d880 x1874663043073536/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826481 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826481.184993:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d880 x1874663043073536/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826481 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826481.185012:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826481.185039:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826481.185054:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043073536:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787826481.185217:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826481.185222:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787826481.185224:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826481.185230:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8538 00000100:00080000:1.0:1787826481.185239:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 10s 00000100:00080000:1.0:1787826481.185245:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787826481.185272:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3b80 x1874663043074176/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787826481.185292:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826481.185296:0:160087:0:(pinger.c:247:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1787826481.185299:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826481.185302:0:160087:0:(pinger.c:247:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1787826481.185354:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043074176:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1787826481.761080:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826482.784922:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826483.809385:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787826484.192936:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826484.192948:0:160087:0:(pinger.c:247:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1787826484.192952:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826484.192956:0:160087:0:(pinger.c:247:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1787826484.192959:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826484.192962:0:160087:0:(pinger.c:247:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1787826484.833204:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826485.280989:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787826470/real 1787826470] req@ffff9181c35b3800 x1874663043073792/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826485 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826485.281031:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3800 x1874663043073792/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826485 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826485.281054:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826485.281084:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826485.281089:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043073792:192.168.203.147@tcp:38:kworker.0 00000100:00080000:1.0:1787826485.281266:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826485.281272:0:160087:0:(pinger.c:247:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1787826485.281275:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826485.281278:0:160087:0:(pinger.c:247:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1787826485.281281:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826485.281284:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787826485.281288:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826485.281295:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8542 00000100:00080000:1.0:1787826485.281307:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 10s 00000100:00080000:1.0:1787826485.281316:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787826485.281330:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787826485.281365:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3800 x1874663043074304/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826485.282979:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043074304:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787826485.857111:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826486.881012:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826487.905020:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826488.928996:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826489.952908:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787826490.400994:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826490.401006:0:111835:0:(pinger.c:247:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1787826490.401009:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826490.401012:0:111835:0:(pinger.c:247:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1787826490.401015:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826490.401018:0:111835:0:(pinger.c:247:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1787826490.977001:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826492.000876:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826493.025271:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826494.049203:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826494.497091:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787826479/real 1787826479] req@ffff9181e007f480 x1874663043074048/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826494 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787826494.497125:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f480 x1874663043074048/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826494 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787826494.497147:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787826494.497177:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787826494.497182:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043074048:192.168.203.147@tcp:38:kworker.0 00000100:00080000:2.0:1787826494.497306:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826494.497311:0:160087:0:(pinger.c:247:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1787826494.497314:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826494.497317:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787826494.497321:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826494.497327:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8550 00000100:00080000:2.0:1787826494.497339:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 10s 00000100:00080000:2.0:1787826494.497347:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826494.497359:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826494.497389:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ce00 x1874663043074432/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826494.497804:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043074432:192.168.203.147@tcp:38:kworker.0 00000100:00080000:2.0:1787826494.497960:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826494.497965:0:160087:0:(pinger.c:247:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1787826495.073651:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826496.096998:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826497.121056:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:2.0:1787826497.569393:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'params' failed (-108). 10000000:00020000:2.0:1787826497.569406:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:2.0:1787826497.577297:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:2.0:1787826497.577306:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff9181c87b19b8 from 1 10000000:01000000:2.0:1787826497.577312:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00010000:00010000:2.0:1787826497.577353:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1842000/0xbef42c0221d6f4e3 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826497.577362:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181d1842000/0xbef42c0221d6f4e3 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826497.577370:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181d1842000/0xbef42c0221d6f4e3 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826497.577385:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5c94e40 going to sleep for 0 seconds 00000100:00100000:2.0:1787826497.577397:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043074560:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:2.0:1787826497.577412:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181d1842000/0xbef42c0221d6f4e3 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826497.577420:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181d1842000/0xbef42c0221d6f4e3 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826497.577427:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1842000/0xbef42c0221d6f4e3 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826497.577434:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181d1842000/0xbef42c0221d6f4e3 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826497.577448:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181d1842000/0xbef42c0221d6f4e3 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826497.577455:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181d1842000 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:2.0:1787826497.577458:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181d1842000/0xbef42c0221d6f4e3 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826497.577464:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181d1842000/0xbef42c0221d6f4e3 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1787826497.577469:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181d1842000/0xbef42c0221d6f4e3 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787826497.577472:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:2.0:1787826497.577474:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787826497.577478:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181d1842000/0xbef42c0221d6f4e3 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787826497.577482:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181d1842000/0xbef42c0221d6f4e3 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787826497.577499:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:2.0:1787826497.577501:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:CONNECTING 00010000:00010000:2.0:1787826498.145052:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000800:00000100:3.0:1787826498.656937:0:2380:0:(socklnd_cb.c:2254:ksocknal_find_timed_out_conn()) A connection with 12345-192.168.203.147@tcp (192.168.203.147:988) timed out; the network or node may be down. 00010000:00010000:3.0:1787826499.169454:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787826499.617116:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level CONNECTING/4 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826499.617127:0:160087:0:(pinger.c:247:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1787826499.617131:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826499.617134:0:160087:0:(pinger.c:247:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1787826499.617137:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826499.617140:0:160087:0:(pinger.c:247:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:2.0:1787826500.193512:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826501.217906:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826501.664901:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787826481/real 1787826481] req@ffff9181c35b3b80 x1874663043074176/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826501 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787826501.664952:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3b80 x1874663043074176/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826501 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787826501.664977:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787826501.665003:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787826501.665019:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043074176:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787826501.665049:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826501.665056:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787826501.665060:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826501.665067:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8552 00000100:00080000:3.0:1787826501.665077:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 15s 00000100:00080000:3.0:1787826501.665085:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787826501.665128:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043074688/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826501.665156:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826501.665160:0:160087:0:(pinger.c:247:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1787826501.665162:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826501.665165:0:160087:0:(pinger.c:247:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1787826501.665197:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043074688:192.168.203.147@tcp:250:kworker.0 00010000:00010000:2.0:1787826502.241000:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1787826502.689072:0:2377:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1021 -> 192.168.203.147:988 00000400:00000100:2.0:1787826502.689117:0:2377:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1787826502.689132:0:2377:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:1.0:1787826502.689238:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826501/real 1787826502] req@ffff9181c5635500 x1874663043074688/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826526 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787826502.689262:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043074688/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826526 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787826502.689278:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787826502.689282:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787826502.689335:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043074688:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1787826503.264906:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826504.290165:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826505.312909:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1787826505.760886:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787826485/real 1787826485] req@ffff9181c35b3800 x1874663043074304/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826505 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787826505.760921:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3800 x1874663043074304/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826505 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787826505.760938:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787826505.760956:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787826505.760958:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043074304:192.168.203.147@tcp:38:kworker.0 00000100:00080000:3.0:1787826505.761115:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826505.761120:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787826505.761123:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826505.761130:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8573 00000100:00080000:3.0:1787826505.761138:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826505.761145:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787826505.761194:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043074816/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826505.761230:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826505.761233:0:160087:0:(pinger.c:247:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1787826505.761236:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826505.761238:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787826505.761240:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826505.761244:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8557 00000100:00080000:3.0:1787826505.761249:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 15s 00000100:00080000:3.0:1787826505.761253:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826505.761262:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826505.761274:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043074944/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826505.761342:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043074816:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787826505.761443:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043074944:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787826506.336990:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826507.360886:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826508.384869:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1787826508.834080:0:2378:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1021 -> 192.168.203.147:988 00000400:00000100:2.0:1787826508.834132:0:2378:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1787826508.834150:0:2378:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:2.0:1787826508.834195:0:2378:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:1.0:1787826508.834276:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826505/real 1787826508] req@ffff9181c5636d80 x1874663043074816/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826530 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787826508.834299:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043074816/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826530 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787826508.834309:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787826508.834314:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787826508.834330:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043074816:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787826508.834338:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826505/real 1787826508] req@ffff9181c5637480 x1874663043074944/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826530 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787826508.834352:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043074944/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826530 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787826508.834360:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787826508.834364:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787826508.834367:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043074944:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787826509.409070:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826510.432922:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787826510.882623:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826510.882644:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787826510.882648:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826510.882662:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8577 00000100:00080000:1.0:1787826510.882675:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787826510.882682:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787826510.882727:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3800 x1874663043075072/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826510.882821:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043075072:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787826510.882942:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826510.882948:0:111835:0:(pinger.c:247:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1787826510.882952:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826510.882956:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787826510.882958:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826510.882964:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8577 00000100:00080000:1.0:1787826510.882969:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787826510.882974:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787826510.882988:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787826510.883005:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3b80 x1874663043075200/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826510.883048:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043075200:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787826511.456991:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1787826511.905241:0:2379:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1021 -> 192.168.203.147:988 00000400:00000100:2.0:1787826511.905289:0:2379:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1787826511.905305:0:2379:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:2.0:1787826511.905346:0:2379:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:3.0:1787826511.905384:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826510/real 1787826511] req@ffff9181c35b3b80 x1874663043075200/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826535 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826511.905410:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3b80 x1874663043075200/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826535 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826511.905421:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826511.905425:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826511.905429:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043075200:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787826511.905437:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826510/real 1787826511] req@ffff9181c35b3800 x1874663043075072/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826535 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826511.905450:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3800 x1874663043075072/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826535 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826511.905458:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826511.905461:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826511.905476:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043075072:192.168.203.147@tcp:250:kworker.0 00010000:00010000:2.0:1787826512.480920:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826513.504896:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826514.529692:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1787826514.977023:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787826494/real 1787826494] req@ffff9181e007ce00 x1874663043074432/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826514 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826514.977069:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ce00 x1874663043074432/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826514 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826514.977093:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826514.977123:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826514.977128:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043074432:192.168.203.147@tcp:38:kworker.0 00000100:00080000:1.0:1787826514.977177:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826514.977184:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787826514.977187:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826514.977194:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8582 00000100:00080000:1.0:1787826514.977205:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787826514.977213:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787826514.977253:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1f80 x1874663043075328/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787826514.977300:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826514.977303:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787826514.977305:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826514.977310:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8566 00000100:00080000:1.0:1787826514.977316:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 15s 00000100:00080000:1.0:1787826514.977320:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787826514.977331:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787826514.977344:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0700 x1874663043075456/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787826514.977353:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826514.977356:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787826514.977358:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826514.977361:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8582 00000100:00080000:1.0:1787826514.977365:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787826514.977368:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787826514.977373:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787826514.977381:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2a00 x1874663043075584/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826514.977847:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043075328:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787826514.977944:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826514.977980:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826514/real 1787826514] req@ffff9181c35b1f80 x1874663043075328/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826539 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826514.977993:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1f80 x1874663043075328/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826539 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826514.978003:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826514.978006:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826514.978025:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043075328:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787826514.978032:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043075456:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826514.978053:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826514.978066:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826514/real 1787826514] req@ffff9181c35b0700 x1874663043075456/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826539 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826514.978076:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0700 x1874663043075456/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826539 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826514.978089:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826514.978092:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826514.978095:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043075456:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826514.978101:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043075584:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826514.978117:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826514.978125:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826514/real 1787826514] req@ffff9181c35b2a00 x1874663043075584/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826539 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826514.978135:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2a00 x1874663043075584/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826539 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826514.978142:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826514.978145:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826514.978148:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043075584:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787826515.552963:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826516.577003:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787826517.600913:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-sptlrpc' failed (-108). 00010000:00010000:3.0:1787826517.600916:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:00020000:1.0:1787826517.600923:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00010000:00010000:3.0:1787826518.625126:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787826519.008847:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:1.0:1787826519.008854:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff9181c5620000 from 341 10000000:01000000:1.0:1787826519.008859:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:1.0:1787826519.008907:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f4ea lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826519.008917:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f4ea lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826519.008929:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f4ea lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826519.008996:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215d6c0 going to sleep for 0 seconds 00000100:00100000:1.0:1787826519.009006:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043075712:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787826519.009022:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f4ea lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826519.009030:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f4ea lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826519.009037:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f4ea lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826519.009045:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f4ea lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826519.009052:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f4ea lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826519.009058:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4e00 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787826519.009062:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f4ea lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826519.009070:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f4ea lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787826519.009077:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f4ea lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826519.009082:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:1.0:1787826519.009086:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787826519.009092:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f4ea lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826519.009097:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f4ea lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826519.009119:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787826519.009123:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:3.0:1787826519.009214:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826519.009224:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787826519.009228:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826519.009239:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8586 00000100:00080000:3.0:1787826519.009249:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826519.009258:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787826519.009294:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043075840/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826519.009327:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826519.009330:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787826519.009331:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826519.009335:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8586 00000100:00080000:3.0:1787826519.009338:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826519.009341:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826519.009350:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826519.009359:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043075968/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826519.009365:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826519.009368:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787826519.009370:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826519.009372:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8586 00000100:00080000:3.0:1787826519.009374:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826519.009377:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826519.009380:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826519.009385:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043076096/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826519.009476:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043075840:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787826519.009588:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043075968:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826519.009623:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043076096:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787826519.648978:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826520.672913:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1787826521.121187:0:2376:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1021 -> 192.168.203.147:988 00000400:00000100:1.0:1787826521.121243:0:2376:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787826521.121263:0:2376:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1787826521.121314:0:2376:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1787826521.121323:0:2376:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:2.0:1787826521.121432:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826519/real 1787826521] req@ffff9181c5637100 x1874663043075840/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826544 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826521.121461:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043075840/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826544 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826521.121479:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826521.121485:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826521.121500:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043075840:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787826521.121508:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826519/real 1787826521] req@ffff9181c5637800 x1874663043075968/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826544 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826521.121520:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043075968/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826544 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826521.121527:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826521.121531:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826521.121534:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043075968:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826521.121540:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826519/real 1787826521] req@ffff9181c5635c00 x1874663043076096/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826544 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826521.121550:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043076096/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826544 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826521.121576:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826521.121579:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826521.121582:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043076096:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787826521.696972:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826522.721086:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826523.745087:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787826524.193027:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826524.193039:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787826524.193044:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826524.193057:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8590 00000100:00080000:3.0:1787826524.193069:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826524.193078:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787826524.193127:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637b80 x1874663043076224/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826524.193160:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826524.193164:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787826524.193166:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826524.193171:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8590 00000100:00080000:3.0:1787826524.193174:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826524.193179:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826524.193190:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826524.193203:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043076352/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826524.193207:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043076224:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787826524.193223:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826524.193227:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787826524.193229:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826524.193234:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8590 00000100:00080000:3.0:1787826524.193238:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826524.193242:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826524.193247:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826524.193259:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634a80 x1874663043076480/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787826524.193288:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826524.193325:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826524/real 1787826524] req@ffff9181c5637b80 x1874663043076224/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826549 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826524.193345:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637b80 x1874663043076224/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826549 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826524.193357:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826524.193362:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826524.193381:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043076224:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787826524.193407:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043076352:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826524.193428:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826524.193447:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826524/real 1787826524] req@ffff9181c5634000 x1874663043076352/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826549 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826524.193459:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043076352/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826549 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826524.193466:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826524.193471:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826524.193474:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043076352:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826524.193481:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043076480:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826524.193502:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826524.193528:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826524/real 1787826524] req@ffff9181c5634a80 x1874663043076480/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826549 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826524.193552:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634a80 x1874663043076480/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826549 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826524.193566:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826524.193570:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826524.193573:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043076480:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787826524.768952:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826525.792957:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826526.816973:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826527.841199:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826528.866480:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787826529.312992:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826529.313003:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787826529.313006:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826529.313016:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8595 00000100:00080000:3.0:1787826529.313026:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826529.313034:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787826529.313076:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634a80 x1874663043076608/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826529.313115:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826529.313119:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787826529.313120:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826529.313125:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8595 00000100:00080000:3.0:1787826529.313128:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826529.313132:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826529.313143:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826529.313157:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043076736/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826529.313166:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826529.313170:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787826529.313172:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826529.313176:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8595 00000100:00080000:3.0:1787826529.313180:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826529.313184:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826529.313189:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826529.313205:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637b80 x1874663043076864/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826529.313307:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043076608:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787826529.313490:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043076736:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826529.313546:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043076864:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787826529.889014:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1787826530.337141:0:2377:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1021 -> 192.168.203.147:988 00000400:00000100:1.0:1787826530.337205:0:2377:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787826530.337223:0:2377:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1787826530.337273:0:2377:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1787826530.337282:0:2377:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:2.0:1787826530.337373:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826529/real 1787826530] req@ffff9181c5637b80 x1874663043076864/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826554 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826530.337402:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637b80 x1874663043076864/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826554 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826530.337414:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826530.337420:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826530.337424:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043076864:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826530.337432:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826529/real 1787826530] req@ffff9181c5634a80 x1874663043076608/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826554 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826530.337445:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634a80 x1874663043076608/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826554 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826530.337454:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826530.337457:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826530.337512:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043076608:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787826530.337562:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826529/real 1787826530] req@ffff9181c5634000 x1874663043076736/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826554 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826530.337576:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043076736/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826554 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826530.337585:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826530.337589:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826530.337592:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043076736:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787826530.912947:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826531.936950:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826532.960937:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826533.984965:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787826534.432972:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826534.432988:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787826534.432992:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826534.433003:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8601 00000100:00080000:3.0:1787826534.433013:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826534.433020:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787826534.433060:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043076992/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826534.433108:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826534.433111:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787826534.433112:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826534.433116:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8601 00000100:00080000:3.0:1787826534.433119:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826534.433122:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826534.433141:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826534.433158:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043077120/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826534.433168:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826534.433172:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787826534.433174:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826534.433177:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8601 00000100:00080000:3.0:1787826534.433181:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826534.433185:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826534.433191:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826534.433201:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043077248/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826534.433733:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043076992:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787826534.433850:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826534.433893:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826534/real 1787826534] req@ffff9181c5635c00 x1874663043076992/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826559 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826534.433912:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043076992/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826559 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826534.433923:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826534.433927:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826534.433951:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043076992:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787826534.433959:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043077120:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826534.433983:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826534.433997:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826534/real 1787826534] req@ffff9181c5637800 x1874663043077120/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826559 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826534.434008:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043077120/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826559 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826534.434018:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826534.434021:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826534.434024:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043077120:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826534.434030:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043077248:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826534.434050:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826534.434061:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826534/real 1787826534] req@ffff9181c5637100 x1874663043077248/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826559 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826534.434073:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043077248/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826559 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826534.434080:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826534.434084:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826534.434087:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043077248:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787826535.009057:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826536.032953:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826537.056934:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826538.080975:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787826539.040916:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-cliir' failed (-108). 10000000:00000400:1.0:1787826539.040929:0:159735:0:(mgc_request.c:1900:mgc_process_log()) MGC192.168.203.147@tcp: IR log lustre-cliir failed, not fatal: rc = -108 10000000:01000000:1.0:1787826539.045645:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:1.0:1787826539.045655:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff9181c5620000 from 52 10000000:01000000:1.0:1787826539.045662:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:1.0:1787826539.045732:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4600/0xbef42c0221d6f4f1 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826539.045782:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b4600/0xbef42c0221d6f4f1 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826539.045798:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b4600/0xbef42c0221d6f4f1 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826539.045824:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215d6c0 going to sleep for 0 seconds 00000100:00100000:1.0:1787826539.045842:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043077376:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787826539.045867:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4600/0xbef42c0221d6f4f1 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826539.045876:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b4600/0xbef42c0221d6f4f1 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826539.045883:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4600/0xbef42c0221d6f4f1 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826539.045891:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b4600/0xbef42c0221d6f4f1 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826539.045898:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b4600/0xbef42c0221d6f4f1 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826539.045904:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4600 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787826539.045909:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4600/0xbef42c0221d6f4f1 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826539.045917:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b4600/0xbef42c0221d6f4f1 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787826539.045924:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4600/0xbef42c0221d6f4f1 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826539.045930:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1787826539.045933:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787826539.045941:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b4600/0xbef42c0221d6f4f1 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826539.045953:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b4600/0xbef42c0221d6f4f1 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826539.045978:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787826539.045982:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:0.0:1787826539.046074:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826539.046079:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787826539.046083:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826539.046089:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8606 00000100:00080000:0.0:1787826539.046100:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826539.046107:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787826539.046142:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7480 x1874663043077504/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787826539.046166:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826539.046171:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787826539.046172:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826539.046178:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8606 00000100:00080000:0.0:1787826539.046182:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826539.046186:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787826539.046198:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826539.046212:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5c00 x1874663043077632/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826539.046217:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043077504:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787826539.046235:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826539.046239:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787826539.046241:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826539.046247:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8606 00000100:00080000:0.0:1787826539.046251:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826539.046256:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787826539.046262:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826539.046281:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6d80 x1874663043077760/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787826539.046292:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826539.046330:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826539/real 1787826539] req@ffff9181c55a7480 x1874663043077504/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826564 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826539.046345:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7480 x1874663043077504/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826564 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826539.046352:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826539.046355:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826539.046369:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043077504:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787826539.046389:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043077632:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826539.046418:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826539.046441:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826539/real 1787826539] req@ffff9181c55a5c00 x1874663043077632/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826564 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826539.046454:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5c00 x1874663043077632/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826564 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826539.046462:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826539.046465:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826539.046468:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043077632:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826539.046474:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043077760:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826539.046494:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826539.046505:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826539/real 1787826539] req@ffff9181c55a6d80 x1874663043077760/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826564 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826539.046513:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6d80 x1874663043077760/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826564 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826539.046518:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826539.046526:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826539.046530:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043077760:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787826539.104982:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826540.129072:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826541.152960:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826542.176982:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826543.200953:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787826544.161096:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826544.161107:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787826544.161113:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826544.161127:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8610 00000100:00080000:3.0:1787826544.161140:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826544.161149:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787826544.161192:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043077888/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826544.161220:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826544.161224:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787826544.161226:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826544.161230:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8610 00000100:00080000:3.0:1787826544.161235:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826544.161239:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826544.161249:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826544.161262:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043078016/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826544.161271:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826544.161274:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787826544.161276:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826544.161280:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8610 00000100:00080000:3.0:1787826544.161283:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826544.161286:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826544.161290:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826544.161299:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043078144/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826544.161707:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043077888:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1787826544.161853:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826544.161891:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826544/real 1787826544] req@ffff9181c5637480 x1874663043077888/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826569 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826544.161911:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043077888/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826569 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826544.161922:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826544.161928:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826544.161943:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043077888:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787826544.161950:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043078016:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787826544.161973:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826544.161987:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826544/real 1787826544] req@ffff9181c5636d80 x1874663043078016/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826569 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826544.162000:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043078016/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826569 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826544.162013:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826544.162017:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826544.162020:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043078016:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787826544.162025:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043078144:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787826544.162043:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826544.162053:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826544/real 1787826544] req@ffff9181c5635500 x1874663043078144/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826569 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826544.162063:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043078144/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826569 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826544.162070:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826544.162073:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826544.162076:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043078144:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787826544.225008:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826545.248980:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826546.272928:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826547.296981:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826548.320904:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787826549.280891:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826549.280909:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787826549.280913:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826549.280927:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8615 00000100:00080000:3.0:1787826549.280938:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826549.280946:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787826549.280991:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043078272/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826549.281028:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826549.281032:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787826549.281033:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826549.281037:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8615 00000100:00080000:3.0:1787826549.281040:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826549.281044:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826549.281054:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826549.281064:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043078400/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826549.281072:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826549.281076:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787826549.281078:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826549.281081:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8615 00000100:00080000:3.0:1787826549.281084:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826549.281088:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826549.281092:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826549.281099:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043078528/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826549.281163:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043078272:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787826549.281277:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043078400:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826549.281307:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043078528:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787826549.344925:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826550.368909:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826551.392939:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1787826551.841355:0:2378:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1021 -> 192.168.203.147:988 00000400:00000100:0.0:1787826551.841414:0:2378:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1787826551.841435:0:2378:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1787826551.841482:0:2378:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1787826551.841492:0:2378:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:2.0:1787826551.842085:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826549/real 1787826551] req@ffff9181c5635500 x1874663043078272/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826574 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826551.842116:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043078272/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826574 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826551.842130:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826551.842135:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826551.842154:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043078272:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787826551.842164:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826549/real 1787826551] req@ffff9181c5636d80 x1874663043078400/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826574 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826551.842176:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043078400/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826574 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826551.842183:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826551.842187:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826551.842190:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043078400:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826551.842197:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826549/real 1787826551] req@ffff9181c5637480 x1874663043078528/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826574 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826551.842208:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043078528/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826574 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826551.842216:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826551.842220:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826551.842226:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043078528:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787826552.416909:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826553.441111:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787826554.400840:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826554.400847:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787826554.400851:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826554.400859:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8621 00000100:00080000:3.0:1787826554.400868:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826554.400873:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787826554.400911:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634700 x1874663043078656/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826554.400938:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826554.400942:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787826554.400944:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826554.400948:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8621 00000100:00080000:3.0:1787826554.400952:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826554.400956:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826554.400966:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826554.400979:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043078784/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826554.400988:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826554.400991:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787826554.400994:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1787826554.400999:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043078656:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787826554.401001:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8621 00000100:00080000:3.0:1787826554.401008:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826554.401012:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826554.401025:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826554.401037:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043078912/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787826554.401063:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826554.401098:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826554/real 1787826554] req@ffff9181c5634700 x1874663043078656/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826579 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826554.401112:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634700 x1874663043078656/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826579 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826554.401122:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826554.401126:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826554.401140:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043078656:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787826554.401147:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043078784:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826554.401172:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826554.401185:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826554/real 1787826554] req@ffff9181c5634380 x1874663043078784/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826579 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826554.401194:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043078784/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826579 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826554.401202:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826554.401204:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826554.401208:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043078784:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826554.401230:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043078912:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826554.401250:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826554.401262:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826554/real 1787826554] req@ffff9181c5636a00 x1874663043078912/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826579 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826554.401272:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043078912/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826579 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826554.401279:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826554.401282:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826554.401285:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043078912:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787826554.464910:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826555.488950:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826556.513001:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826557.536948:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826558.560940:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787826559.520945:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826559.520961:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787826559.520966:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 10000000:01000000:3.0:1787826559.520972:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-client' failed (-108). 10000000:00020000:3.0:1787826559.520981:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00080000:2.0:1787826559.520989:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8626 00000100:00080000:2.0:1787826559.521005:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826559.521016:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787826559.521076:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007fb80 x1874663043079040/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787826559.521129:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826559.521134:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787826559.521136:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826559.521141:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8626 00000100:00080000:2.0:1787826559.521145:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826559.521149:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826559.521162:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826559.521177:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ca80 x1874663043079168/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787826559.521191:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826559.521195:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787826559.521197:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826559.521202:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8626 00000100:00080000:2.0:1787826559.521206:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826559.521211:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826559.521217:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826559.521233:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e680 x1874663043079296/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826559.521410:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043079040:192.168.203.147@tcp:250:kworker.0 00000800:00000100:0.0:1787826559.521486:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826559.521522:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826559/real 1787826559] req@ffff9181e007fb80 x1874663043079040/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826584 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826559.521540:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007fb80 x1874663043079040/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826584 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826559.521551:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826559.521555:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826559.521559:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043079040:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787826559.521567:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043079168:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826559.521589:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826559.521601:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826559/real 1787826559] req@ffff9181e007ca80 x1874663043079168/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826584 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826559.521614:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ca80 x1874663043079168/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826584 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826559.521621:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826559.521624:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826559.521628:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043079168:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787826559.521634:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043079296:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826559.521658:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826559.521670:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826559/real 1787826559] req@ffff9181e007e680 x1874663043079296/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826584 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826559.521693:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e680 x1874663043079296/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826584 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826559.521699:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826559.521701:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826559.521703:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043079296:192.168.203.147@tcp:38:kworker.0 10000000:01000000:3.0:1787826559.524701:0:159735:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:3.0:1787826559.524708:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff9181c5620000 from 7 10000000:01000000:3.0:1787826559.524714:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00010000:00010000:3.0:1787826559.524798:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f4f8 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826559.524809:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f4f8 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826559.524823:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f4f8 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826559.524850:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d225f780 going to sleep for 0 seconds 00000100:00100000:3.0:1787826559.524865:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043079424:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:3.0:1787826559.524883:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f4f8 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826559.524894:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f4f8 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826559.524901:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f4f8 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826559.524908:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f4f8 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826559.524921:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f4f8 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826559.524928:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3481e00 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:3.0:1787826559.524931:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f4f8 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826559.524939:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f4f8 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1787826559.524945:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f4f8 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787826559.524952:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:3.0:1787826559.524956:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826559.524964:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f4f8 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826559.524971:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f4f8 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787826559.524997:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:3.0:1787826559.525001:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:2.0:1787826559.525104:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826559.525111:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787826559.525122:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826559.525133:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8631 00000100:00080000:2.0:1787826559.525141:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826559.525149:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787826559.525180:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007dc00 x1874663043079552/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826559.525247:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043079552:192.168.203.147@tcp:250:kworker.0 00000800:00000100:0.0:1787826559.525310:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826559.525340:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826559/real 1787826559] req@ffff9181e007dc00 x1874663043079552/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826584 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826559.525356:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007dc00 x1874663043079552/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826584 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826559.525366:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826559.525370:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826559.525383:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043079552:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1787826559.584995:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826560.608994:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826561.633005:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826562.657067:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826563.680968:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787826564.640909:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826564.640924:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787826564.640929:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826564.640941:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8631 00000100:00080000:2.0:1787826564.640952:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826564.640961:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787826564.641007:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c380 x1874663043079680/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787826564.641039:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826564.641043:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787826564.641045:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826564.641049:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8631 00000100:00080000:2.0:1787826564.641053:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826564.641057:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826564.641068:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826564.641081:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c000 x1874663043079808/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787826564.641092:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826564.641095:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787826564.641097:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826564.641101:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8631 00000100:00080000:2.0:1787826564.641106:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826564.641109:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826564.641115:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826564.641125:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043079680:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787826564.641132:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007dc00 x1874663043079936/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787826564.641204:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826564.641247:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826564/real 1787826564] req@ffff9181e007c380 x1874663043079680/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826589 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826564.641279:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c380 x1874663043079680/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826589 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826564.641296:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826564.641300:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826564.641319:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043079680:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787826564.641328:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043079808:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826564.641353:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826564.641375:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826564/real 1787826564] req@ffff9181e007c000 x1874663043079808/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826589 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826564.641391:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c000 x1874663043079808/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826589 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826564.641399:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826564.641403:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826564.641406:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043079808:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787826564.641430:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043079936:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826564.641449:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826564.641460:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826564/real 1787826564] req@ffff9181e007dc00 x1874663043079936/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826589 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826564.641470:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007dc00 x1874663043079936/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826589 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826564.641477:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826564.641481:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826564.641484:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043079936:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787826564.705014:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826565.728940:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826566.752980:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826567.777007:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826568.800996:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787826569.760889:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826569.760902:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787826569.760907:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826569.760920:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8636 00000100:00080000:2.0:1787826569.760932:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826569.760941:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787826569.760987:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e680 x1874663043080064/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787826569.761022:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826569.761026:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787826569.761028:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826569.761032:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8636 00000100:00080000:2.0:1787826569.761035:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826569.761040:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826569.761055:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826569.761069:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043080064:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787826569.761070:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ca80 x1874663043080192/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787826569.761085:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826569.761088:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787826569.761089:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826569.761093:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8636 00000100:00080000:2.0:1787826569.761097:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826569.761101:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826569.761106:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826569.761119:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007fb80 x1874663043080320/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787826569.761138:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826569.761175:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826569/real 1787826569] req@ffff9181e007e680 x1874663043080064/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826594 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826569.761195:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e680 x1874663043080064/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826594 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826569.761206:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826569.761210:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826569.761237:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043080064:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787826569.761266:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043080192:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826569.761289:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826569.761302:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826569/real 1787826569] req@ffff9181e007ca80 x1874663043080192/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826594 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826569.761315:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ca80 x1874663043080192/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826594 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826569.761322:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826569.761326:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826569.761329:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043080192:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787826569.761335:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043080320:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826569.761356:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826569.761367:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826569/real 1787826569] req@ffff9181e007fb80 x1874663043080320/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826594 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826569.761379:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007fb80 x1874663043080320/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826594 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826569.761387:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826569.761390:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826569.761393:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043080320:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787826569.825091:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826570.848973:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826571.873098:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826572.897174:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826573.921013:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787826574.880922:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826574.880935:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787826574.880940:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826574.880953:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8641 00000100:00080000:0.0:1787826574.880965:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826574.880974:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787826574.881021:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5880 x1874663043080448/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787826574.881058:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826574.881062:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787826574.881064:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826574.881068:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8641 00000100:00080000:0.0:1787826574.881072:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826574.881075:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787826574.881086:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826574.881100:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5f80 x1874663043080576/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787826574.881116:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826574.881126:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787826574.881131:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1787826574.881132:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043080448:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787826574.881139:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8641 00000100:00080000:0.0:1787826574.881148:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826574.881154:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787826574.881159:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826574.881175:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4700 x1874663043080704/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787826574.881232:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826574.881270:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826574/real 1787826574] req@ffff9181c55a5880 x1874663043080448/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826599 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826574.881287:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5880 x1874663043080448/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826599 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826574.881298:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826574.881302:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826574.881375:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043080448:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787826574.881489:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043080576:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826574.881512:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826574.881524:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826574/real 1787826574] req@ffff9181c55a5f80 x1874663043080576/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826599 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826574.881535:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5f80 x1874663043080576/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826599 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826574.881542:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826574.881545:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826574.881548:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043080576:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826574.881554:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043080704:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826574.881570:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826574.881579:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826574/real 1787826574] req@ffff9181c55a4700 x1874663043080704/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826599 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826574.881590:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4700 x1874663043080704/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826599 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826574.881597:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826574.881600:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826574.881603:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043080704:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787826574.945028:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826575.968990:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826576.993488:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826578.016984:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826579.040941:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:3.0:1787826580.000929:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'params' failed (-108). 10000000:00020000:3.0:1787826580.000940:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00080000:0.0:1787826580.001023:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826580.001028:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787826580.001032:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826580.001040:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8646 00000100:00080000:0.0:1787826580.001051:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826580.001058:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787826580.001093:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4380 x1874663043080832/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787826580.001117:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826580.001121:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787826580.001122:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826580.001126:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8646 00000100:00080000:0.0:1787826580.001130:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826580.001134:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787826580.001156:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826580.001169:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5180 x1874663043080960/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787826580.001177:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826580.001179:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787826580.001181:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826580.001184:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8646 00000100:00080000:0.0:1787826580.001187:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826580.001196:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787826580.001200:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826580.001210:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5500 x1874663043081088/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826580.001736:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043080832:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787826580.001820:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826580.001855:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826580/real 1787826580] req@ffff9181c55a4380 x1874663043080832/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826605 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826580.001872:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4380 x1874663043080832/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826605 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826580.001883:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826580.001887:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826580.001891:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043080832:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787826580.001898:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043080960:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826580.001919:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826580.001932:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826580/real 1787826580] req@ffff9181c55a5180 x1874663043080960/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826605 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826580.001944:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5180 x1874663043080960/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826605 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826580.001959:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826580.001962:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826580.001965:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043080960:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826580.001971:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043081088:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826580.001988:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826580.001998:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826580/real 1787826580] req@ffff9181c55a5500 x1874663043081088/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826605 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826580.002009:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5500 x1874663043081088/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826605 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826580.002016:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826580.002020:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826580.002023:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043081088:192.168.203.147@tcp:38:kworker.0 10000000:01000000:3.0:1787826580.007193:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:3.0:1787826580.007207:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff9181c87b19b8 from 1 10000000:01000000:3.0:1787826580.007213:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00010000:00010000:3.0:1787826580.007376:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f4ff lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826580.007386:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f4ff lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826580.007396:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f4ff lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826580.007418:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d225f780 going to sleep for 0 seconds 00000100:00100000:3.0:1787826580.007431:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043081216:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:3.0:1787826580.007446:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f4ff lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826580.007454:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f4ff lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826580.007463:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f4ff lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826580.007470:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f4ff lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826580.007481:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f4ff lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826580.007488:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3481e00 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:3.0:1787826580.007492:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f4ff lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826580.007501:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f4ff lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1787826580.007508:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f4ff lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787826580.007515:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:3.0:1787826580.007519:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826580.007526:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f4ff lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826580.007533:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f4ff lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787826580.007557:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:3.0:1787826580.007560:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:1.0:1787826580.007616:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826580.007624:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787826580.007628:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826580.007637:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8651 00000100:00080000:1.0:1787826580.007646:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787826580.007654:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787826580.007689:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2d80 x1874663043081344/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826580.007770:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043081344:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787826580.007835:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826580.007869:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826580/real 1787826580] req@ffff9181c35b2d80 x1874663043081344/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826605 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826580.007885:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2d80 x1874663043081344/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826605 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826580.007894:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826580.007899:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826580.007916:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043081344:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1787826580.064943:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826581.089130:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826582.112954:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826583.140400:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826584.160906:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787826585.122529:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826585.122542:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787826585.122547:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826585.122559:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8651 00000100:00080000:0.0:1787826585.122572:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826585.122581:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787826585.122628:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4e00 x1874663043081472/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787826585.122673:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826585.122677:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787826585.122679:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826585.122683:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8651 00000100:00080000:0.0:1787826585.122687:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826585.122692:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787826585.122703:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826585.122717:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7800 x1874663043081600/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787826585.122727:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826585.122730:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787826585.122732:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826585.122737:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8651 00000100:00080000:0.0:1787826585.122740:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826585.122796:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787826585.122803:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826585.122816:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5500 x1874663043081728/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826585.122918:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043081472:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787826585.123029:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043081600:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826585.123058:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043081728:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787826585.184986:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826586.209008:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826587.232935:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1787826588.193292:0:2379:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:2.0:1787826588.193337:0:2379:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1787826588.193354:0:2379:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:0.0:1787826588.193444:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826585/real 1787826588] req@ffff9181c55a4e00 x1874663043081472/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826610 ref 2 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826588.193467:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4e00 x1874663043081472/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826610 ref 2 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826588.193478:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826588.193482:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826588.193500:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043081472:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787826588.193608:0:2379:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:0.0:1787826588.193892:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826585/real 1787826588] req@ffff9181c55a7800 x1874663043081600/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826610 ref 2 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826588.193913:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7800 x1874663043081600/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826610 ref 2 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826588.193921:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826588.193925:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826588.193928:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043081600:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826588.194114:0:2379:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:0.0:1787826588.194213:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826585/real 1787826588] req@ffff9181c55a5500 x1874663043081728/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826610 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826588.194227:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5500 x1874663043081728/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826610 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826588.194234:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826588.194238:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826588.194241:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043081728:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787826588.256889:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826589.281016:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787826590.240959:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826590.240968:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787826590.240972:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826590.240982:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8656 00000100:00080000:0.0:1787826590.240993:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826590.241000:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787826590.243209:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5500 x1874663043081856/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787826590.243259:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826590.243264:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787826590.243266:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826590.243279:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8656 00000100:00080000:0.0:1787826590.243289:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826590.243296:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787826590.243307:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826590.243322:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5180 x1874663043081984/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787826590.243330:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826590.243333:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787826590.243335:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826590.243339:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8656 00000100:00080000:0.0:1787826590.243341:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826590.243345:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787826590.243349:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826590.243357:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4380 x1874663043082112/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826590.243632:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043081856:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787826590.243883:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826590.243920:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826590/real 1787826590] req@ffff9181c55a5500 x1874663043081856/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826615 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826590.243937:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5500 x1874663043081856/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826615 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826590.243947:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826590.243951:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826590.243967:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043081856:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787826590.243974:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043081984:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826590.244007:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826590.244020:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826590/real 1787826590] req@ffff9181c55a5180 x1874663043081984/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826615 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826590.244035:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5180 x1874663043081984/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826615 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826590.244046:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826590.244050:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826590.244054:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043081984:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826590.244065:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043082112:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826590.244097:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826590.244110:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826590/real 1787826590] req@ffff9181c55a4380 x1874663043082112/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826615 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826590.244122:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4380 x1874663043082112/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826615 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826590.244129:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826590.244132:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826590.244135:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043082112:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787826590.304935:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826591.329141:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826592.353149:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826593.376961:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826594.401038:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787826595.360942:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826595.360953:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787826595.360956:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826595.360966:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8661 00000100:00080000:1.0:1787826595.360977:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787826595.360984:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787826595.361021:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0a80 x1874663043082240/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787826595.361060:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826595.361064:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787826595.361066:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826595.361070:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8661 00000100:00080000:1.0:1787826595.361073:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787826595.361077:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787826595.361085:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787826595.361098:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3480 x1874663043082368/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787826595.361110:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1787826595.361113:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043082240:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787826595.361114:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787826595.361129:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826595.361135:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8661 00000100:00080000:1.0:1787826595.361139:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787826595.361143:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787826595.361148:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787826595.361161:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0e00 x1874663043082496/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787826595.361178:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826595.361215:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826595/real 1787826595] req@ffff9181c35b0a80 x1874663043082240/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826620 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826595.361234:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0a80 x1874663043082240/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826620 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826595.361250:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826595.361254:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826595.361276:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043082240:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787826595.361300:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043082368:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826595.361324:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826595.361337:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826595/real 1787826595] req@ffff9181c35b3480 x1874663043082368/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826620 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826595.361349:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3480 x1874663043082368/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826620 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826595.361356:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826595.361360:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826595.361363:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043082368:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826595.361369:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043082496:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826595.361405:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826595.361419:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826595/real 1787826595] req@ffff9181c35b0e00 x1874663043082496/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826620 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826595.361437:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0e00 x1874663043082496/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826620 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826595.361446:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826595.361449:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826595.361455:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043082496:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787826595.424915:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826596.448935:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826597.472914:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826598.496960:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826599.521071:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:3.0:1787826600.480904:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-sptlrpc' failed (-108). 10000000:00020000:3.0:1787826600.480913:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00080000:0.0:1787826600.481261:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826600.481268:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787826600.481271:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826600.481281:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8667 00000100:00080000:0.0:1787826600.481291:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826600.481299:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787826600.481333:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4700 x1874663043082624/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787826600.481367:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826600.481371:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787826600.481372:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826600.481377:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8667 00000100:00080000:0.0:1787826600.481380:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826600.481384:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787826600.481393:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826600.481405:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5f80 x1874663043082752/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787826600.481415:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826600.481418:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787826600.481420:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826600.481424:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8667 00000100:00080000:0.0:1787826600.481427:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826600.481430:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787826600.481434:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826600.481444:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5880 x1874663043082880/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826600.481532:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043082624:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787826600.481591:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826600.481624:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826600/real 1787826600] req@ffff9181c55a4700 x1874663043082624/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826625 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826600.481641:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4700 x1874663043082624/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826625 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826600.482333:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826600.482339:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826600.482344:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043082624:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787826600.482352:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043082752:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826600.482384:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826600.482399:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826600/real 1787826600] req@ffff9181c55a5f80 x1874663043082752/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826625 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826600.482412:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5f80 x1874663043082752/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826625 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826600.482420:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826600.482423:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826600.482426:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043082752:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826600.482432:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043082880:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826600.482453:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826600.482464:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826600/real 1787826600] req@ffff9181c55a5880 x1874663043082880/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826625 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826600.482476:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5880 x1874663043082880/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826625 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826600.482483:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826600.482486:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826600.482489:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043082880:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787826600.544977:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826601.568907:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:3.0:1787826602.144911:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:3.0:1787826602.144922:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff9181c5620000 from 341 10000000:01000000:3.0:1787826602.144931:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:3.0:1787826602.145026:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f506 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826602.145039:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f506 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826602.145053:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f506 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826602.145079:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d225f780 going to sleep for 0 seconds 00000100:00100000:3.0:1787826602.145093:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043083008:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:3.0:1787826602.145111:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f506 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826602.145119:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f506 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826602.145127:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f506 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826602.145135:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f506 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826602.145149:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f506 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826602.145156:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3481e00 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:3.0:1787826602.145160:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f506 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826602.145169:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f506 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1787826602.145176:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f506 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787826602.145220:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:3.0:1787826602.145225:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826602.145233:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f506 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826602.145241:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f506 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787826602.145270:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:3.0:1787826602.145274:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:0.0:1787826602.145460:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826602.145496:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787826602.145500:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826602.145511:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8672 00000100:00080000:0.0:1787826602.145522:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826602.145530:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787826602.145565:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6d80 x1874663043083136/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787826602.146230:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826602.146234:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787826602.146236:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826602.146242:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8672 00000100:00080000:0.0:1787826602.146247:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826602.146253:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787826602.146263:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826602.146268:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043083136:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787826602.146329:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826602.146359:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826602/real 1787826602] req@ffff9181c55a6d80 x1874663043083136/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826627 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826602.146369:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5c00 x1874663043083264/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826602.146370:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6d80 x1874663043083136/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826627 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826602.146395:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826602.146403:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826602.146408:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00080000:0.0:1787826602.146410:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787826602.146414:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826602.146419:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8672 00000100:00080000:0.0:1787826602.146424:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1787826602.146427:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043083136:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787826602.146432:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787826602.146440:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826602.146453:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7480 x1874663043083392/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826602.146465:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043083264:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826602.146491:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826602.146504:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826602/real 1787826602] req@ffff9181c55a5c00 x1874663043083264/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826627 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826602.146526:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5c00 x1874663043083264/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826627 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826602.146540:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826602.146543:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826602.146547:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043083264:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826602.146559:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043083392:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826602.146585:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826602.146596:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826602/real 1787826602] req@ffff9181c55a7480 x1874663043083392/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826627 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826602.146606:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7480 x1874663043083392/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826627 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826602.146613:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826602.146616:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826602.146619:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043083392:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787826602.592908:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826603.616853:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826604.641005:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826605.664876:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826606.688915:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787826607.648874:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826607.648885:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787826607.648889:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826607.648898:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8673 00000100:00080000:0.0:1787826607.648909:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826607.648916:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787826607.648965:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7b80 x1874663043083520/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787826607.648994:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826607.648998:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787826607.649000:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826607.649004:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8673 00000100:00080000:0.0:1787826607.649007:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826607.649011:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787826607.649020:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826607.649033:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6680 x1874663043083648/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826607.649042:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043083520:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787826607.649046:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826607.649053:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787826607.649055:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826607.649059:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8673 00000100:00080000:0.0:1787826607.649062:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826607.649066:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787826607.649071:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826607.649084:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7100 x1874663043083776/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787826607.649110:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826607.649144:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826607/real 1787826607] req@ffff9181c55a7b80 x1874663043083520/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826632 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826607.649160:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7b80 x1874663043083520/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826632 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826607.649171:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826607.649174:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826607.649194:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043083520:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787826607.649217:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043083648:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826607.649243:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826607.649256:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826607/real 1787826607] req@ffff9181c55a6680 x1874663043083648/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826632 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826607.649269:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6680 x1874663043083648/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826632 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826607.649277:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826607.649280:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826607.649283:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043083648:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826607.649288:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043083776:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826607.649308:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826607.649320:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826607/real 1787826607] req@ffff9181c55a7100 x1874663043083776/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826632 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826607.649332:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7100 x1874663043083776/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826632 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826607.649340:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826607.649343:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826607.649347:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043083776:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787826607.712978:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826608.737000:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826609.760889:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826610.784905:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826611.808846:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787826612.769110:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826612.769120:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787826612.769124:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826612.769133:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8679 00000100:00080000:3.0:1787826612.769144:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826612.769150:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787826612.769190:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634e00 x1874663043083904/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826612.769219:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826612.769222:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787826612.769224:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826612.769229:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8679 00000100:00080000:3.0:1787826612.769232:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826612.769235:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826612.769244:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826612.769257:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635f80 x1874663043084032/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826612.769265:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826612.769267:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787826612.769269:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826612.769272:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8679 00000100:00080000:3.0:1787826612.769275:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826612.769278:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826612.769282:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826612.769290:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043084160/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826612.771676:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043083904:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787826612.771828:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826612.771878:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826612/real 1787826612] req@ffff9181c5634e00 x1874663043083904/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826637 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826612.771898:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634e00 x1874663043083904/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826637 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826612.771930:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826612.771936:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826612.771958:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043083904:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787826612.771966:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043084032:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826612.771988:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826612.771999:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826612/real 1787826612] req@ffff9181c5635f80 x1874663043084032/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826637 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826612.772010:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635f80 x1874663043084032/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826637 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826612.772017:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826612.772021:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826612.772024:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043084032:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826612.772030:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043084160:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826612.772047:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826612.772055:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826612/real 1787826612] req@ffff9181c5636a00 x1874663043084160/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826637 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826612.772065:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043084160/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826637 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826612.772078:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826612.772082:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826612.772085:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043084160:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787826612.832962:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826613.856938:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826614.880967:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826615.905448:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826616.928923:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787826617.888869:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826617.888877:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787826617.888880:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826617.888899:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8684 00000100:00080000:1.0:1787826617.888910:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787826617.888918:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787826617.888953:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2680 x1874663043084288/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787826617.888988:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826617.888992:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787826617.888994:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826617.889002:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8684 00000100:00080000:1.0:1787826617.889006:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787826617.889010:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787826617.889019:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787826617.889032:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1180 x1874663043084416/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826617.889035:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043084288:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787826617.889041:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826617.889044:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787826617.889046:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826617.889049:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8684 00000100:00080000:1.0:1787826617.889051:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787826617.889055:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787826617.889059:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787826617.889068:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0000 x1874663043084544/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787826617.889104:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826617.889138:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826617/real 1787826617] req@ffff9181c35b2680 x1874663043084288/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826642 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826617.889158:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2680 x1874663043084288/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826642 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826617.889168:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826617.889171:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826617.889187:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043084288:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787826617.889211:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043084416:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826617.889237:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826617.889250:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826617/real 1787826617] req@ffff9181c35b1180 x1874663043084416/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826642 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826617.889261:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1180 x1874663043084416/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826642 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826617.889269:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826617.889272:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826617.889275:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043084416:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826617.889279:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043084544:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826617.889298:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826617.889309:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826617/real 1787826617] req@ffff9181c35b0000 x1874663043084544/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826642 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826617.889320:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0000 x1874663043084544/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826642 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826617.889327:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826617.889330:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826617.889333:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043084544:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787826617.952928:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826618.976907:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826620.000956:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826621.025889:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826622.048978:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:3.0:1787826622.496943:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-cliir' failed (-108). 10000000:00000400:3.0:1787826622.496952:0:159735:0:(mgc_request.c:1900:mgc_process_log()) MGC192.168.203.147@tcp: IR log lustre-cliir failed, not fatal: rc = -108 10000000:01000000:3.0:1787826622.502377:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:3.0:1787826622.502384:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff9181c5620000 from 52 10000000:01000000:3.0:1787826622.502389:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:3.0:1787826622.502439:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f50d lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826622.502450:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f50d lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826622.502461:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f50d lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826622.502483:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d225f780 going to sleep for 0 seconds 00000100:00100000:3.0:1787826622.502495:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043084672:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:3.0:1787826622.502509:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f50d lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826622.502516:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f50d lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826622.502523:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f50d lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826622.502531:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f50d lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826622.502537:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f50d lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826622.502543:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3481e00 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:3.0:1787826622.502547:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f50d lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826622.502554:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f50d lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1787826622.502561:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f50d lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787826622.502566:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:3.0:1787826622.502569:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826622.502574:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f50d lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826622.502581:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f50d lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787826622.502602:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:3.0:1787826622.502605:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:1.0:1787826622.502715:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826622.502721:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787826622.502724:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826622.502730:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8689 00000100:00080000:1.0:1787826622.502740:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787826622.503029:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787826622.503051:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1c00 x1874663043084800/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787826622.503077:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826622.503080:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787826622.503082:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826622.503086:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8689 00000100:00080000:1.0:1787826622.503090:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787826622.503094:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787826622.503102:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787826622.503116:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0380 x1874663043084928/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826622.503123:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043084800:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787826622.503129:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826622.503133:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787826622.503135:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826622.503139:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8689 00000100:00080000:1.0:1787826622.503143:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787826622.503148:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787826622.503153:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787826622.503166:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1500 x1874663043085056/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787826622.503185:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826622.503228:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826622/real 1787826622] req@ffff9181c35b1c00 x1874663043084800/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826647 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826622.503243:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1c00 x1874663043084800/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826647 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826622.503252:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826622.503270:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826622.503282:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043084800:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787826622.503299:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043084928:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826622.503321:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826622.503332:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826622/real 1787826622] req@ffff9181c35b0380 x1874663043084928/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826647 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826622.503342:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0380 x1874663043084928/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826647 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826622.503349:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826622.503353:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826622.503356:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043084928:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826622.503361:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043085056:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826622.503390:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826622.503401:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826622/real 1787826622] req@ffff9181c35b1500 x1874663043085056/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826647 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826622.503412:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1500 x1874663043085056/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826647 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826622.503419:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826622.503422:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826622.503425:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043085056:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787826623.073305:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826624.096960:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826625.121957:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826626.145076:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826627.169124:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787826627.616987:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826627.616994:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787826627.616997:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826627.617006:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8694 00000100:00080000:2.0:1787826627.617016:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826627.617028:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787826627.617063:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ce00 x1874663043085184/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787826627.617086:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826627.617089:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787826627.617091:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826627.617095:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8694 00000100:00080000:2.0:1787826627.617099:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826627.617103:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826627.617111:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826627.617123:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f480 x1874663043085312/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826627.617126:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043085184:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787826627.617131:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826627.617134:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787826627.617135:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826627.617139:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8694 00000100:00080000:2.0:1787826627.617142:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826627.617145:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826627.617149:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826627.617157:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d880 x1874663043085440/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787826627.617201:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826627.617240:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826627/real 1787826627] req@ffff9181e007ce00 x1874663043085184/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826652 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826627.617258:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ce00 x1874663043085184/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826652 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826627.617278:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826627.617282:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826627.617297:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043085184:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787826627.617319:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043085312:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826627.617346:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826627.617363:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826627/real 1787826627] req@ffff9181e007f480 x1874663043085312/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826652 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826627.617379:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f480 x1874663043085312/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826652 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826627.617388:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826627.617392:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826627.617395:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043085312:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787826627.617412:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043085440:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826627.617435:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826627.617447:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826627/real 1787826627] req@ffff9181e007d880 x1874663043085440/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826652 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826627.617459:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d880 x1874663043085440/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826652 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826627.617467:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826627.617470:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826627.617473:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043085440:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787826628.192947:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826629.217158:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826630.241871:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826631.265083:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826632.288936:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787826632.736894:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826632.736905:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787826632.736917:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826632.736929:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8699 00000100:00080000:0.0:1787826632.736940:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826632.736947:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787826632.736984:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7100 x1874663043085568/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787826632.737011:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826632.737015:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787826632.737017:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826632.737021:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8699 00000100:00080000:0.0:1787826632.737025:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826632.737030:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787826632.737039:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826632.737049:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043085568:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787826632.737055:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6680 x1874663043085696/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787826632.737068:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826632.737071:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787826632.737073:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826632.737078:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8699 00000100:00080000:0.0:1787826632.737081:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826632.737085:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787826632.737090:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826632.737100:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7b80 x1874663043085824/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787826632.737113:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826632.737148:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826632/real 1787826632] req@ffff9181c55a7100 x1874663043085568/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826657 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826632.737172:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7100 x1874663043085568/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826657 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826632.737183:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826632.737187:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826632.737204:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043085568:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787826632.737226:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043085696:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826632.737251:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826632.737268:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826632/real 1787826632] req@ffff9181c55a6680 x1874663043085696/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826657 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826632.737281:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6680 x1874663043085696/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826657 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826632.737289:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826632.737292:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826632.737295:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043085696:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826632.737302:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043085824:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826632.737321:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826632.737331:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826632/real 1787826632] req@ffff9181c55a7b80 x1874663043085824/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826657 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826632.737343:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7b80 x1874663043085824/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826657 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826632.737355:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826632.737358:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826632.737361:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043085824:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787826633.313048:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826634.337064:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826635.360912:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826636.385404:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826637.409014:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787826637.856891:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826637.856900:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787826637.856904:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826637.856922:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8704 00000100:00080000:3.0:1787826637.856934:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826637.856941:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787826637.856978:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043085952/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826637.857005:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826637.857008:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787826637.857009:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826637.857013:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8704 00000100:00080000:3.0:1787826637.857016:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826637.857019:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826637.857026:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826637.857035:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634700 x1874663043086080/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826637.857041:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826637.857043:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787826637.857044:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826637.857046:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8704 00000100:00080000:3.0:1787826637.857047:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826637.857049:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826637.857052:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826637.857057:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043086208/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826637.857348:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043085952:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787826637.857415:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826637.857447:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826637/real 1787826637] req@ffff9181c5634380 x1874663043085952/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826662 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826637.857464:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043085952/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826662 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826637.857474:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826637.857478:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826637.857504:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043085952:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787826637.857512:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043086080:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826637.857536:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826637.857548:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826637/real 1787826637] req@ffff9181c5634700 x1874663043086080/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826662 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826637.857560:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634700 x1874663043086080/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826662 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826637.857567:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826637.857571:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826637.857573:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043086080:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826637.857579:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043086208:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826637.857599:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826637.857609:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826637/real 1787826637] req@ffff9181c5637480 x1874663043086208/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826662 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826637.857620:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043086208/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826662 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826637.857626:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826637.857629:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826637.857633:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043086208:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787826638.433066:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826639.457983:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826640.481055:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826641.504907:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826642.529371:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787826642.978914:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826642.978924:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787826642.978929:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 10000000:01000000:3.0:1787826642.978933:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-client' failed (-108). 00000100:00080000:2.0:1787826642.978942:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8709 10000000:00020000:3.0:1787826642.978945:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00080000:2.0:1787826642.978955:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826642.978965:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787826642.979001:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f100 x1874663043086336/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787826642.979037:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826642.979041:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787826642.979043:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826642.979046:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8709 00000100:00080000:2.0:1787826642.979049:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826642.979054:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826642.979062:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826642.979074:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f800 x1874663043086464/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787826642.979085:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826642.979089:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787826642.979091:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826642.979095:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8709 00000100:00080000:2.0:1787826642.979099:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826642.979104:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826642.979109:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826642.979113:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043086336:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787826642.979124:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c700 x1874663043086592/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787826642.979273:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826642.979329:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826642/real 1787826642] req@ffff9181e007f100 x1874663043086336/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826667 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826642.979350:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f100 x1874663043086336/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826667 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826642.979361:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826642.979366:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826642.979377:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043086336:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787826642.979389:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043086464:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826642.979413:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826642.979425:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826642/real 1787826642] req@ffff9181e007f800 x1874663043086464/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826667 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826642.979438:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f800 x1874663043086464/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826667 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826642.979447:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826642.979450:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826642.979453:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043086464:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787826642.979479:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043086592:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826642.979500:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826642.979511:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826642/real 1787826642] req@ffff9181e007c700 x1874663043086592/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826667 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826642.979524:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c700 x1874663043086592/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826667 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826642.979549:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826642.979553:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826642.979556:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043086592:192.168.203.147@tcp:38:kworker.0 10000000:01000000:3.0:1787826642.982675:0:159735:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:3.0:1787826642.982682:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff9181c5620000 from 7 10000000:01000000:3.0:1787826642.982686:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00010000:00010000:3.0:1787826642.982782:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f514 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826642.982793:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f514 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826642.982807:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f514 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826642.982834:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d225f780 going to sleep for 0 seconds 00000100:00100000:3.0:1787826642.982845:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043086720:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:3.0:1787826642.982860:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f514 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826642.982870:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f514 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826642.982876:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f514 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826642.982882:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f514 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826642.982890:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f514 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826642.982906:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3481e00 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:3.0:1787826642.982912:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f514 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826642.982919:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f514 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1787826642.982926:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f514 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787826642.982931:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:3.0:1787826642.982935:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787826642.982941:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f514 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826642.982948:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f514 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787826642.982971:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:3.0:1787826642.982975:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:2.0:1787826642.985800:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826642.985810:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787826642.985813:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826642.985823:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8714 00000100:00080000:2.0:1787826642.985832:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826642.985839:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787826642.985875:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e300 x1874663043086848/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826642.986204:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043086848:192.168.203.147@tcp:250:kworker.0 00000800:00000100:0.0:1787826642.986291:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826642.986337:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826642/real 1787826642] req@ffff9181e007e300 x1874663043086848/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826667 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826642.986359:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e300 x1874663043086848/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826667 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826642.986378:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826642.986383:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826642.986409:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043086848:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787826643.553455:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826644.576976:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826645.601465:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826646.624913:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826647.649050:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787826648.096893:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826648.096918:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787826648.096923:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826648.096935:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8714 00000100:00080000:2.0:1787826648.096948:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826648.096954:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787826648.097004:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d180 x1874663043086976/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787826648.097036:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826648.097040:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787826648.097041:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826648.097046:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8714 00000100:00080000:2.0:1787826648.097049:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826648.097053:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826648.097068:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826648.097083:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d500 x1874663043087104/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787826648.097092:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826648.097096:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787826648.097097:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826648.097101:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8714 00000100:00080000:2.0:1787826648.097105:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826648.097110:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826648.097119:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826648.097120:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043086976:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787826648.097135:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ea00 x1874663043087232/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826648.097234:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043087104:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787826648.097267:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043087232:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1787826648.672899:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826649.696983:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826650.720905:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1787826651.169095:0:2376:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:1.0:1787826651.169150:0:2376:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787826651.169169:0:2376:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1787826651.169228:0:2376:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1787826651.169238:0:2376:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:0.0:1787826651.169338:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826648/real 1787826651] req@ffff9181e007ea00 x1874663043087232/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826673 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826651.169373:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ea00 x1874663043087232/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826673 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826651.169384:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826651.169389:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826651.169393:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043087232:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787826651.169400:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826648/real 1787826651] req@ffff9181e007d180 x1874663043086976/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826673 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826651.169411:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d180 x1874663043086976/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826673 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826651.169417:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826651.169421:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826651.169520:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043086976:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787826651.172134:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826648/real 1787826651] req@ffff9181e007d500 x1874663043087104/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826673 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826651.172160:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d500 x1874663043087104/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826673 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826651.172169:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826651.172173:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826651.172176:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043087104:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787826651.745033:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826652.768870:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787826653.216908:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826653.216922:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787826653.216926:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826653.216939:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8719 00000100:00080000:2.0:1787826653.216951:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826653.216959:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787826653.217013:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007df80 x1874663043087360/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787826653.217051:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826653.217055:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787826653.217057:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826653.217061:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8719 00000100:00080000:2.0:1787826653.217065:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826653.217069:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826653.217079:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826653.217092:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ed80 x1874663043087488/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787826653.217101:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826653.217104:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787826653.217106:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826653.217110:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8719 00000100:00080000:2.0:1787826653.217113:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826653.217117:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826653.217123:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826653.217135:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d500 x1874663043087616/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826653.217429:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043087360:192.168.203.147@tcp:250:kworker.0 00000800:00000100:0.0:1787826653.217510:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826653.217554:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826653/real 1787826653] req@ffff9181e007df80 x1874663043087360/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826678 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826653.217574:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007df80 x1874663043087360/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826678 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826653.217585:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826653.217590:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826653.217620:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043087360:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787826653.217628:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043087488:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826653.217662:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826653.217675:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826653/real 1787826653] req@ffff9181e007ed80 x1874663043087488/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826678 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826653.217687:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ed80 x1874663043087488/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826678 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826653.217695:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826653.217699:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826653.217702:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043087488:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787826653.217707:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043087616:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826653.217728:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826653.217768:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826653/real 1787826653] req@ffff9181e007d500 x1874663043087616/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826678 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826653.217781:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d500 x1874663043087616/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826678 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826653.217788:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826653.217791:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826653.217794:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043087616:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787826653.792994:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826654.816901:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826655.840904:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826656.864950:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826657.889168:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787826658.337344:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826658.337357:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787826658.337361:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826658.337375:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8724 00000100:00080000:3.0:1787826658.337388:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826658.337397:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787826658.337448:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043087744/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826658.337484:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826658.337487:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787826658.337489:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826658.337493:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8724 00000100:00080000:3.0:1787826658.337496:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826658.337500:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826658.337510:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826658.337523:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043087872/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826658.337533:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826658.337536:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787826658.337540:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1787826658.337541:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043087744:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787826658.337546:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8724 00000100:00080000:3.0:1787826658.337551:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826658.337555:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826658.337561:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826658.337573:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043088000/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787826658.337636:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826658.337677:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826658/real 1787826658] req@ffff9181c5636d80 x1874663043087744/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826683 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826658.337695:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043087744/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826683 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826658.337706:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826658.337710:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826658.337724:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043087744:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787826658.337794:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043087872:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826658.337818:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826658.337830:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826658/real 1787826658] req@ffff9181c5635500 x1874663043087872/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826683 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826658.337842:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043087872/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826683 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826658.337849:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826658.337852:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826658.337856:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043087872:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787826658.337862:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043088000:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826658.337877:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826658.337886:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826658/real 1787826658] req@ffff9181c5637100 x1874663043088000/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826683 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826658.337896:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043088000/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826683 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826658.337902:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826658.337905:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826658.337908:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043088000:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1787826658.912870:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826659.936942:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826660.961377:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826661.984908:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826663.008987:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787826663.456857:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826663.456867:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787826663.456871:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826663.456883:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8730 10000000:01000000:3.0:1787826663.456891:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'params' failed (-108). 00000100:00080000:2.0:1787826663.456894:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 10000000:00020000:3.0:1787826663.456899:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00080000:2.0:1787826663.456905:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 10000000:01000000:3.0:1787826663.456912:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:3.0:1787826663.456916:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff9181c87b19b8 from 1 10000000:01000000:3.0:1787826663.456920:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00000100:00100000:2.0:1787826663.456953:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d180 x1874663043088128/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:3.0:1787826663.456955:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f51b lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826663.456976:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f51b lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826663.456990:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f51b lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787826663.456993:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826663.456999:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787826663.457001:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826663.457007:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8730 00000100:00080000:2.0:1787826663.457012:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787826663.457012:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d225f780 going to sleep for 0 seconds 00000100:00080000:2.0:1787826663.457019:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787826663.457025:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043088256:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000008:00100000:2.0:1787826663.457031:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:3.0:1787826663.457040:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f51b lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826663.457045:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043088128:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787826663.457048:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ea00 x1874663043088384/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:3.0:1787826663.457059:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f51b lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787826663.457067:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:3.0:1787826663.457070:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f51b lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787826663.457073:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787826663.457078:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00010000:00010000:3.0:1787826663.457083:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f51b lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787826663.457085:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8730 00000100:00080000:2.0:1787826663.457093:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00010000:00010000:3.0:1787826663.457096:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f51b lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787826663.457101:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:3.0:1787826663.457106:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3481e00 already unused, calling callback (ffffffffc1273c20) 00000008:00100000:2.0:1787826663.457108:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 10000000:00010000:3.0:1787826663.457111:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f51b lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000800:00000100:0.0:1787826663.457115:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826663.457126:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e300 x1874663043088512/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:3.0:1787826663.457129:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f51b lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1787826663.457153:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f51b lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787826663.457159:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:3.0:1787826663.457164:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00000100:00100000:0.0:1787826663.457165:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826663/real 1787826663] req@ffff9181e007d180 x1874663043088128/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826688 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:3.0:1787826663.457171:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f51b lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787826663.457183:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181c3481e00/0xbef42c0221d6f51b lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826663.457195:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d180 x1874663043088128/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826688 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 10000000:01000000:3.0:1787826663.457204:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:0.0:1787826663.457207:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826663.457212:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826663.457217:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043088128:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787826663.457217:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00100000:0.0:1787826663.457240:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043088384:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826663.457264:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826663.457276:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826663/real 1787826663] req@ffff9181e007ea00 x1874663043088384/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826688 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826663.457289:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ea00 x1874663043088384/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826688 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826663.457294:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826663.457296:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826663.457298:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043088384:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787826663.457302:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043088512:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826663.457320:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826663.457331:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826663/real 1787826663] req@ffff9181e007e300 x1874663043088512/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826688 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826663.457342:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e300 x1874663043088512/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826688 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826663.457350:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826663.457353:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826663.457356:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043088512:192.168.203.147@tcp:38:kworker.0 00000100:00080000:2.0:1787826663.460029:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826663.460039:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787826663.460042:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826663.460050:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8735 00000100:00080000:2.0:1787826663.460059:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826663.460065:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787826663.460094:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c700 x1874663043088640/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826663.460200:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043088640:192.168.203.147@tcp:250:kworker.0 00000800:00000100:0.0:1787826663.460259:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826663.460287:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826663/real 1787826663] req@ffff9181e007c700 x1874663043088640/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826688 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826663.460302:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c700 x1874663043088640/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826688 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826663.460310:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826663.460314:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826663.460332:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043088640:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787826664.032889:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826665.056934:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826666.080931:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826667.104889:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826668.128996:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787826668.576908:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826668.576918:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787826668.576923:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826668.576935:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8735 00000100:00080000:0.0:1787826668.576949:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826668.577013:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787826668.577056:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7480 x1874663043088768/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787826668.577091:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826668.577095:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787826668.577097:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826668.577101:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8735 00000100:00080000:0.0:1787826668.577105:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826668.577110:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787826668.577122:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826668.577144:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043088768:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787826668.577147:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5c00 x1874663043088896/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787826668.577167:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826668.577172:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787826668.577174:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826668.577179:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8735 00000100:00080000:0.0:1787826668.577182:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826668.577188:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787826668.577194:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826668.577211:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6d80 x1874663043089024/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787826668.577240:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826668.577287:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826668/real 1787826668] req@ffff9181c55a7480 x1874663043088768/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826693 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826668.577305:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7480 x1874663043088768/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826693 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826668.577316:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826668.577321:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826668.577346:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043088768:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787826668.577374:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043088896:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826668.577410:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826668.577426:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826668/real 1787826668] req@ffff9181c55a5c00 x1874663043088896/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826693 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826668.577438:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5c00 x1874663043088896/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826693 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826668.577446:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826668.577450:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826668.577453:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043088896:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826668.577459:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043089024:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826668.577478:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826668.577498:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826668/real 1787826668] req@ffff9181c55a6d80 x1874663043089024/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826693 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826668.577509:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6d80 x1874663043089024/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826693 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826668.577522:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826668.577525:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826668.577537:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043089024:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787826669.153138:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826670.176984:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826671.200889:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826672.225569:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826673.249067:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787826673.697322:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826673.697335:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787826673.697340:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826673.697352:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8740 00000100:00080000:3.0:1787826673.697365:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826673.697373:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787826673.697421:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043089152/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826673.697547:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043089152:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787826673.697554:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826673.697561:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787826673.697567:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826673.697576:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8740 00000100:00080000:3.0:1787826673.697586:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826673.697595:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826673.697607:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000800:00000100:2.0:1787826673.697637:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826673.697654:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043089280/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826673.697670:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826673.697674:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787826673.697676:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826673.697681:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8740 00000100:00100000:2.0:1787826673.697686:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826673/real 1787826673] req@ffff9181c5637800 x1874663043089152/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826698 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826673.697687:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826673.697696:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826673.697706:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826673.697713:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043089152/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826698 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826673.697723:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043089408/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787826673.697727:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826673.697737:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826673.697800:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043089152:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787826673.697830:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043089280:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826673.698474:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826673.698488:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826673/real 1787826673] req@ffff9181c5635c00 x1874663043089280/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826698 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826673.698512:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043089280/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826698 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826673.698523:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826673.698526:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826673.698529:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043089280:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826673.698536:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043089408:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826673.698555:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826673.698565:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826673/real 1787826673] req@ffff9181c5634000 x1874663043089408/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826698 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826673.698582:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043089408/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826698 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826673.698590:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826673.698593:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826673.698595:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043089408:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1787826674.273019:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826675.297611:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826676.320854:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826677.344978:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826678.368886:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787826678.816882:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826678.816892:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787826678.816896:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826678.816907:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8745 00000100:00080000:1.0:1787826678.816917:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787826678.816924:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787826678.816959:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3100 x1874663043089536/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787826678.816988:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826678.816992:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787826678.816995:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826678.816999:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8745 00000100:00080000:1.0:1787826678.817002:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787826678.817006:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787826678.817016:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787826678.817035:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1500 x1874663043089664/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826678.817039:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043089536:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787826678.817052:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826678.817055:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787826678.817057:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826678.817060:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8745 00000100:00080000:1.0:1787826678.817063:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787826678.817066:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787826678.817070:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787826678.817080:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0380 x1874663043089792/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787826678.817106:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826678.817140:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826678/real 1787826678] req@ffff9181c35b3100 x1874663043089536/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826703 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826678.817156:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3100 x1874663043089536/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826703 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826678.817166:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826678.817169:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826678.817181:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043089536:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787826678.817200:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043089664:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826678.817234:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826678.817247:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826678/real 1787826678] req@ffff9181c35b1500 x1874663043089664/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826703 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826678.817259:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1500 x1874663043089664/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826703 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826678.817266:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826678.817268:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826678.817270:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043089664:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826678.817274:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043089792:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826678.817288:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826678.817295:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826678/real 1787826678] req@ffff9181c35b0380 x1874663043089792/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826703 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826678.817302:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0380 x1874663043089792/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826703 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826678.817307:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826678.817308:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826678.817310:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043089792:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787826679.394611:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826680.416853:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826681.441574:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826682.464886:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826683.488909:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787826683.936846:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826683.936857:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787826683.936861:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 10000000:01000000:1.0:1787826683.936867:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-sptlrpc' failed (-108). 00000100:00080000:3.0:1787826683.936872:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8750 10000000:00020000:1.0:1787826683.936875:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00080000:3.0:1787826683.936884:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826683.936895:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787826683.936927:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634a80 x1874663043089920/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826683.936959:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826683.936963:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787826683.936965:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826683.936970:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8750 00000100:00080000:3.0:1787826683.936973:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826683.936977:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826683.936986:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826683.936999:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637b80 x1874663043090048/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826683.937002:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043089920:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787826683.937017:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826683.937021:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787826683.937023:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826683.937028:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8750 00000100:00080000:3.0:1787826683.937031:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826683.937035:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826683.937041:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826683.937054:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043090176/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787826683.937078:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826683.937108:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826683/real 1787826683] req@ffff9181c5634a80 x1874663043089920/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826708 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826683.937120:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634a80 x1874663043089920/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826708 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826683.937134:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826683.937137:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826683.937140:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043089920:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787826683.937154:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043090048:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826683.937168:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826683.937180:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826683/real 1787826683] req@ffff9181c5637b80 x1874663043090048/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826708 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826683.937187:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637b80 x1874663043090048/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826708 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826683.937192:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826683.937194:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826683.937196:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043090048:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826683.937203:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043090176:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826683.937219:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826683.937235:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826683/real 1787826683] req@ffff9181c5634000 x1874663043090176/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826708 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826683.937247:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043090176/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826708 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826683.937254:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826683.937263:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826683.937266:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043090176:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1787826684.512862:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787826685.344856:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:1.0:1787826685.344863:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff9181c5620000 from 341 10000000:01000000:1.0:1787826685.344868:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:1.0:1787826685.344911:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f522 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826685.344919:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f522 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826685.344929:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f522 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826685.344948:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215d6c0 going to sleep for 0 seconds 00000100:00100000:1.0:1787826685.344959:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043090304:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787826685.344974:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f522 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826685.344983:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f522 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826685.345000:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f522 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826685.345006:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f522 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826685.345010:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f522 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826685.345014:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4200 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787826685.345016:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f522 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826685.345021:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f522 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787826685.345026:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f522 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826685.345029:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:1.0:1787826685.345031:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787826685.345035:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f522 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826685.345038:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f522 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826685.345054:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787826685.345056:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:3.0:1787826685.345087:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826685.345091:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787826685.345093:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826685.345098:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8755 00000100:00080000:3.0:1787826685.345105:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826685.345108:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787826685.345124:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043090432/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826685.345141:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826685.345144:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787826685.345145:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826685.345148:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8755 00000100:00080000:3.0:1787826685.345149:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826685.345151:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826685.345158:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826685.345170:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043090560/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826685.345180:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826685.345184:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:2.0:1787826685.345188:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043090432:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787826685.345188:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826685.345194:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8755 00000100:00080000:3.0:1787826685.345199:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826685.345203:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826685.345208:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826685.345217:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043090688/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787826685.345244:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826685.345268:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826685/real 1787826685] req@ffff9181c5635c00 x1874663043090432/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826710 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826685.345278:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043090432/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826710 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826685.345285:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826685.345288:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826685.345297:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043090432:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787826685.345311:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043090560:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826685.345325:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826685.345332:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826685/real 1787826685] req@ffff9181c5637800 x1874663043090560/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826710 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826685.345340:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043090560/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826710 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826685.345344:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826685.345347:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826685.345349:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043090560:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826685.345352:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043090688:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826685.345361:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826685.345366:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826685/real 1787826685] req@ffff9181c5637100 x1874663043090688/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826710 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826685.345373:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043090688/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826710 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826685.345377:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826685.345379:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826685.345380:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043090688:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787826685.536905:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826686.560889:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826687.584847:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826688.608856:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826689.632966:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787826690.592860:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826690.592872:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787826690.592876:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826690.592885:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8757 00000100:00080000:3.0:1787826690.592896:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826690.592903:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787826690.592937:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043090816/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826690.592964:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826690.592968:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787826690.592970:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826690.592974:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8757 00000100:00080000:3.0:1787826690.592978:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826690.592983:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826690.592993:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826690.593005:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043090816:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787826690.593011:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043090944/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826690.593023:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826690.593026:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787826690.593028:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826690.593032:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8757 00000100:00080000:3.0:1787826690.593036:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826690.593040:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826690.593045:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826690.593057:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043091072/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787826690.593064:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826690.593094:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826690/real 1787826690] req@ffff9181c5635500 x1874663043090816/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826715 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826690.593106:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043090816/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826715 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826690.593113:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826690.593121:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826690.593137:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043090816:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787826690.593160:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043090944:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826690.593187:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826690.593202:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826690/real 1787826690] req@ffff9181c5636d80 x1874663043090944/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826715 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826690.593226:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043090944/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826715 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826690.593234:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826690.593237:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826690.593241:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043090944:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826690.593247:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043091072:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826690.593268:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826690.593279:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826690/real 1787826690] req@ffff9181c5637480 x1874663043091072/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826715 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826690.593290:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043091072/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826715 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826690.593298:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826690.593301:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826690.593304:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043091072:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1787826690.656937:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826691.680864:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826692.704902:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826693.728917:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826694.752937:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787826695.712906:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826695.712913:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787826695.712917:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826695.712924:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8762 00000100:00080000:1.0:1787826695.712932:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787826695.712938:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787826695.712966:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1c00 x1874663043091200/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787826695.712985:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826695.712987:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787826695.712988:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826695.712991:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8762 00000100:00080000:1.0:1787826695.712994:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787826695.712998:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787826695.713006:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787826695.713019:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0000 x1874663043091328/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787826695.713030:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826695.713036:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:2.0:1787826695.713036:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043091200:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787826695.713038:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826695.713046:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8762 00000100:00080000:1.0:1787826695.713050:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787826695.713054:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787826695.713059:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787826695.713071:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1180 x1874663043091456/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787826695.713110:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826695.713154:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826695/real 1787826695] req@ffff9181c35b1c00 x1874663043091200/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826720 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826695.713170:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1c00 x1874663043091200/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826720 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826695.713180:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826695.713184:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826695.713195:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043091200:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787826695.713214:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043091328:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826695.713233:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826695.713241:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826695/real 1787826695] req@ffff9181c35b0000 x1874663043091328/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826720 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826695.713249:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0000 x1874663043091328/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826720 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826695.713254:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826695.713256:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826695.713258:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043091328:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826695.713261:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043091456:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826695.713270:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826695.713276:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826695/real 1787826695] req@ffff9181c35b1180 x1874663043091456/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826720 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826695.713282:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1180 x1874663043091456/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826720 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826695.713287:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826695.713288:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826695.713290:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043091456:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787826695.776905:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826696.800857:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826697.825022:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826698.848857:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826699.872878:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787826700.832863:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826700.832874:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787826700.832878:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826700.832888:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8767 00000100:00080000:0.0:1787826700.832898:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826700.832905:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787826700.832941:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5880 x1874663043091584/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787826700.832966:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826700.832969:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787826700.832971:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826700.832975:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8767 00000100:00080000:0.0:1787826700.832978:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826700.832982:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787826700.832991:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826700.833004:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5f80 x1874663043091712/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826700.833007:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043091584:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787826700.833022:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826700.833026:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787826700.833027:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826700.833032:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8767 00000100:00080000:0.0:1787826700.833036:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826700.833039:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787826700.833044:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826700.833055:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4700 x1874663043091840/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787826700.833065:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826700.833094:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826700/real 1787826700] req@ffff9181c55a5880 x1874663043091584/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826725 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826700.833105:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5880 x1874663043091584/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826725 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826700.833112:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826700.833114:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826700.833125:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043091584:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787826700.833147:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043091712:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826700.833167:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826700.833179:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826700/real 1787826700] req@ffff9181c55a5f80 x1874663043091712/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826725 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826700.833193:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5f80 x1874663043091712/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826725 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826700.833198:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826700.833200:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826700.833202:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043091712:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826700.833206:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043091840:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826700.833224:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826700.833234:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826700/real 1787826700] req@ffff9181c55a4700 x1874663043091840/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826725 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826700.833245:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4700 x1874663043091840/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826725 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826700.833252:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826700.833255:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826700.833259:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043091840:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1787826700.896853:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826701.920933:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826702.945513:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826703.968965:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826704.993877:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787826705.440855:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-cliir' failed (-108). 10000000:00000400:1.0:1787826705.440862:0:159735:0:(mgc_request.c:1900:mgc_process_log()) MGC192.168.203.147@tcp: IR log lustre-cliir failed, not fatal: rc = -108 10000000:01000000:1.0:1787826705.444444:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:1.0:1787826705.444451:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff9181c5620000 from 52 10000000:01000000:1.0:1787826705.444457:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:1.0:1787826705.444502:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f529 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826705.444513:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f529 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826705.444525:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f529 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826705.444546:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215d6c0 going to sleep for 0 seconds 00000100:00100000:1.0:1787826705.444555:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043091968:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787826705.444574:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f529 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826705.444582:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f529 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826705.444587:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f529 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826705.444593:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f529 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826705.444605:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f529 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826705.444612:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4c00 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787826705.444615:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f529 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826705.444622:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f529 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787826705.444629:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f529 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826705.444635:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1787826705.444638:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787826705.444643:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f529 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826705.444650:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f529 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826705.444672:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787826705.444675:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:2.0:1787826705.444775:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826705.444781:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787826705.444785:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826705.444792:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8772 00000100:00080000:2.0:1787826705.444801:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826705.444807:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787826705.444830:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f800 x1874663043092096/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787826705.444851:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826705.444855:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787826705.444857:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826705.444862:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8772 00000100:00080000:2.0:1787826705.444866:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826705.444870:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787826705.444874:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043092096:192.168.203.147@tcp:250:kworker.0 00000008:00100000:2.0:1787826705.444881:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826705.444898:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f100 x1874663043092224/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787826705.444913:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826705.444917:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787826705.444918:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826705.444922:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8772 00000100:00080000:2.0:1787826705.444926:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826705.444930:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826705.444942:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000800:00000100:0.0:1787826705.444943:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826705.444960:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d880 x1874663043092352/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826705.444990:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826705/real 1787826705] req@ffff9181e007f800 x1874663043092096/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826730 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826705.445005:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f800 x1874663043092096/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826730 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826705.445013:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826705.445017:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826705.445028:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043092096:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787826705.445052:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043092224:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826705.445081:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826705.445095:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826705/real 1787826705] req@ffff9181e007f100 x1874663043092224/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826730 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826705.445117:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f100 x1874663043092224/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826730 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826705.445125:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826705.445128:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826705.445132:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043092224:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787826705.445137:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043092352:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826705.445158:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826705.445169:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826705/real 1787826705] req@ffff9181e007d880 x1874663043092352/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826730 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826705.445180:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d880 x1874663043092352/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826730 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826705.445188:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826705.445192:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826705.445196:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043092352:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787826706.016922:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826707.040851:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826708.064890:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826709.088946:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826710.112926:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787826710.560869:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826710.560879:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787826710.560882:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826710.560891:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8777 00000100:00080000:3.0:1787826710.560901:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826710.560909:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787826710.560945:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634700 x1874663043092480/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826710.560969:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826710.560973:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787826710.560975:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826710.560979:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8777 00000100:00080000:3.0:1787826710.560983:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826710.560987:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826710.560995:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826710.561009:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043092608/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826710.561019:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826710.561023:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787826710.561024:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1787826710.561030:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043092480:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787826710.561037:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8777 00000100:00080000:3.0:1787826710.561042:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826710.561046:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826710.561050:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826710.561062:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043092736/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826710.561116:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043092608:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787826710.561149:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043092736:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787826711.136931:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826712.160896:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826713.184930:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1787826713.633048:0:2377:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:1.0:1787826713.633094:0:2377:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787826713.633109:0:2377:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1787826713.633143:0:2377:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1787826713.633152:0:2377:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:0.0:1787826713.633184:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826710/real 1787826713] req@ffff9181c5636a00 x1874663043092736/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826735 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826713.633210:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043092736/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826735 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826713.633232:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826713.633236:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826713.633240:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043092736:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787826713.633247:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826710/real 1787826713] req@ffff9181c5634700 x1874663043092480/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826735 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826713.633260:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634700 x1874663043092480/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826735 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826713.633267:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826713.633270:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826713.633284:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043092480:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787826713.633291:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826710/real 1787826713] req@ffff9181c5634380 x1874663043092608/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826735 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826713.633303:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043092608/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826735 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826713.633310:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826713.633313:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826713.633317:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043092608:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1787826714.208866:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826715.232893:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787826715.680849:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826715.680860:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787826715.680865:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826715.680877:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8782 00000100:00080000:3.0:1787826715.680889:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826715.680896:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787826715.680930:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635f80 x1874663043092864/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826715.680967:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826715.680971:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787826715.680973:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826715.680978:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8782 00000100:00080000:3.0:1787826715.680981:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826715.680985:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826715.680995:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826715.681009:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634e00 x1874663043092992/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826715.681019:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043092864:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787826715.681021:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826715.681029:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787826715.681031:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826715.681036:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8782 00000100:00080000:3.0:1787826715.681039:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826715.681043:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826715.681048:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826715.681061:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043093120/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787826715.681089:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826715.681125:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826715/real 1787826715] req@ffff9181c5635f80 x1874663043092864/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826740 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826715.681145:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635f80 x1874663043092864/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826740 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826715.681154:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826715.681158:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826715.681177:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043092864:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787826715.681200:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043092992:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826715.681240:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826715.681254:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826715/real 1787826715] req@ffff9181c5634e00 x1874663043092992/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826740 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826715.681266:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634e00 x1874663043092992/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826740 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826715.681273:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826715.681276:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826715.681279:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043092992:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787826715.681285:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043093120:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826715.681303:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826715.681313:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826715/real 1787826715] req@ffff9181c5634380 x1874663043093120/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826740 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826715.681328:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043093120/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826740 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826715.681335:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826715.681338:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826715.681341:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043093120:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1787826716.256879:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826717.280930:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826718.304931:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826719.328911:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826720.353057:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787826720.800887:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826720.800900:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787826720.800904:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826720.800917:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8787 00000100:00080000:3.0:1787826720.800929:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826720.800937:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787826720.800977:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634700 x1874663043093248/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826720.801005:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826720.801008:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787826720.801009:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826720.801013:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8787 00000100:00080000:3.0:1787826720.801015:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826720.801017:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826720.801026:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826720.801038:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043093376/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826720.801045:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826720.801047:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787826720.801048:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826720.801051:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8787 00000100:00080000:3.0:1787826720.801052:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826720.801055:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826720.801058:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826720.801064:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043093248:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787826720.801067:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043093504/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787826720.801140:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826720.801193:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826720/real 1787826720] req@ffff9181c5634700 x1874663043093248/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826745 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826720.801220:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634700 x1874663043093248/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826745 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826720.801228:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826720.801231:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826720.801241:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043093248:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787826720.801247:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043093376:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826720.801264:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826720.801274:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826720/real 1787826720] req@ffff9181c5636a00 x1874663043093376/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826745 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826720.801291:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043093376/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826745 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826720.801299:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826720.801302:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826720.801306:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043093376:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787826720.801333:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043093504:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826720.801350:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826720.801358:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826720/real 1787826720] req@ffff9181c5637480 x1874663043093504/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826745 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826720.801366:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043093504/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826745 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826720.801371:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826720.801373:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826720.801375:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043093504:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787826721.378403:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826722.400887:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826723.424910:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826724.448848:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826725.473008:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787826725.920847:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826725.920855:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787826725.920858:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826725.920866:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8792 10000000:01000000:1.0:1787826725.920878:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-client' failed (-108). 00000100:00080000:3.0:1787826725.920883:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 10000000:00020000:1.0:1787826725.920884:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:1.0:1787826725.920890:0:159735:0:(mgc_request.c:611:do_requeue()) updating log params 00000100:00080000:3.0:1787826725.920892:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 10000000:01000000:1.0:1787826725.920894:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff9181c5620000 from 7 10000000:01000000:1.0:1787826725.920899:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00000100:00100000:3.0:1787826725.920940:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043093632/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787826725.920957:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f530 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787826725.920959:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826725.920965:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787826725.920968:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826725.920974:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8792 00010000:00010000:1.0:1787826725.920976:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f530 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787826725.920978:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826725.920984:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:1.0:1787826725.920994:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f530 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000008:00100000:3.0:1787826725.920994:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826725.921003:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043093632:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787826725.921010:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043093888/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787826725.921016:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215d6c0 going to sleep for 0 seconds 00000100:00080000:3.0:1787826725.921026:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1787826725.921027:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043093760:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00080000:3.0:1787826725.921033:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787826725.921037:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00010000:00010000:1.0:1787826725.921043:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f530 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787826725.921043:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8792 00000100:00080000:3.0:1787826725.921049:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826725.921057:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:1.0:1787826725.921058:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f530 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000800:00000100:0.0:1787826725.921066:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:3.0:1787826725.921066:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:1.0:1787826725.921074:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f530 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826725.921084:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f530 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787826725.921084:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043094016/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826725.921102:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826725/real 1787826725] req@ffff9181c5636d80 x1874663043093632/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826750 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:1.0:1787826725.921104:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f530 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826725.921117:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4400 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787826725.921123:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f530 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826725.921146:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043093632/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826750 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:1.0:1787826725.921148:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f530 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787826725.921166:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f530 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787826725.921169:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826725.921177:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 10000000:01000000:1.0:1787826725.921178:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:1.0:1787826725.921185:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00000100:00100000:0.0:1787826725.921186:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043093632:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787826725.921191:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f530 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826725.921200:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f530 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826725.921219:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00100000:0.0:1787826725.921221:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043093888:192.168.203.147@tcp:38:kworker.0 00000100:00080000:1.0:1787826725.921223:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000800:00000100:0.0:1787826725.921243:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826725.921265:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826725/real 1787826725] req@ffff9181c5635500 x1874663043093888/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826750 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826725.921268:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1787826725.921275:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043093888/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826750 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826725.921285:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787826725.921288:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826725.921289:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826725.921295:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00080000:3.0:1787826725.921297:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8797 00000100:00100000:0.0:1787826725.921301:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043093888:192.168.203.147@tcp:38:kworker.0 00000100:00080000:3.0:1787826725.921303:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826725.921309:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787826725.921311:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043094016:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787826725.921321:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043094144/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787826725.921336:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826725.921350:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826725/real 1787826725] req@ffff9181c5637100 x1874663043094016/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826750 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826725.921362:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043094016/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826750 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826725.921367:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826725.921370:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826725.921372:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043094016:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787826725.921382:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043094144:192.168.203.147@tcp:250:kworker.0 00000800:00000100:0.0:1787826725.921395:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826725.921401:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826725/real 1787826725] req@ffff9181c5637800 x1874663043094144/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826750 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826725.921408:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043094144/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826750 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826725.921412:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826725.921414:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826725.921425:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043094144:192.168.203.147@tcp:250:kworker.0 00010000:00010000:0.0:1787826726.497171:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826727.520901:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826728.544912:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826729.568972:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826730.592903:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787826731.040853:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826731.040870:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787826731.040878:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826731.040888:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8797 00000100:00080000:3.0:1787826731.040899:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826731.040905:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787826731.040946:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043094272/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826731.040974:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826731.040978:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787826731.040980:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826731.040990:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8797 00000100:00080000:3.0:1787826731.040994:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826731.041004:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826731.041013:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826731.041015:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043094272:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787826731.041031:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043094400/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826731.041041:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826731.041044:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787826731.041045:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826731.041048:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8797 00000100:00080000:3.0:1787826731.041052:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826731.041054:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826731.041057:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826731.041066:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637b80 x1874663043094528/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787826731.041069:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826731.041119:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826731/real 1787826731] req@ffff9181c5635c00 x1874663043094272/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826756 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826731.041134:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043094272/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826756 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826731.041141:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826731.041143:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826731.041154:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043094272:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787826731.041173:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043094400:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826731.041191:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826731.041202:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826731/real 1787826731] req@ffff9181c5634000 x1874663043094400/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826756 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826731.041219:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043094400/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826756 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826731.041230:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826731.041233:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826731.041237:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043094400:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787826731.041243:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043094528:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826731.041267:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826731.041275:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826731/real 1787826731] req@ffff9181c5637b80 x1874663043094528/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826756 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826731.041283:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637b80 x1874663043094528/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826756 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826731.041290:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826731.041294:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826731.041297:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043094528:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787826731.617217:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826732.640913:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826733.664917:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826734.688966:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826735.712926:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787826736.160908:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826736.160921:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787826736.160925:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826736.160937:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8802 00000100:00080000:3.0:1787826736.160951:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826736.160959:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787826736.161015:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634a80 x1874663043094656/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826736.161047:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826736.161051:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787826736.161053:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826736.161057:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8802 00000100:00080000:3.0:1787826736.161071:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826736.161077:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826736.161089:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826736.161104:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637b80 x1874663043094784/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826736.161109:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043094656:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787826736.161119:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826736.161123:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787826736.161125:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826736.161130:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8802 00000100:00080000:3.0:1787826736.161134:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826736.161138:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826736.161143:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826736.161156:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043094912/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787826736.161186:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826736.161226:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826736/real 1787826736] req@ffff9181c5634a80 x1874663043094656/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826761 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826736.161245:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634a80 x1874663043094656/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826761 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826736.161256:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826736.161260:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826736.161274:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043094656:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787826736.161302:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043094784:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826736.161324:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826736.161337:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826736/real 1787826736] req@ffff9181c5637b80 x1874663043094784/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826761 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826736.161349:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637b80 x1874663043094784/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826761 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826736.161356:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826736.161359:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826736.161362:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043094784:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787826736.161368:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043094912:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826736.161384:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826736.161394:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826736/real 1787826736] req@ffff9181c5634000 x1874663043094912/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826761 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826736.161405:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043094912/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826761 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826736.161413:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826736.161416:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826736.161419:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043094912:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787826736.737122:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826737.760920:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826738.784948:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826739.809025:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826740.833063:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787826741.281313:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826741.281326:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787826741.281331:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826741.281343:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8807 00000100:00080000:0.0:1787826741.281357:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826741.281365:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787826741.281415:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4380 x1874663043095040/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787826741.281449:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826741.281453:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787826741.281455:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826741.281459:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8807 00000100:00080000:0.0:1787826741.281463:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826741.281467:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787826741.281478:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826741.281492:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5180 x1874663043095168/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787826741.281502:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826741.281505:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787826741.281508:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826741.281512:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8807 00000100:00080000:0.0:1787826741.281516:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826741.281520:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787826741.281540:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043095040:192.168.203.147@tcp:250:kworker.0 00000008:00100000:0.0:1787826741.281543:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826741.281567:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5500 x1874663043095296/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787826741.281730:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826741.281812:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826741/real 1787826741] req@ffff9181c55a4380 x1874663043095040/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826766 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826741.281830:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4380 x1874663043095040/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826766 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826741.281841:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826741.281845:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826741.281869:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043095040:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787826741.281877:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043095168:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826741.281903:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826741.281917:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826741/real 1787826741] req@ffff9181c55a5180 x1874663043095168/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826766 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826741.281928:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5180 x1874663043095168/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826766 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826741.281935:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826741.281938:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826741.281941:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043095168:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826741.281966:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043095296:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826741.281986:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826741.281997:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826741/real 1787826741] req@ffff9181c55a5500 x1874663043095296/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826766 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826741.282009:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5500 x1874663043095296/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826766 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826741.282017:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826741.282020:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826741.282023:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043095296:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787826741.857048:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826742.880981:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826743.904946:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826744.930259:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826745.952949:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787826746.401062:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'params' failed (-108). 10000000:00020000:1.0:1787826746.401078:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00080000:2.0:1787826746.401085:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 10000000:01000000:1.0:1787826746.401086:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:1.0:1787826746.401091:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff9181c87b19b8 from 1 00000100:00080000:2.0:1787826746.401092:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 10000000:01000000:1.0:1787826746.401098:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00000100:00080000:2.0:1787826746.401098:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826746.401110:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8813 00000100:00080000:2.0:1787826746.401120:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826746.401129:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787826746.401170:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f480 x1874663043095424/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787826746.401201:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f537 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826746.401214:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f537 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787826746.401218:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826746.401224:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787826746.401226:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00010000:00010000:1.0:1787826746.401231:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f537 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787826746.401232:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8813 00000100:00080000:2.0:1787826746.401239:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826746.401246:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826746.401258:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787826746.401260:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215d6c0 going to sleep for 0 seconds 00000100:00100000:0.0:1787826746.401271:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043095424:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787826746.401278:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043095552:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00100000:2.0:1787826746.401280:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ce00 x1874663043095680/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787826746.401307:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826746.401310:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787826746.401311:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826746.401314:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8813 00010000:00010000:1.0:1787826746.401317:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f537 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787826746.401319:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826746.401326:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:1.0:1787826746.401332:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f537 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000008:00100000:2.0:1787826746.401334:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:1.0:1787826746.401344:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f537 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826746.401352:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007fb80 x1874663043095808/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787826746.401353:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f537 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826746.401373:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f537 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826746.401397:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4c00 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787826746.401401:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f537 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826746.401410:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f537 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787826746.401419:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f537 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826746.401425:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1787826746.401429:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787826746.401436:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f537 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826746.401443:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f537 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826746.401477:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787826746.401480:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:CONNECTING 00000800:00000100:0.0:1787826746.401878:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826746.401916:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826746/real 1787826746] req@ffff9181e007f480 x1874663043095424/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826771 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826746.401936:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f480 x1874663043095424/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826771 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826746.401947:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826746.401968:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826746.401979:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043095424:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787826746.402017:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1787826746.402021:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043095680:192.168.203.147@tcp:38:kworker.0 00000100:00080000:2.0:1787826746.402025:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787826746.402030:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826746.402040:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8818 00000100:00080000:2.0:1787826746.402048:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826746.402054:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:0.0:1787826746.402060:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826746.402078:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ca80 x1874663043095936/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826746.402083:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826746/real 1787826746] req@ffff9181e007ce00 x1874663043095680/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826771 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826746.402125:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ce00 x1874663043095680/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826771 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826746.402137:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826746.402141:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826746.402144:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043095680:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787826746.402152:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043095808:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826746.402174:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826746.402185:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826746/real 1787826746] req@ffff9181e007fb80 x1874663043095808/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826771 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826746.402202:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007fb80 x1874663043095808/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826771 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826746.402214:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826746.402218:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826746.402221:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043095808:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787826746.402236:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043095936:192.168.203.147@tcp:250:kworker.0 00000800:00000100:0.0:1787826746.402254:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826746.402264:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826746/real 1787826746] req@ffff9181e007ca80 x1874663043095936/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826771 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826746.402274:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ca80 x1874663043095936/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826771 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826746.402281:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826746.402284:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826746.402298:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043095936:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787826746.977405:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826748.000917:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826749.024961:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826750.048982:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826751.072983:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787826751.520908:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826751.520921:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787826751.520925:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826751.520937:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8818 00000100:00080000:3.0:1787826751.520949:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826751.520969:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787826751.521016:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043096064/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826751.521047:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826751.521051:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787826751.521052:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826751.521056:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8818 00000100:00080000:3.0:1787826751.521060:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826751.521064:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826751.521075:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826751.521097:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043096192/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826751.521104:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043096064:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787826751.521110:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826751.521115:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787826751.521117:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826751.521121:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8818 00000100:00080000:3.0:1787826751.521126:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826751.521130:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826751.521135:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826751.521149:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043096320/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787826751.521185:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826751.521225:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826751/real 1787826751] req@ffff9181c5635c00 x1874663043096064/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826776 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826751.521287:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043096064/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826776 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826751.521298:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826751.521303:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826751.521496:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043096064:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787826751.521539:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043096192:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826751.521559:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826751.521586:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826751/real 1787826751] req@ffff9181c5637800 x1874663043096192/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826776 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826751.521598:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043096192/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826776 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826751.521606:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826751.521610:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826751.521613:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043096192:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787826751.521619:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043096320:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826751.521647:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826751.521658:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826751/real 1787826751] req@ffff9181c5637100 x1874663043096320/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826776 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826751.521670:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043096320/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826776 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826751.521684:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826751.521686:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826751.521689:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043096320:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787826752.096947:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826753.121664:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826754.144910:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826755.168894:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826756.193183:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787826756.640933:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826756.640941:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787826756.640944:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826756.640952:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8823 00000100:00080000:2.0:1787826756.640961:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826756.640966:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787826756.640997:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e680 x1874663043096448/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787826756.641020:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826756.641023:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787826756.641024:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826756.641027:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8823 00000100:00080000:2.0:1787826756.641029:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826756.641031:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826756.641037:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826756.641044:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007dc00 x1874663043096576/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787826756.641054:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826756.641056:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787826756.641057:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826756.641060:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8823 00000100:00080000:2.0:1787826756.641062:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826756.641064:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826756.641066:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826756.641072:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c000 x1874663043096704/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826756.641089:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043096448:192.168.203.147@tcp:250:kworker.0 00000800:00000100:0.0:1787826756.641147:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826756.641176:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826756/real 1787826756] req@ffff9181e007e680 x1874663043096448/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826781 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826756.641189:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e680 x1874663043096448/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826781 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826756.641196:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826756.641198:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826756.641210:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043096448:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787826756.641215:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043096576:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826756.641230:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826756.641239:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826756/real 1787826756] req@ffff9181e007dc00 x1874663043096576/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826781 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826756.641249:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007dc00 x1874663043096576/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826781 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826756.641255:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826756.641257:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826756.641259:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043096576:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787826756.641265:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043096704:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826756.641288:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826756.641299:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826756/real 1787826756] req@ffff9181e007c000 x1874663043096704/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826781 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826756.641310:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c000 x1874663043096704/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826781 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826756.641318:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826756.641321:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826756.641324:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043096704:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787826757.216892:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826758.240934:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826759.264890:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826760.288924:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826761.312913:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787826761.761982:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826761.761993:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787826761.761997:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826761.762006:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8828 00000100:00080000:3.0:1787826761.762017:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826761.762024:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787826761.762059:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043096832/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826761.762086:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826761.762090:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787826761.762092:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826761.762095:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8828 00000100:00080000:3.0:1787826761.762098:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826761.762102:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826761.762110:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826761.762121:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043096960/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826761.762129:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826761.762131:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787826761.762132:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826761.762135:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8828 00000100:00080000:3.0:1787826761.762137:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826761.762140:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826761.762143:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826761.762150:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043097088/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826761.762166:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043096832:192.168.203.147@tcp:250:kworker.0 00000800:00000100:0.0:1787826761.762246:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826761.762293:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826761/real 1787826761] req@ffff9181c5635500 x1874663043096832/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826786 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826761.762315:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043096832/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826786 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826761.762324:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826761.762327:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826761.762343:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043096832:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787826761.762349:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043096960:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826761.762367:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826761.762377:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826761/real 1787826761] req@ffff9181c5636d80 x1874663043096960/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826786 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826761.762386:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043096960/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826786 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826761.762392:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826761.762394:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826761.762397:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043096960:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787826761.762421:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043097088:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826761.762432:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826761.762438:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826761/real 1787826761] req@ffff9181c5637480 x1874663043097088/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826786 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826761.762445:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043097088/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826786 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826761.762450:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826761.762452:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826761.762454:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043097088:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787826762.336907:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826763.360917:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826764.384950:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826765.408949:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826766.432940:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787826766.880886:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826766.880895:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787826766.880900:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826766.880910:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8833 00000100:00080000:1.0:1787826766.880920:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787826766.880926:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787826766.880959:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2680 x1874663043097216/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787826766.880987:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826766.880990:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787826766.880992:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826766.880997:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8833 00000100:00080000:1.0:1787826766.881000:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787826766.881003:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787826766.881012:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787826766.881022:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0e00 x1874663043097344/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787826766.881033:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826766.881037:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:0.0:1787826766.881038:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043097216:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787826766.881039:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826766.881047:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8833 00000100:00080000:1.0:1787826766.881050:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787826766.881053:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787826766.881057:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787826766.881068:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3480 x1874663043097472/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787826766.881114:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 10000000:01000000:1.0:1787826766.881143:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-sptlrpc' failed (-108). 10000000:00020000:1.0:1787826766.881147:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00100000:0.0:1787826766.881152:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826766/real 1787826766] req@ffff9181c35b2680 x1874663043097216/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826791 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826766.881187:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2680 x1874663043097216/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826791 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826766.881209:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826766.881213:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826766.881217:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043097216:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787826766.881239:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043097344:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826766.881260:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826766.881341:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826766/real 1787826766] req@ffff9181c35b0e00 x1874663043097344/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826791 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826766.881356:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0e00 x1874663043097344/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826791 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826766.881363:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826766.881367:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826766.881370:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043097344:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787826766.881376:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043097472:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826766.881398:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826766.881410:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826766/real 1787826766] req@ffff9181c35b3480 x1874663043097472/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826791 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826766.881421:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3480 x1874663043097472/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826791 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826766.881429:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826766.881432:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826766.881435:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043097472:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787826767.456943:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787826768.032854:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:1.0:1787826768.032862:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff9181c5620000 from 341 10000000:01000000:1.0:1787826768.032867:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:1.0:1787826768.032912:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f53e lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826768.032923:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f53e lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826768.032938:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f53e lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826768.032958:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215d6c0 going to sleep for 0 seconds 00000100:00100000:1.0:1787826768.032975:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043097600:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787826768.032990:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f53e lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826768.032998:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f53e lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826768.033004:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f53e lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826768.033011:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f53e lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826768.033022:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f53e lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826768.033028:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4a00 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787826768.033031:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f53e lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826768.033039:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f53e lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787826768.033045:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f53e lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826768.033051:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:1.0:1787826768.033054:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787826768.033060:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f53e lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826768.033067:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f53e lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826768.033088:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787826768.033091:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:0.0:1787826768.033158:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826768.033164:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787826768.033166:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826768.033171:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8838 00000100:00080000:0.0:1787826768.033178:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826768.033183:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787826768.033205:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7800 x1874663043097728/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787826768.033225:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826768.033229:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787826768.033230:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826768.033233:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8838 00000100:00080000:0.0:1787826768.033237:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826768.033241:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787826768.033249:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826768.033266:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4e00 x1874663043097856/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787826768.033277:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1787826768.033280:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043097728:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787826768.033281:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787826768.033285:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826768.033292:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8838 00000100:00080000:0.0:1787826768.033295:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826768.033298:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787826768.033302:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826768.033312:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5500 x1874663043097984/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787826768.033347:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826768.033376:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826768/real 1787826768] req@ffff9181c55a7800 x1874663043097728/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826793 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826768.033390:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7800 x1874663043097728/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826793 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826768.033399:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826768.033403:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826768.033419:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043097728:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787826768.033438:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043097856:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826768.033458:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826768.033469:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826768/real 1787826768] req@ffff9181c55a4e00 x1874663043097856/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826793 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826768.033480:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4e00 x1874663043097856/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826793 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826768.033487:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826768.033489:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826768.033492:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043097856:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826768.033497:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043097984:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826768.033513:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826768.033523:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826768/real 1787826768] req@ffff9181c55a5500 x1874663043097984/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826793 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826768.033540:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5500 x1874663043097984/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826793 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826768.033546:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826768.033548:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826768.033550:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043097984:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787826768.480930:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826769.504914:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826770.528927:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826771.552899:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826772.576923:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787826773.537003:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826773.537012:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787826773.537016:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826773.537027:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8839 00000100:00080000:1.0:1787826773.537037:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787826773.537045:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787826773.537079:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0a80 x1874663043098112/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787826773.537123:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826773.537128:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787826773.537130:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826773.537135:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8839 00000100:00080000:1.0:1787826773.537139:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787826773.537144:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787826773.537165:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787826773.537181:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2d80 x1874663043098240/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787826773.537192:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826773.537196:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787826773.537218:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787826773.537220:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043098112:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787826773.537227:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8839 00000100:00080000:1.0:1787826773.537233:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787826773.537238:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787826773.537245:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787826773.537259:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2a00 x1874663043098368/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826773.537621:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043098240:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787826773.537673:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043098368:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787826773.601058:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826774.625774:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826775.648915:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1787826776.609034:0:2378:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:0.0:1787826776.609077:0:2378:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1787826776.609094:0:2378:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1787826776.609137:0:2378:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1787826776.609148:0:2378:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:3.0:1787826776.609271:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826773/real 1787826776] req@ffff9181c35b2a00 x1874663043098368/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826798 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826776.609306:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2a00 x1874663043098368/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826798 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826776.609320:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826776.609326:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826776.609330:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043098368:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787826776.609339:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826773/real 1787826776] req@ffff9181c35b0a80 x1874663043098112/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826798 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826776.609352:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0a80 x1874663043098112/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826798 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826776.609360:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826776.609363:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826776.609387:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043098112:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787826776.609395:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826773/real 1787826776] req@ffff9181c35b2d80 x1874663043098240/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826798 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826776.609406:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2d80 x1874663043098240/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826798 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826776.609412:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826776.609417:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826776.609420:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043098240:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787826776.672974:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826777.696949:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787826778.656941:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826778.656948:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787826778.656951:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826778.656959:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8845 00000100:00080000:0.0:1787826778.656969:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826778.656974:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787826778.657000:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5180 x1874663043098496/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787826778.657026:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826778.657030:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787826778.657032:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826778.657036:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8845 00000100:00080000:0.0:1787826778.657039:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826778.657043:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787826778.657052:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826778.657066:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4380 x1874663043098624/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787826778.657078:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826778.657081:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787826778.657084:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787826778.657086:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043098496:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787826778.657091:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8845 00000100:00080000:0.0:1787826778.657096:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826778.657100:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787826778.657105:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826778.657115:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4700 x1874663043098752/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787826778.657158:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826778.657196:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826778/real 1787826778] req@ffff9181c55a5180 x1874663043098496/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826803 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826778.657214:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5180 x1874663043098496/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826803 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826778.657225:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826778.657229:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826778.657245:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043098496:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787826778.657270:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043098624:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787826778.657295:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826778.657308:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826778/real 1787826778] req@ffff9181c55a4380 x1874663043098624/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826803 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826778.657322:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4380 x1874663043098624/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826803 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826778.657331:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826778.657335:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826778.657354:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043098624:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787826778.657360:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043098752:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787826778.657380:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826778.657391:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826778/real 1787826778] req@ffff9181c55a4700 x1874663043098752/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826803 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826778.657404:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4700 x1874663043098752/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826803 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826778.657412:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826778.657415:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826778.657418:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043098752:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787826778.720931:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826779.744971:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826780.768956:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826781.792924:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826782.816899:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787826783.777343:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826783.777354:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787826783.777358:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826783.777367:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8850 00000100:00080000:1.0:1787826783.777376:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787826783.777382:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787826783.777415:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0700 x1874663043098880/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787826783.777447:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826783.777451:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787826783.777452:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826783.777457:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8850 00000100:00080000:1.0:1787826783.777460:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787826783.777464:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787826783.777473:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787826783.777485:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1f80 x1874663043099008/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787826783.777495:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826783.777498:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787826783.777499:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826783.777503:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8850 00000100:00080000:1.0:1787826783.777506:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787826783.777510:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787826783.777514:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787826783.777524:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3800 x1874663043099136/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826783.777583:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043098880:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1787826783.777652:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826783.777686:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826783/real 1787826783] req@ffff9181c35b0700 x1874663043098880/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826808 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826783.777699:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0700 x1874663043098880/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826808 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826783.777705:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826783.777708:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826783.777718:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043098880:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787826783.777726:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043099008:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787826783.777789:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826783.777802:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826783/real 1787826783] req@ffff9181c35b1f80 x1874663043099008/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826808 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826783.777811:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1f80 x1874663043099008/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826808 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826783.777816:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826783.777818:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826783.777820:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043099008:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787826783.777824:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043099136:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787826783.777844:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826783.777856:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826783/real 1787826783] req@ffff9181c35b3800 x1874663043099136/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826808 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826783.777870:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3800 x1874663043099136/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826808 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826783.777883:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826783.777886:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826783.777889:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043099136:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787826783.841084:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826784.864931:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826785.889616:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826786.912951:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826787.937172:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787826788.384908:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-cliir' failed (-108). 10000000:00000400:1.0:1787826788.384918:0:159735:0:(mgc_request.c:1900:mgc_process_log()) MGC192.168.203.147@tcp: IR log lustre-cliir failed, not fatal: rc = -108 10000000:01000000:1.0:1787826788.392972:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:1.0:1787826788.392978:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff9181c5620000 from 52 10000000:01000000:1.0:1787826788.392983:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:1.0:1787826788.393028:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f545 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826788.393039:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f545 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826788.393049:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f545 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826788.393068:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215d6c0 going to sleep for 0 seconds 00000100:00100000:1.0:1787826788.393077:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043099264:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787826788.393090:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f545 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826788.393098:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f545 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826788.393104:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f545 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826788.393111:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f545 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826788.393118:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f545 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826788.393125:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b5000 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787826788.393128:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f545 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826788.393134:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f545 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787826788.393141:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f545 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826788.393147:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1787826788.393150:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787826788.393156:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f545 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826788.393162:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f545 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826788.393181:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787826788.393184:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:0.0:1787826788.393286:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826788.393291:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787826788.393294:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826788.393299:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8855 00000100:00080000:0.0:1787826788.393308:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826788.393313:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787826788.393330:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5f80 x1874663043099392/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787826788.393359:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826788.393363:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787826788.393365:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826788.393369:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8855 00000100:00080000:0.0:1787826788.393372:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826788.393376:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787826788.393383:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826788.393394:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5880 x1874663043099520/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787826788.393402:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826788.393404:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787826788.393406:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826788.393409:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8855 00000100:00080000:0.0:1787826788.393412:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826788.393415:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787826788.393419:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826788.393426:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6d80 x1874663043099648/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826788.393513:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043099392:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1787826788.393566:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826788.393591:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826788/real 1787826788] req@ffff9181c55a5f80 x1874663043099392/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826813 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826788.393603:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5f80 x1874663043099392/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826813 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826788.393612:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826788.393616:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826788.393631:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043099392:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787826788.393637:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043099520:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787826788.393656:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826788.393667:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826788/real 1787826788] req@ffff9181c55a5880 x1874663043099520/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826813 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826788.393677:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5880 x1874663043099520/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826813 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826788.393684:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826788.393687:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826788.393690:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043099520:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787826788.393706:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043099648:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787826788.393723:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826788.393733:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826788/real 1787826788] req@ffff9181c55a6d80 x1874663043099648/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826813 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826788.393771:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6d80 x1874663043099648/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826813 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826788.393779:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826788.393782:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826788.393785:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043099648:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787826788.960920:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826789.985039:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826791.008931:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826792.032935:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826793.056930:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787826793.504932:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826793.504940:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787826793.504943:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826793.504952:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8860 00000100:00080000:3.0:1787826793.504965:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826793.504972:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787826793.505006:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043099776/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826793.505036:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826793.505040:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787826793.505042:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826793.505046:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8860 00000100:00080000:3.0:1787826793.505050:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826793.505054:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826793.505063:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826793.505075:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634700 x1874663043099904/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826793.505083:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1787826793.505084:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043099776:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787826793.505087:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787826793.505090:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826793.505095:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8860 00000100:00080000:3.0:1787826793.505097:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826793.505100:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826793.505103:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826793.505110:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043100032/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787826793.505154:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826793.505188:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826793/real 1787826793] req@ffff9181c5636a00 x1874663043099776/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826818 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826793.505206:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043099776/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826818 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826793.505217:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826793.505221:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826793.505244:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043099776:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787826793.505270:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043099904:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826793.505295:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826793.505308:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826793/real 1787826793] req@ffff9181c5634700 x1874663043099904/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826818 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826793.505320:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634700 x1874663043099904/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826818 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826793.505328:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826793.505332:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826793.505335:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043099904:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826793.505352:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043100032:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826793.505372:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826793.505383:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826793/real 1787826793] req@ffff9181c5634380 x1874663043100032/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826818 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826793.505394:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043100032/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826818 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826793.505401:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826793.505404:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826793.505407:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043100032:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787826794.080873:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826795.105006:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826796.129835:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826797.153066:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826798.176903:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787826798.624919:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826798.624929:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787826798.624933:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826798.624943:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8865 00000100:00080000:0.0:1787826798.624955:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826798.624962:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787826798.624999:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5c00 x1874663043100160/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787826798.625035:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826798.625038:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787826798.625040:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826798.625044:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8865 00000100:00080000:0.0:1787826798.625048:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826798.625052:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787826798.625061:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826798.625083:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7480 x1874663043100288/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787826798.625096:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826798.625099:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787826798.625101:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826798.625104:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8865 00000100:00080000:0.0:1787826798.625107:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826798.625111:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787826798.625116:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826798.625126:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7b80 x1874663043100416/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826798.625251:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043100160:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787826798.625309:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826798.625341:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826798/real 1787826798] req@ffff9181c55a5c00 x1874663043100160/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826823 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826798.625358:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5c00 x1874663043100160/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826823 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826798.625368:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826798.625372:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826798.625387:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043100160:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787826798.625395:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043100288:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826798.625416:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826798.625430:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826798/real 1787826798] req@ffff9181c55a7480 x1874663043100288/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826823 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826798.625442:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7480 x1874663043100288/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826823 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826798.625450:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826798.625454:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826798.625457:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043100288:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826798.625462:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043100416:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826798.625479:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826798.625489:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826798/real 1787826798] req@ffff9181c55a7b80 x1874663043100416/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826823 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826798.625500:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7b80 x1874663043100416/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826823 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826798.625507:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826798.625510:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826798.625512:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043100416:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787826799.200921:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826800.225003:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826801.248888:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826802.272920:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826803.296886:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787826803.744848:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826803.744856:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787826803.744858:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826803.744869:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8870 00000100:00080000:3.0:1787826803.744877:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826803.744883:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787826803.744913:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634e00 x1874663043100544/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826803.744931:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826803.744933:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787826803.744935:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826803.744938:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8870 00000100:00080000:3.0:1787826803.744940:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826803.744942:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826803.744949:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826803.744957:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635f80 x1874663043100672/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826803.744964:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826803.744966:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787826803.744967:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1787826803.744971:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043100544:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787826803.744972:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8870 00000100:00080000:3.0:1787826803.744976:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826803.744981:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826803.744984:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826803.744992:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043100800/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787826803.745030:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826803.745065:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826803/real 1787826803] req@ffff9181c5634e00 x1874663043100544/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826828 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826803.745082:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634e00 x1874663043100544/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826828 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826803.745093:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826803.745096:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826803.745111:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043100544:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787826803.745134:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043100672:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826803.745157:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826803.745176:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826803/real 1787826803] req@ffff9181c5635f80 x1874663043100672/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826828 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826803.745188:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635f80 x1874663043100672/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826828 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826803.745201:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826803.745204:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826803.745208:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043100672:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826803.745213:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043100800:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826803.745232:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826803.745246:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826803/real 1787826803] req@ffff9181c5634380 x1874663043100800/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826828 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826803.745254:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043100800/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826828 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826803.745259:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826803.745261:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826803.745262:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043100800:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787826804.320897:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826805.344996:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826806.368923:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826807.392915:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826808.416894:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787826808.864858:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826808.864866:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787826808.864869:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826808.864878:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8875 00000100:00080000:0.0:1787826808.864888:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826808.864893:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 10000000:01000000:1.0:1787826808.864897:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-client' failed (-108). 10000000:00020000:1.0:1787826808.864909:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:1.0:1787826808.864914:0:159735:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:1.0:1787826808.864917:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff9181c5620000 from 7 00000100:00100000:0.0:1787826808.864920:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6680 x1874663043100928/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 10000000:01000000:1.0:1787826808.864921:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00000100:00080000:0.0:1787826808.864945:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826808.864948:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787826808.864950:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826808.864952:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8875 00000100:00080000:0.0:1787826808.864955:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826808.864958:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:1.0:1787826808.864959:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6f54c lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000008:00100000:0.0:1787826808.864965:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:1.0:1787826808.864968:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6f54c lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826808.864976:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7100 x1874663043101184/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787826808.864979:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6f54c lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826808.864990:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043100928:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787826808.864994:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826808.864999:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787826808.865000:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1787826808.865005:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215d6c0 going to sleep for 0 seconds 00000100:00080000:0.0:1787826808.865007:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8875 00000100:00080000:0.0:1787826808.865012:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1787826808.865014:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043101056:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00080000:0.0:1787826808.865019:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787826808.865026:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:1.0:1787826808.865028:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6f54c lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826808.865035:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6f54c lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826808.865041:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7b80 x1874663043101312/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787826808.865042:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6f54c lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826808.865060:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6f54c lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000800:00000100:2.0:1787826808.865060:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00010000:00010000:1.0:1787826808.865072:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6f54c lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826808.865079:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b5400 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787826808.865083:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6f54c lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826808.865092:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6f54c lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826808.865099:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826808/real 1787826808] req@ffff9181c55a6680 x1874663043100928/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826833 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 10000000:00010000:1.0:1787826808.865112:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6f54c lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826808.865119:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00000100:00100000:2.0:1787826808.865123:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6680 x1874663043100928/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826833 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:1.0:1787826808.865124:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787826808.865132:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6f54c lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826808.865139:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6f54c lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787826808.865156:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826808.865161:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826808.865165:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043100928:192.168.203.147@tcp:250:kworker.0 10000000:01000000:1.0:1787826808.865170:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787826808.865173:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00100000:2.0:1787826808.865186:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043101184:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826808.865212:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:0.0:1787826808.865217:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826808.865223:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787826808.865225:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1787826808.865229:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826808/real 1787826808] req@ffff9181c55a7100 x1874663043101184/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826833 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826808.865233:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8880 00000100:00080000:0.0:1787826808.865240:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826808.865246:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787826808.865251:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7100 x1874663043101184/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826833 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826808.865260:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:0.0:1787826808.865264:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7480 x1874663043101440/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787826808.865267:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826808.865276:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043101184:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826808.865289:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043101312:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826808.865315:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826808.865327:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826808/real 1787826808] req@ffff9181c55a7b80 x1874663043101312/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826833 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826808.865337:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7b80 x1874663043101312/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826833 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826808.865344:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826808.865347:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826808.865350:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043101312:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826808.865364:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043101440:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787826808.865381:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826808.865392:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826808/real 1787826808] req@ffff9181c55a7480 x1874663043101440/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826833 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826808.865399:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7480 x1874663043101440/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826833 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826808.865403:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826808.865405:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826808.865414:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043101440:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787826809.440976:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826810.464901:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826811.488931:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826812.513085:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826813.536950:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787826813.985016:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826813.985025:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787826813.985028:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826813.985038:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8880 00000100:00080000:0.0:1787826813.985048:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826813.985056:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787826813.985089:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5c00 x1874663043101568/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787826813.985116:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826813.985120:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787826813.985122:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826813.985125:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8880 00000100:00080000:0.0:1787826813.985128:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826813.985132:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787826813.985140:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826813.985151:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6d80 x1874663043101696/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787826813.985160:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826813.985163:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787826813.985164:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1787826813.985169:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043101568:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787826813.985170:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8880 00000100:00080000:0.0:1787826813.985175:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826813.985180:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787826813.985185:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826813.985193:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5880 x1874663043101824/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787826813.985243:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826813.985271:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826813/real 1787826813] req@ffff9181c55a5c00 x1874663043101568/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826838 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826813.985293:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5c00 x1874663043101568/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826838 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826813.985303:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826813.985307:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826813.985321:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043101568:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787826813.985344:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043101696:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826813.985363:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826813.985376:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826813/real 1787826813] req@ffff9181c55a6d80 x1874663043101696/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826838 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826813.985389:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6d80 x1874663043101696/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826838 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826813.985401:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826813.985404:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826813.985408:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043101696:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826813.985413:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043101824:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826813.985432:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826813.985441:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826813/real 1787826813] req@ffff9181c55a5880 x1874663043101824/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826838 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826813.985448:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5880 x1874663043101824/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826838 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826813.985453:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826813.985455:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826813.985457:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043101824:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787826814.560909:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826815.584909:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826816.608882:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826817.632931:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826818.656953:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787826819.104875:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826819.104886:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787826819.104891:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826819.104902:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8885 00000100:00080000:3.0:1787826819.104913:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826819.104920:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787826819.104956:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634700 x1874663043101952/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826819.104991:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826819.104994:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787826819.104996:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826819.105000:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8885 00000100:00080000:3.0:1787826819.105004:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826819.105008:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826819.105017:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826819.105029:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043102080/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826819.105040:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1787826819.105041:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043101952:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787826819.105046:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787826819.105050:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826819.105055:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8885 00000100:00080000:3.0:1787826819.105059:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826819.105063:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826819.105068:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826819.105080:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043102208/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787826819.105100:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826819.105127:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826819/real 1787826819] req@ffff9181c5634700 x1874663043101952/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826844 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826819.105138:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634700 x1874663043101952/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826844 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826819.105145:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826819.105147:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826819.105160:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043101952:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787826819.105182:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043102080:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826819.105208:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826819.105221:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826819/real 1787826819] req@ffff9181c5636a00 x1874663043102080/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826844 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826819.105233:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043102080/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826844 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826819.105240:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826819.105243:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826819.105246:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043102080:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826819.105251:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043102208:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826819.105269:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826819.105280:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826819/real 1787826819] req@ffff9181c5637480 x1874663043102208/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826844 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826819.105291:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043102208/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826844 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826819.105298:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826819.105300:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826819.105303:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043102208:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787826819.680925:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826820.705001:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826821.728991:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826822.752926:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826823.776975:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787826824.224880:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826824.224894:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787826824.224898:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826824.224907:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8890 00000100:00080000:0.0:1787826824.224917:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826824.224923:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787826824.224967:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5f80 x1874663043102336/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787826824.224989:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826824.224993:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787826824.224994:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826824.224998:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8890 00000100:00080000:0.0:1787826824.225002:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826824.225011:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787826824.225023:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043102336:192.168.203.147@tcp:250:kworker.0 00000008:00100000:0.0:1787826824.225024:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826824.225040:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4700 x1874663043102464/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787826824.225050:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826824.225053:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787826824.225055:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826824.225059:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8890 00000100:00080000:0.0:1787826824.225063:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826824.225066:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787826824.225071:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826824.225081:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4380 x1874663043102592/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787826824.225085:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826824.225120:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826824/real 1787826824] req@ffff9181c55a5f80 x1874663043102336/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826849 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826824.225138:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5f80 x1874663043102336/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826849 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826824.225147:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826824.225151:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826824.225164:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043102336:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787826824.225186:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043102464:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826824.225207:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826824.225218:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826824/real 1787826824] req@ffff9181c55a4700 x1874663043102464/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826849 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826824.225230:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4700 x1874663043102464/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826849 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826824.225237:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826824.225240:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826824.225243:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043102464:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826824.225248:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043102592:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826824.225262:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826824.225269:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826824/real 1787826824] req@ffff9181c55a4380 x1874663043102592/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826849 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826824.225280:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4380 x1874663043102592/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826849 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826824.225293:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826824.225296:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826824.225298:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043102592:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787826824.800916:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826825.825158:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826826.848917:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826827.872949:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826828.896954:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787826829.344972:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'params' failed (-108). 10000000:00020000:1.0:1787826829.344983:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:1.0:1787826829.344988:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:1.0:1787826829.344991:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff9181c87b19b8 from 1 10000000:01000000:1.0:1787826829.344994:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00000100:00080000:0.0:1787826829.344998:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826829.345004:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787826829.345008:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826829.345017:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8895 00000100:00080000:0.0:1787826829.345026:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826829.345033:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787826829.345057:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5180 x1874663043102848/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787826829.345063:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5200/0xbef42c0221d6f553 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826829.345076:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b5200/0xbef42c0221d6f553 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826829.345088:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b5200/0xbef42c0221d6f553 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787826829.345094:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826829.345098:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787826829.345100:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826829.345106:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8895 00000100:00100000:1.0:1787826829.345107:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215d6c0 going to sleep for 0 seconds 00000100:00080000:0.0:1787826829.345110:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826829.345116:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787826829.345119:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043102720:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000008:00100000:0.0:1787826829.345126:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:1.0:1787826829.345133:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5200/0xbef42c0221d6f553 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826829.345141:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5500 x1874663043102976/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787826829.345143:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b5200/0xbef42c0221d6f553 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826829.345163:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5200/0xbef42c0221d6f553 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787826829.345164:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826829.345174:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787826829.345177:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826829.345185:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8895 00010000:00010000:1.0:1787826829.345188:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b5200/0xbef42c0221d6f553 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787826829.345190:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826829.345198:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:1.0:1787826829.345202:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b5200/0xbef42c0221d6f553 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000008:00100000:0.0:1787826829.345205:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826829.345223:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4e00 x1874663043103104/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787826829.345269:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b5200 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787826829.345274:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5200/0xbef42c0221d6f553 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826829.345282:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b5200/0xbef42c0221d6f553 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787826829.345290:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5200/0xbef42c0221d6f553 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826829.345297:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1787826829.345300:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787826829.345306:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b5200/0xbef42c0221d6f553 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826829.345313:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b5200/0xbef42c0221d6f553 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826829.345335:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787826829.345337:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:CONNECTING 00000100:00100000:2.0:1787826829.345468:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043102848:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787826829.345530:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826829.345560:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826829/real 1787826829] req@ffff9181c55a5180 x1874663043102848/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826854 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826829.345576:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5180 x1874663043102848/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826854 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826829.345586:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826829.345602:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826829.345618:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043102848:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787826829.345625:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043102976:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826829.345648:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826829.345661:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826829/real 1787826829] req@ffff9181c55a5500 x1874663043102976/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826854 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826829.345674:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5500 x1874663043102976/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826854 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826829.345682:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826829.345686:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826829.345690:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043102976:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826829.345698:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043103104:192.168.203.147@tcp:38:kworker.0 00000100:00080000:0.0:1787826829.345703:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826829.345706:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787826829.345708:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826829.345712:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8901 00000100:00080000:0.0:1787826829.345715:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826829.345719:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:2.0:1787826829.345729:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826829.345732:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7800 x1874663043103232/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826829.345777:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826829/real 1787826829] req@ffff9181c55a4e00 x1874663043103104/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826854 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826829.345793:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4e00 x1874663043103104/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826854 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826829.345803:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826829.345806:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826829.345810:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043103104:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826829.345831:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043103232:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787826829.345851:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826829.345864:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826829/real 1787826829] req@ffff9181c55a7800 x1874663043103232/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826854 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826829.345876:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7800 x1874663043103232/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826854 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826829.345884:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826829.345887:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826829.345890:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043103232:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787826829.920908:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826830.944966:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826831.969006:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826832.993039:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826834.016991:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787826834.464915:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826834.464926:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787826834.464930:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826834.464940:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8901 00000100:00080000:2.0:1787826834.464949:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826834.464956:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787826834.464987:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c380 x1874663043103360/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787826834.465022:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826834.465026:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787826834.465027:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826834.465031:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8901 00000100:00080000:2.0:1787826834.465034:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826834.465037:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826834.465045:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826834.465058:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c000 x1874663043103488/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787826834.465068:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826834.465072:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787826834.465073:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826834.465077:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8901 00000100:00080000:2.0:1787826834.465079:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826834.465083:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826834.465088:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826834.465095:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043103360:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787826834.465103:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007dc00 x1874663043103616/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787826834.465192:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826834.465240:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826834/real 1787826834] req@ffff9181e007c380 x1874663043103360/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826859 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826834.465267:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c380 x1874663043103360/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826859 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826834.465278:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826834.465282:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826834.465299:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043103360:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787826834.465308:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043103488:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787826834.465333:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826834.465348:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826834/real 1787826834] req@ffff9181e007c000 x1874663043103488/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826859 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826834.465356:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c000 x1874663043103488/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826859 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826834.465360:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826834.465362:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826834.465365:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043103488:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787826834.465401:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043103616:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787826834.465428:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826834.465439:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826834/real 1787826834] req@ffff9181e007dc00 x1874663043103616/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826859 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826834.465449:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007dc00 x1874663043103616/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826859 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826834.465462:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826834.465465:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826834.465468:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043103616:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787826835.040961:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826836.065014:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826837.088922:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826838.112920:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826839.136898:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787826839.584867:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826839.584875:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787826839.584878:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826839.584886:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8906 00000100:00080000:3.0:1787826839.584895:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826839.584900:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787826839.584934:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043103744/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826839.584960:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826839.584963:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787826839.584965:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826839.584969:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8906 00000100:00080000:3.0:1787826839.584972:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826839.584977:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826839.584987:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826839.585000:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043103744:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787826839.585000:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043103872/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826839.585027:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826839.585030:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787826839.585032:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826839.585037:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8906 00000100:00080000:3.0:1787826839.585048:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826839.585052:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826839.585057:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826839.585069:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043104000/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826839.585103:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043103872:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826839.585136:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043104000:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1787826840.161145:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826841.185537:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826842.208918:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1787826842.657589:0:2379:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:0.0:1787826842.657626:0:2379:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1787826842.657642:0:2379:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1787826842.657674:0:2379:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1787826842.657682:0:2379:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:2.0:1787826842.657719:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826839/real 1787826842] req@ffff9181c5635500 x1874663043103872/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826864 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826842.657779:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043103872/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826864 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826842.657792:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826842.657796:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826842.657800:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043103872:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826842.657808:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826839/real 1787826842] req@ffff9181c5637100 x1874663043104000/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826864 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826842.657828:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043104000/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826864 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826842.657836:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826842.657840:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826842.657844:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043104000:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826842.657862:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826839/real 1787826842] req@ffff9181c5636d80 x1874663043103744/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826864 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826842.657879:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043103744/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826864 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826842.657887:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826842.657890:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826842.657910:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043103744:192.168.203.147@tcp:250:kworker.0 00010000:00010000:0.0:1787826843.233645:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826844.257409:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787826844.704836:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826844.704845:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787826844.704849:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826844.704859:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8911 00000100:00080000:3.0:1787826844.704868:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826844.704875:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787826844.704905:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043104128/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826844.704930:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826844.704934:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787826844.704935:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826844.704939:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8911 00000100:00080000:3.0:1787826844.704942:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826844.704946:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826844.704954:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826844.704966:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043104256/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826844.704980:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043104128:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787826844.704983:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826844.704990:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787826844.704991:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826844.704996:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8911 00000100:00080000:3.0:1787826844.704999:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826844.705003:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826844.705007:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826844.705030:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043104384/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787826844.705059:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826844.705094:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826844/real 1787826844] req@ffff9181c5637800 x1874663043104128/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826869 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826844.705110:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043104128/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826869 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826844.705120:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826844.705124:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826844.705141:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043104128:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787826844.705163:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043104256:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826844.705185:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826844.705197:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826844/real 1787826844] req@ffff9181c5635c00 x1874663043104256/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826869 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826844.705208:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043104256/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826869 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826844.705216:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826844.705219:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826844.705222:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043104256:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826844.705228:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043104384:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826844.705247:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826844.705257:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826844/real 1787826844] req@ffff9181c5634000 x1874663043104384/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826869 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826844.705268:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043104384/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826869 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826844.705275:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826844.705279:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826844.705282:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043104384:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1787826845.280874:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826846.305019:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826847.328885:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826848.352905:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826849.376914:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787826849.824847:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826849.824855:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787826849.824858:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826849.824866:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8916 00000100:00080000:3.0:1787826849.824874:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826849.824879:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787826849.824909:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637b80 x1874663043104512/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826849.824931:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826849.824934:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787826849.824936:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 10000000:01000000:1.0:1787826849.824940:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-sptlrpc' failed (-108). 00000100:00080000:3.0:1787826849.824941:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8916 10000000:00020000:1.0:1787826849.824952:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00080000:3.0:1787826849.824953:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826849.824961:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826849.824972:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826849.824987:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634a80 x1874663043104640/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826849.824991:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043104512:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787826849.825006:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826849.825008:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787826849.825010:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826849.825013:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8916 00000100:00080000:3.0:1787826849.825016:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826849.825019:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826849.825023:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826849.825034:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043104768/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787826849.825067:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826849.825101:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826849/real 1787826849] req@ffff9181c5637b80 x1874663043104512/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826874 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826849.825117:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637b80 x1874663043104512/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826874 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826849.825125:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826849.825129:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826849.825132:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043104512:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787826849.825153:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043104640:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826849.825175:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826849.825187:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826849/real 1787826849] req@ffff9181c5634a80 x1874663043104640/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826874 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826849.825195:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634a80 x1874663043104640/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826874 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826849.825200:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826849.825202:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826849.825204:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043104640:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826849.825208:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043104768:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826849.825226:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826849.825236:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826849/real 1787826849] req@ffff9181c5634000 x1874663043104768/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826874 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826849.825248:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043104768/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826874 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826849.825256:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826849.825259:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826849.825262:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043104768:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1787826850.400860:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787826851.424887:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:1.0:1787826851.424895:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff9181c5620000 from 341 10000000:01000000:1.0:1787826851.424901:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:0.0:1787826851.424914:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826851.424966:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f55a lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826851.424977:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f55a lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826851.424987:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f55a lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826851.425006:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215d6c0 going to sleep for 0 seconds 00000100:00100000:1.0:1787826851.425017:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043104896:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787826851.425036:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f55a lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826851.425045:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f55a lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826851.425052:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f55a lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826851.425060:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f55a lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826851.425067:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f55a lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826851.425075:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b5800 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787826851.425079:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f55a lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826851.425087:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f55a lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787826851.425094:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f55a lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826851.425101:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:1.0:1787826851.425105:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787826851.425112:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f55a lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826851.425116:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f55a lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826851.425136:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787826851.425138:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:3.0:1787826851.425197:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826851.425202:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787826851.425205:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826851.425211:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8921 00000100:00080000:3.0:1787826851.425219:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826851.425225:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787826851.425243:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043105024/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826851.425273:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826851.425280:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787826851.425281:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826851.425287:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8921 00000100:00080000:3.0:1787826851.425291:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826851.425295:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826851.425307:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826851.425316:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043105024:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787826851.425321:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043105152/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826851.425329:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826851.425333:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787826851.425334:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826851.425338:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8921 00000100:00080000:3.0:1787826851.425341:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826851.425344:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826851.425348:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826851.425356:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043105280/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787826851.425395:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826851.425427:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826851/real 1787826851] req@ffff9181c5635c00 x1874663043105024/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826876 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826851.425443:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043105024/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826876 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826851.425450:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826851.425453:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826851.425463:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043105024:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787826851.425485:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043105152:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826851.425501:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826851.425509:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826851/real 1787826851] req@ffff9181c5637800 x1874663043105152/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826876 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826851.425517:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043105152/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826876 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826851.425522:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826851.425524:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826851.425526:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043105152:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826851.425529:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043105280:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826851.425538:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826851.425543:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826851/real 1787826851] req@ffff9181c5636d80 x1874663043105280/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826876 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826851.425550:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043105280/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826876 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826851.425559:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826851.425561:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826851.425563:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043105280:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1787826852.448890:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826853.472934:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826854.496891:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826855.521122:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787826856.480794:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826856.480803:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787826856.480807:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826856.480818:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8923 00000100:00080000:3.0:1787826856.480828:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826856.480835:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787826856.480869:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043105408/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826856.480900:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826856.480905:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787826856.480907:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826856.480911:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8923 00000100:00080000:3.0:1787826856.480914:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826856.480918:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826856.480927:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826856.480939:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043105536/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826856.480948:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826856.480952:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787826856.480954:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826856.480957:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8923 00000100:00080000:3.0:1787826856.480960:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826856.480964:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826856.480968:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826856.480977:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043105664/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826856.481063:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043105408:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787826856.481121:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826856.481152:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826856/real 1787826856] req@ffff9181c5637100 x1874663043105408/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826881 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826856.481167:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043105408/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826881 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826856.481176:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826856.481179:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826856.481196:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043105408:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787826856.481203:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043105536:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826856.481236:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826856.481249:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826856/real 1787826856] req@ffff9181c5635500 x1874663043105536/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826881 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826856.481262:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043105536/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826881 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826856.481271:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826856.481275:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826856.481279:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043105536:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826856.481286:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043105664:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826856.481304:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826856.481313:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826856/real 1787826856] req@ffff9181c5637480 x1874663043105664/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826881 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826856.481324:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043105664/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826881 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826856.481332:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826856.481335:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826856.481345:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043105664:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1787826856.544910:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826857.568858:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826858.592871:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826859.616907:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826860.640887:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787826861.600830:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826861.600838:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787826861.600842:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826861.600852:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8928 00000100:00080000:3.0:1787826861.600862:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826861.600870:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787826861.600903:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043105792/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826861.600933:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826861.600936:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787826861.600938:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826861.600942:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8928 00000100:00080000:3.0:1787826861.600945:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826861.600950:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826861.600958:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826861.600971:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634700 x1874663043105920/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787826861.600979:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043105792:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787826861.600983:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826861.600989:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787826861.600990:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826861.600995:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8928 00000100:00080000:3.0:1787826861.600998:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826861.601001:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826861.601006:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826861.601017:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043106048/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787826861.601051:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826861.601086:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826861/real 1787826861] req@ffff9181c5636a00 x1874663043105792/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826886 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826861.601103:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043105792/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826886 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826861.601110:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826861.601113:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826861.601128:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043105792:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787826861.601150:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043105920:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826861.601176:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826861.601202:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826861/real 1787826861] req@ffff9181c5634700 x1874663043105920/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826886 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826861.601214:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634700 x1874663043105920/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826886 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826861.601223:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826861.601226:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826861.601229:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043105920:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787826861.601235:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043106048:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787826861.601255:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787826861.601266:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826861/real 1787826861] req@ffff9181c5634380 x1874663043106048/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826886 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787826861.601278:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043106048/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826886 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787826861.601284:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787826861.601286:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787826861.601288:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043106048:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1787826861.664872:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826862.688985:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826863.712930:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826864.736926:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826865.760869:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787826866.720886:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826866.720898:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787826866.720901:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826866.720912:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8933 00000100:00080000:1.0:1787826866.720923:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787826866.720930:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787826866.720965:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3b80 x1874663043106176/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787826866.720997:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826866.721013:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787826866.721015:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826866.721020:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8933 00000100:00080000:1.0:1787826866.721024:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787826866.721028:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787826866.721032:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043106176:192.168.203.147@tcp:250:kworker.0 00000008:00100000:1.0:1787826866.721039:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787826866.721058:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3800 x1874663043106304/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787826866.721069:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826866.721072:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787826866.721074:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826866.721077:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8933 00000100:00080000:1.0:1787826866.721081:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787826866.721084:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787826866.721090:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1787826866.721090:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787826866.721103:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1f80 x1874663043106432/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826866.721137:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826866/real 1787826866] req@ffff9181c35b3b80 x1874663043106176/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826891 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826866.721173:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3b80 x1874663043106176/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826891 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826866.721183:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826866.721187:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826866.721206:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043106176:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787826866.721232:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043106304:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787826866.721262:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826866.721291:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826866/real 1787826866] req@ffff9181c35b3800 x1874663043106304/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826891 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826866.721309:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3800 x1874663043106304/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826891 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826866.721319:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826866.721323:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826866.721328:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043106304:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787826866.721334:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043106432:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787826866.721355:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826866.721365:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826866/real 1787826866] req@ffff9181c35b1f80 x1874663043106432/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826891 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826866.721376:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1f80 x1874663043106432/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826891 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826866.721389:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826866.721392:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826866.721395:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043106432:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1787826866.785080:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826867.810349:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826868.832897:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826869.856870:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826870.880891:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787826871.840852:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826871.840866:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787826871.840871:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826871.840884:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8938 10000000:01000000:0.0:1787826871.840888:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-cliir' failed (-108). 10000000:00000400:0.0:1787826871.840893:0:159735:0:(mgc_request.c:1900:mgc_process_log()) MGC192.168.203.147@tcp: IR log lustre-cliir failed, not fatal: rc = -108 00000100:00080000:1.0:1787826871.840898:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787826871.840918:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787826871.840964:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0700 x1874663043106560/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787826871.841002:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826871.841006:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787826871.841008:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826871.841013:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8938 00000100:00080000:1.0:1787826871.841017:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787826871.841022:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787826871.841031:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826871.841046:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043106560:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787826871.841047:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2d80 x1874663043106688/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787826871.841063:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826871.841066:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787826871.841068:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826871.841073:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8938 00000100:00080000:1.0:1787826871.841077:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787826871.841081:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787826871.841086:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787826871.841096:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0a80 x1874663043106816/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787826871.841135:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826871.841167:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826871/real 1787826871] req@ffff9181c35b0700 x1874663043106560/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826896 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826871.841195:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0700 x1874663043106560/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826896 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826871.841206:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826871.841210:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826871.841215:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043106560:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787826871.841241:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043106688:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787826871.841264:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826871.841275:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826871/real 1787826871] req@ffff9181c35b2d80 x1874663043106688/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826896 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826871.841283:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2d80 x1874663043106688/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826896 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826871.841288:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826871.841290:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826871.841292:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043106688:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787826871.841297:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043106816:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787826871.841309:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826871.841315:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826871/real 1787826871] req@ffff9181c35b0a80 x1874663043106816/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826896 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826871.841325:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0a80 x1874663043106816/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826896 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826871.841332:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826871.841335:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826871.841338:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043106816:192.168.203.147@tcp:38:kworker.0 10000000:01000000:0.0:1787826871.845018:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:0.0:1787826871.845022:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff9181c5620000 from 52 10000000:01000000:0.0:1787826871.845026:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:0.0:1787826871.845070:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f561 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826871.845078:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f561 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826871.845086:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f561 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826871.845100:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d01a5780 going to sleep for 0 seconds 00000100:00100000:0.0:1787826871.845109:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043106944:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:0.0:1787826871.845133:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f561 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826871.845140:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f561 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826871.845145:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f561 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826871.845149:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f561 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826871.845153:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f561 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826871.845161:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0513400 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:0.0:1787826871.845164:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f561 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826871.845169:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f561 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1787826871.845173:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f561 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787826871.845176:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:0.0:1787826871.845178:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787826871.845182:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f561 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826871.845186:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f561 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787826871.845204:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:0.0:1787826871.845206:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:1.0:1787826871.845294:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826871.845304:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787826871.845307:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826871.845313:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8943 00000100:00080000:1.0:1787826871.845322:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787826871.845327:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787826871.845348:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2a00 x1874663043107072/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826871.845444:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043107072:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1787826871.845533:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826871.845573:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826871/real 1787826871] req@ffff9181c35b2a00 x1874663043107072/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826896 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826871.845592:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2a00 x1874663043107072/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826896 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826871.845603:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826871.845606:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826871.845625:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043107072:192.168.203.147@tcp:250:kworker.0 00010000:00010000:0.0:1787826871.904868:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826872.928878:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826873.952908:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826874.976980:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826876.000888:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787826876.960869:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826876.960877:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787826876.960880:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826876.960893:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8943 00000100:00080000:2.0:1787826876.960903:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826876.960922:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787826876.960959:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e680 x1874663043107200/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787826876.960985:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826876.960989:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787826876.960991:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826876.960995:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8943 00000100:00080000:2.0:1787826876.961006:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826876.961011:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826876.961019:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826876.961034:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ca80 x1874663043107328/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826876.961038:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043107200:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787826876.961050:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826876.961057:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787826876.961059:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826876.961064:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8943 00000100:00080000:2.0:1787826876.961068:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826876.961072:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826876.961077:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826876.961102:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007fb80 x1874663043107456/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787826876.961126:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826876.961161:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826876/real 1787826876] req@ffff9181e007e680 x1874663043107200/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826901 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826876.961178:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e680 x1874663043107200/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826901 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826876.961187:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826876.961191:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826876.961207:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043107200:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787826876.961231:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043107328:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787826876.961257:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826876.961278:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826876/real 1787826876] req@ffff9181e007ca80 x1874663043107328/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826901 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826876.961290:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ca80 x1874663043107328/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826901 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826876.961297:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826876.961300:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826876.961304:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043107328:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787826876.961309:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043107456:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787826876.961329:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826876.961350:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826876/real 1787826876] req@ffff9181e007fb80 x1874663043107456/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826901 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826876.961365:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007fb80 x1874663043107456/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826901 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826876.961378:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826876.961381:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826876.961384:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043107456:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1787826877.024886:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826878.048878:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826879.072917:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826880.096918:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826881.120898:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787826882.080881:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826882.080891:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787826882.080895:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826882.080905:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8948 00000100:00080000:2.0:1787826882.080916:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826882.080923:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787826882.080961:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ce00 x1874663043107584/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787826882.080994:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826882.080998:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787826882.081000:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826882.081005:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8948 00000100:00080000:2.0:1787826882.081008:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826882.081012:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826882.081020:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826882.081035:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f480 x1874663043107712/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787826882.081059:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787826882.081061:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043107584:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787826882.081065:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787826882.081069:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826882.081075:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8948 00000100:00080000:2.0:1787826882.081079:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826882.081083:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826882.081089:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826882.081102:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d880 x1874663043107840/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787826882.081135:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826882.081174:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826882/real 1787826882] req@ffff9181e007ce00 x1874663043107584/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826907 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826882.081192:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ce00 x1874663043107584/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826907 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826882.081202:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826882.081206:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826882.081228:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043107584:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787826882.081250:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043107712:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787826882.081274:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826882.081286:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826882/real 1787826882] req@ffff9181e007f480 x1874663043107712/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826907 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826882.081298:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f480 x1874663043107712/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826907 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826882.081306:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826882.081309:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826882.081312:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043107712:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787826882.081318:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043107840:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787826882.081333:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826882.081342:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826882/real 1787826882] req@ffff9181e007d880 x1874663043107840/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826907 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826882.081353:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d880 x1874663043107840/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826907 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826882.081360:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826882.081363:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826882.081366:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043107840:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787826882.144913:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826883.168885:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826884.192894:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826885.216910:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826886.240897:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787826887.200892:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826887.200901:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787826887.200905:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826887.200915:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8953 00000100:00080000:1.0:1787826887.200925:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787826887.200933:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787826887.200968:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3480 x1874663043107968/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787826887.200997:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826887.201000:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787826887.201002:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826887.201025:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8953 00000100:00080000:1.0:1787826887.201029:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787826887.201034:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787826887.201044:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826887.201057:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043107968:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787826887.201059:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0e00 x1874663043108096/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787826887.201075:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826887.201078:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787826887.201080:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826887.201084:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8953 00000100:00080000:1.0:1787826887.201088:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787826887.201092:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787826887.201098:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787826887.201110:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2680 x1874663043108224/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787826887.201123:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826887.201158:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826887/real 1787826887] req@ffff9181c35b3480 x1874663043107968/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826912 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826887.201175:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3480 x1874663043107968/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826912 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826887.201185:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826887.201190:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826887.201203:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043107968:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787826887.201225:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043108096:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787826887.201252:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826887.201266:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826887/real 1787826887] req@ffff9181c35b0e00 x1874663043108096/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826912 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826887.201278:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0e00 x1874663043108096/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826912 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826887.201285:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826887.201289:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826887.201291:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043108096:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787826887.201297:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043108224:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787826887.201317:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826887.201327:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826887/real 1787826887] req@ffff9181c35b2680 x1874663043108224/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826912 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826887.201337:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2680 x1874663043108224/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826912 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826887.201345:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826887.201348:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826887.201351:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043108224:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787826887.264937:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826888.288930:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826889.313015:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826890.336957:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826891.360961:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787826892.320908:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826892.320918:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 10000000:01000000:0.0:1787826892.320923:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-client' failed (-108). 00000100:00080000:2.0:1787826892.320923:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 10000000:00020000:0.0:1787826892.320937:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00080000:2.0:1787826892.320941:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8958 10000000:01000000:0.0:1787826892.320946:0:159735:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:0.0:1787826892.320951:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff9181c5620000 from 7 00000100:00080000:2.0:1787826892.320954:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 10000000:01000000:0.0:1787826892.320956:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00000100:00080000:2.0:1787826892.320964:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787826892.321016:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f100 x1874663043108480/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:0.0:1787826892.321033:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f568 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826892.321046:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f568 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787826892.321049:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826892.321065:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787826892.321067:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826892.321073:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8958 00000100:00080000:2.0:1787826892.321079:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00010000:00010000:0.0:1787826892.321083:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f568 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787826892.321085:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826892.321101:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826892.321117:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d01a5780 going to sleep for 0 seconds 00000100:00100000:2.0:1787826892.321120:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f800 x1874663043108608/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826892.321134:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043108352:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00080000:2.0:1787826892.321137:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826892.321149:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787826892.321152:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826892.321157:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8958 00010000:00010000:0.0:1787826892.321160:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f568 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787826892.321163:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826892.321174:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826892.321181:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:0.0:1787826892.321184:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f568 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826892.321194:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f568 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826892.321201:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f568 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787826892.321208:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c700 x1874663043108736/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:0.0:1787826892.321215:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f568 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826892.321236:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0513400 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:0.0:1787826892.321240:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f568 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826892.321248:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f568 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1787826892.321257:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f568 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787826892.321263:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:0.0:1787826892.321267:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787826892.321278:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f568 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826892.321285:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f568 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787826892.321309:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:0.0:1787826892.321312:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:CONNECTING 00000100:00100000:3.0:1787826892.321361:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043108480:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1787826892.321423:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826892.321455:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826892/real 1787826892] req@ffff9181e007f100 x1874663043108480/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826917 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826892.321478:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f100 x1874663043108480/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826917 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826892.321492:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826892.321510:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826892.321527:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043108480:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787826892.321535:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043108608:192.168.203.147@tcp:38:kworker.0 00000100:00080000:2.0:1787826892.321536:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826892.321542:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787826892.321544:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826892.321549:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8964 00000100:00080000:2.0:1787826892.321556:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826892.321562:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787826892.321577:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e300 x1874663043108864/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787826892.321837:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826892.321854:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826892/real 1787826892] req@ffff9181e007f800 x1874663043108608/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826917 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826892.321866:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f800 x1874663043108608/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826917 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826892.321874:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826892.321877:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826892.321879:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043108608:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787826892.321885:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043108736:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787826892.321905:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826892.321914:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826892/real 1787826892] req@ffff9181e007c700 x1874663043108736/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826917 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826892.321924:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c700 x1874663043108736/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826917 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826892.321936:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826892.321939:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826892.321942:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043108736:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787826892.321966:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043108864:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1787826892.321996:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826892.322009:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826892/real 1787826892] req@ffff9181e007e300 x1874663043108864/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826917 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826892.322020:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e300 x1874663043108864/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826917 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826892.322026:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826892.322029:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826892.322037:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043108864:192.168.203.147@tcp:250:kworker.0 00010000:00010000:2.0:1787826892.384915:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826893.408955:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826894.434174:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826895.456898:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826896.481019:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787826897.440858:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826897.440867:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787826897.440870:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826897.440877:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8964 00000100:00080000:2.0:1787826897.440886:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826897.440894:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787826897.440928:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ea00 x1874663043108992/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787826897.440968:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826897.440971:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787826897.440972:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826897.440975:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8964 00000100:00080000:2.0:1787826897.440978:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826897.440982:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826897.440988:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826897.440998:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d180 x1874663043109120/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787826897.441004:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826897.441006:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787826897.441007:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826897.441009:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8964 00000100:00080000:2.0:1787826897.441011:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826897.441013:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826897.441016:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826897.441021:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d500 x1874663043109248/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826897.441103:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043108992:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1787826897.441149:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826897.441175:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826897/real 1787826897] req@ffff9181e007ea00 x1874663043108992/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826922 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826897.441186:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ea00 x1874663043108992/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826922 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826897.441192:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826897.441195:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826897.441211:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043108992:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787826897.441218:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043109120:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787826897.441241:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826897.441253:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826897/real 1787826897] req@ffff9181e007d180 x1874663043109120/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826922 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826897.441263:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d180 x1874663043109120/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826922 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826897.441270:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826897.441273:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826897.441276:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043109120:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787826897.441282:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043109248:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787826897.441304:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826897.441315:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826897/real 1787826897] req@ffff9181e007d500 x1874663043109248/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826922 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826897.441324:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d500 x1874663043109248/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826922 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826897.441331:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826897.441334:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826897.441336:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043109248:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787826897.504928:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826898.529012:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826899.552946:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826900.576903:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826901.600981:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787826902.560852:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826902.560861:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787826902.560865:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826902.560874:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8969 00000100:00080000:2.0:1787826902.560884:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826902.560893:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787826902.560937:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ed80 x1874663043109376/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787826902.560968:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826902.560971:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787826902.560973:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826902.560978:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8969 00000100:00080000:2.0:1787826902.560981:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826902.560985:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826902.560994:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826902.561008:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007df80 x1874663043109504/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787826902.561016:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826902.561019:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787826902.561021:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826902.561025:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8969 00000100:00080000:2.0:1787826902.561028:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826902.561033:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826902.561038:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826902.561050:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d500 x1874663043109632/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826902.561095:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043109376:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787826902.561185:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043109504:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787826902.561281:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043109632:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787826902.624926:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826903.648879:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826904.672880:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1787826905.633061:0:2376:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:0.0:1787826905.633104:0:2376:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1787826905.633119:0:2376:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1787826905.633156:0:2376:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1787826905.633164:0:2376:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:3.0:1787826905.633203:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826902/real 1787826905] req@ffff9181e007ed80 x1874663043109376/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826927 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826905.633228:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ed80 x1874663043109376/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826927 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826905.633240:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826905.633244:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826905.633263:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043109376:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787826905.633271:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826902/real 1787826905] req@ffff9181e007df80 x1874663043109504/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826927 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826905.633284:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007df80 x1874663043109504/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826927 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826905.633290:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826905.633294:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826905.633297:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043109504:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787826905.633304:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826902/real 1787826905] req@ffff9181e007d500 x1874663043109632/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826927 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826905.633315:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d500 x1874663043109632/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826927 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826905.633326:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826905.633330:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826905.633334:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043109632:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787826905.696903:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826906.721022:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787826907.680936:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826907.680946:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787826907.680949:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826907.680959:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8974 00000100:00080000:0.0:1787826907.680969:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826907.680976:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787826907.681009:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7800 x1874663043109760/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787826907.681036:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826907.681040:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787826907.681042:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826907.681046:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8974 00000100:00080000:0.0:1787826907.681049:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826907.681053:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787826907.681064:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826907.681077:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4e00 x1874663043109888/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787826907.681087:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826907.681090:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787826907.681094:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787826907.681096:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043109760:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787826907.681101:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8974 00000100:00080000:0.0:1787826907.681107:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826907.681111:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787826907.681116:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826907.681130:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5500 x1874663043110016/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787826907.681181:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826907.681221:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826907/real 1787826907] req@ffff9181c55a7800 x1874663043109760/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826932 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826907.681241:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7800 x1874663043109760/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826932 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826907.681253:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826907.681263:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826907.681286:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043109760:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787826907.681316:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043109888:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787826907.681345:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826907.681358:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826907/real 1787826907] req@ffff9181c55a4e00 x1874663043109888/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826932 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826907.681381:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4e00 x1874663043109888/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826932 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826907.681389:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826907.681393:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826907.681397:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043109888:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787826907.681403:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043110016:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787826907.681423:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826907.681435:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826907/real 1787826907] req@ffff9181c55a5500 x1874663043110016/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826932 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826907.681446:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5500 x1874663043110016/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826932 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826907.681454:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826907.681457:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826907.681461:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043110016:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787826907.744975:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826908.768957:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826909.792918:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826910.816877:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826911.840947:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787826912.800856:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826912.800863:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787826912.800866:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826912.800875:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8979 00000100:00080000:2.0:1787826912.800884:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 10000000:01000000:0.0:1787826912.800891:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'params' failed (-108). 00000100:00080000:2.0:1787826912.800892:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 10000000:00020000:0.0:1787826912.800898:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00100000:2.0:1787826912.800929:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d180 x1874663043110144/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787826912.800952:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826912.800954:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787826912.800955:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826912.800959:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8979 00000100:00080000:2.0:1787826912.800961:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826912.800964:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826912.800972:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826912.800990:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ea00 x1874663043110272/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826912.800994:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043110144:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787826912.801009:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826912.801013:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787826912.801014:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826912.801018:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8979 00000100:00080000:2.0:1787826912.801021:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826912.801025:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826912.801029:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826912.801039:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e300 x1874663043110400/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787826912.801061:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826912.801089:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826912/real 1787826912] req@ffff9181e007d180 x1874663043110144/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826937 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826912.801101:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d180 x1874663043110144/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826937 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826912.801109:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826912.801111:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826912.801114:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043110144:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787826912.801131:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043110272:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787826912.801146:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826912.801153:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826912/real 1787826912] req@ffff9181e007ea00 x1874663043110272/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826937 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826912.801160:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ea00 x1874663043110272/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826937 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826912.801169:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826912.801171:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826912.801173:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043110272:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787826912.801176:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043110400:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787826912.801188:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826912.801196:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826912/real 1787826912] req@ffff9181e007e300 x1874663043110400/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826937 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826912.801215:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e300 x1874663043110400/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826937 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826912.801226:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826912.801229:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826912.801233:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043110400:192.168.203.147@tcp:38:kworker.0 10000000:01000000:0.0:1787826912.805551:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:0.0:1787826912.805558:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff9181c87b19b8 from 1 10000000:01000000:0.0:1787826912.805562:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00010000:00010000:0.0:1787826912.805609:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f56f lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826912.805621:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f56f lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826912.805633:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f56f lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787826912.805653:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d01a5780 going to sleep for 0 seconds 00000100:00100000:0.0:1787826912.805663:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043110528:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:0.0:1787826912.805677:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f56f lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826912.805685:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f56f lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826912.805691:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f56f lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826912.805698:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f56f lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826912.805705:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f56f lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826912.805712:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0513400 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:0.0:1787826912.805716:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f56f lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826912.805729:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f56f lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1787826912.805736:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f56f lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787826912.805761:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:0.0:1787826912.805765:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787826912.805774:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f56f lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787826912.805781:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f56f lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787826912.805807:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:0.0:1787826912.805811:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:3.0:1787826912.805884:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826912.805890:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787826912.805893:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826912.805899:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8984 00000100:00080000:3.0:1787826912.805907:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826912.805912:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787826912.805931:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635f80 x1874663043110656/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826912.805982:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043110656:192.168.203.147@tcp:250:kworker.0 00000800:00000100:0.0:1787826912.806030:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826912.806059:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826912/real 1787826912] req@ffff9181c5635f80 x1874663043110656/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826937 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826912.806074:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635f80 x1874663043110656/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826937 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826912.806083:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826912.806086:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826912.806099:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043110656:192.168.203.147@tcp:250:kworker.0 00010000:00010000:2.0:1787826912.864899:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826913.888911:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826914.912974:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826915.936920:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826916.960954:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787826917.920876:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826917.920888:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787826917.920893:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826917.920906:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8984 00000100:00080000:2.0:1787826917.920919:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826917.920927:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787826917.920975:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c700 x1874663043110784/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787826917.921004:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826917.921007:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787826917.921009:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826917.921012:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8984 00000100:00080000:2.0:1787826917.921016:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826917.921019:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826917.921028:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826917.921039:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f800 x1874663043110912/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787826917.921051:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826917.921054:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:0.0:1787826917.921056:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043110784:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787826917.921056:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826917.921061:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8984 00000100:00080000:2.0:1787826917.921067:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826917.921069:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826917.921072:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826917.921078:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f100 x1874663043111040/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787826917.921135:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826917.921176:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826917/real 1787826917] req@ffff9181e007c700 x1874663043110784/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826942 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826917.921195:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c700 x1874663043110784/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826942 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826917.921205:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826917.921208:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826917.921226:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043110784:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787826917.921250:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043110912:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826917.921270:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826917.921282:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826917/real 1787826917] req@ffff9181e007f800 x1874663043110912/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826942 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826917.921293:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f800 x1874663043110912/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826942 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826917.921300:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826917.921303:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826917.921306:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043110912:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787826917.921311:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043111040:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826917.921326:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826917.921335:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826917/real 1787826917] req@ffff9181e007f100 x1874663043111040/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826942 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826917.921351:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f100 x1874663043111040/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826942 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826917.921357:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826917.921360:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826917.921363:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043111040:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787826917.984980:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826919.009016:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826920.032975:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826921.056932:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826922.081010:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787826923.040878:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826923.040889:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787826923.040893:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826923.040906:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8989 00000100:00080000:3.0:1787826923.040916:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826923.040923:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787826923.040962:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634e00 x1874663043111168/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826923.040987:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826923.040991:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787826923.040993:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826923.040997:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8989 00000100:00080000:3.0:1787826923.041001:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826923.041006:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826923.041017:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826923.041031:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043111168:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787826923.041036:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635f80 x1874663043111296/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787826923.041048:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787826923.041052:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787826923.041054:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787826923.041058:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8989 00000100:00080000:3.0:1787826923.041062:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787826923.041065:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787826923.041069:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826923.041079:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043111424/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787826923.041119:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826923.041158:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826923/real 1787826923] req@ffff9181c5634e00 x1874663043111168/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826948 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826923.041172:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634e00 x1874663043111168/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826948 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826923.041180:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826923.041184:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826923.041196:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043111168:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787826923.041226:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043111296:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826923.041249:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826923.041259:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826923/real 1787826923] req@ffff9181c5635f80 x1874663043111296/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826948 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826923.041272:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635f80 x1874663043111296/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826948 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826923.041277:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826923.041280:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826923.041282:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043111296:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787826923.041285:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043111424:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826923.041297:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826923.041303:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826923/real 1787826923] req@ffff9181c5634380 x1874663043111424/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826948 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826923.041309:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043111424/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826948 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826923.041314:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826923.041316:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826923.041317:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043111424:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787826923.104928:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826924.128875:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826925.152959:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787826926.176837:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826927.200927:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787826928.160878:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826928.160888:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787826928.160891:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826928.160901:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8994 00000100:00080000:2.0:1787826928.160911:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826928.160918:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787826928.160952:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d880 x1874663043111552/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787826928.160978:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826928.160982:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787826928.160984:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826928.160989:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8994 00000100:00080000:2.0:1787826928.160992:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826928.160995:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826928.161004:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826928.161019:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f480 x1874663043111680/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787826928.161029:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1787826928.161031:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043111552:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787826928.161033:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787826928.161036:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826928.161044:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8994 00000100:00080000:2.0:1787826928.161047:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826928.161050:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826928.161055:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826928.161066:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ce00 x1874663043111808/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787826928.161101:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826928.161140:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826928/real 1787826928] req@ffff9181e007d880 x1874663043111552/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826953 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826928.161158:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d880 x1874663043111552/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826953 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826928.161168:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826928.161172:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826928.161190:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043111552:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787826928.161217:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043111680:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826928.161241:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826928.161253:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826928/real 1787826928] req@ffff9181e007f480 x1874663043111680/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826953 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826928.161264:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f480 x1874663043111680/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826953 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826928.161272:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826928.161275:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826928.161278:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043111680:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787826928.161283:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043111808:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826928.161299:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826928.161308:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826928/real 1787826928] req@ffff9181e007ce00 x1874663043111808/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826953 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826928.161318:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ce00 x1874663043111808/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826953 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826928.161325:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826928.161328:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826928.161331:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043111808:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1787826928.224893:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826929.250413:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826930.272938:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826931.296917:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826932.320842:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787826933.280850:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826933.280860:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787826933.280863:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 10000000:01000000:1.0:1787826933.280870:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-sptlrpc' failed (-108). 00000100:00080000:0.0:1787826933.280875:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 8999 10000000:00020000:1.0:1787826933.280877:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00080000:0.0:1787826933.280887:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826933.280895:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787826933.280925:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5180 x1874663043111936/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787826933.280953:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826933.280956:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787826933.280958:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826933.280962:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8999 00000100:00080000:0.0:1787826933.280965:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826933.280970:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787826933.280978:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787826933.280991:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043111936:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787826933.280992:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4380 x1874663043112064/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787826933.281006:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826933.281009:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787826933.281011:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826933.281015:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 8999 00000100:00080000:0.0:1787826933.281018:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826933.281023:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787826933.281028:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826933.281040:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4700 x1874663043112192/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787826933.281050:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826933.281083:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826933/real 1787826933] req@ffff9181c55a5180 x1874663043111936/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826958 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826933.281105:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5180 x1874663043111936/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826958 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826933.281112:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826933.281115:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826933.281118:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043111936:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787826933.281133:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043112064:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787826933.281149:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826933.281157:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826933/real 1787826933] req@ffff9181c55a4380 x1874663043112064/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826958 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826933.281168:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4380 x1874663043112064/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826958 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826933.281175:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826933.281178:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826933.281181:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043112064:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787826933.281192:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043112192:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787826933.281212:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826933.281223:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826933/real 1787826933] req@ffff9181c55a4700 x1874663043112192/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826958 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826933.281233:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4700 x1874663043112192/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826958 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826933.281241:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826933.281244:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826933.281247:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043112192:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787826933.344926:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826934.368949:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787826934.496899:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:1.0:1787826934.496910:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff9181c5620000 from 341 10000000:01000000:1.0:1787826934.496917:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:1.0:1787826934.496981:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f576 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826934.496995:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f576 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826934.497009:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f576 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826934.497048:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215d240 going to sleep for 0 seconds 00000100:00100000:1.0:1787826934.497063:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043112320:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787826934.497097:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f576 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826934.497107:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f576 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826934.497115:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f576 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826934.497124:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f576 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826934.497133:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f576 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826934.497141:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4e00 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787826934.497146:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f576 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826934.497154:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f576 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787826934.497163:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f576 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826934.497170:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:1.0:1787826934.497174:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787826934.497181:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f576 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826934.497189:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f576 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826934.497216:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787826934.497219:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:0.0:1787826934.497302:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826934.497307:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787826934.497311:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826934.497318:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9005 00000100:00080000:0.0:1787826934.497328:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826934.497334:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787826934.497365:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5f80 x1874663043112448/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787826934.497394:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787826934.497398:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787826934.497400:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826934.497410:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9005 00000100:00080000:0.0:1787826934.497413:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826934.497418:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787826934.497428:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826934.497441:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5880 x1874663043112576/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787826934.497450:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787826934.497450:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043112448:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787826934.497454:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787826934.497455:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787826934.497459:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9005 00000100:00080000:0.0:1787826934.497462:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787826934.497466:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787826934.497471:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826934.497479:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6d80 x1874663043112704/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787826934.497528:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826934.497564:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826934/real 1787826934] req@ffff9181c55a5f80 x1874663043112448/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826959 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826934.497582:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5f80 x1874663043112448/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826959 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826934.497595:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826934.497599:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826934.497613:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043112448:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787826934.497634:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043112576:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787826934.497658:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826934.497671:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826934/real 1787826934] req@ffff9181c55a5880 x1874663043112576/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826959 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826934.497703:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5880 x1874663043112576/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826959 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826934.497710:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826934.497712:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826934.497714:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043112576:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787826934.497718:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043112704:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787826934.497734:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826934.497762:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826934/real 1787826934] req@ffff9181c55a6d80 x1874663043112704/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826959 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826934.497774:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6d80 x1874663043112704/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826959 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826934.497782:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826934.497785:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826934.497788:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043112704:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787826935.392942:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826936.416919:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826937.440979:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826938.464949:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826939.489053:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787826939.936957:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826939.936970:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787826939.936974:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826939.936985:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9006 00000100:00080000:1.0:1787826939.936997:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787826939.937005:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787826939.937050:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1180 x1874663043112832/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787826939.937083:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826939.937088:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787826939.937090:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826939.937094:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9006 00000100:00080000:1.0:1787826939.937098:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787826939.937102:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787826939.937113:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787826939.937126:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0000 x1874663043112960/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787826939.937135:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826939.937139:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787826939.937140:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826939.937145:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9006 00000100:00080000:1.0:1787826939.937148:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787826939.937152:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787826939.937157:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787826939.937166:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1c00 x1874663043113088/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826939.937413:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043112832:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1787826939.937487:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826939.937524:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826939/real 1787826939] req@ffff9181c35b1180 x1874663043112832/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826964 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826939.937541:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1180 x1874663043112832/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826964 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826939.937557:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826939.937562:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826939.937582:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043112832:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787826939.937589:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043112960:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787826939.937609:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826939.937621:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826939/real 1787826939] req@ffff9181c35b0000 x1874663043112960/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826964 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826939.937632:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0000 x1874663043112960/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826964 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826939.937639:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826939.937642:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826939.937656:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043112960:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787826939.937662:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043113088:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787826939.937679:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826939.937689:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826939/real 1787826939] req@ffff9181c35b1c00 x1874663043113088/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826964 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826939.937698:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1c00 x1874663043113088/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826964 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826939.937705:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826939.937708:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826939.937711:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043113088:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787826940.512919:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826941.536935:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826942.560922:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826943.588798:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826944.608956:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787826945.056896:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826945.056906:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787826945.056910:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826945.056920:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9011 00000100:00080000:1.0:1787826945.056932:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787826945.056939:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787826945.056975:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0380 x1874663043113216/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787826945.057010:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826945.057014:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787826945.057016:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826945.057020:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9011 00000100:00080000:1.0:1787826945.057023:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787826945.057028:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787826945.057036:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787826945.057050:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1500 x1874663043113344/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787826945.057065:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826945.057068:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787826945.057070:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826945.057075:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9011 00000100:00080000:1.0:1787826945.057078:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787826945.057082:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787826945.057087:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787826945.057097:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3100 x1874663043113472/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826945.057247:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043113216:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1787826945.057311:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826945.057355:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826945/real 1787826945] req@ffff9181c35b0380 x1874663043113216/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826970 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826945.057371:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0380 x1874663043113216/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826970 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826945.057381:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826945.057385:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826945.057400:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043113216:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787826945.057408:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043113344:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787826945.057431:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826945.057443:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826945/real 1787826945] req@ffff9181c35b1500 x1874663043113344/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826970 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826945.057462:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1500 x1874663043113344/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826970 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826945.057470:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826945.057473:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826945.057476:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043113344:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787826945.057482:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043113472:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787826945.057502:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826945.057512:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826945/real 1787826945] req@ffff9181c35b3100 x1874663043113472/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826970 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826945.057523:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3100 x1874663043113472/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826970 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826945.057531:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826945.057534:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826945.057537:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043113472:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787826945.633042:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826946.656896:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826947.680895:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826948.705271:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826949.728938:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787826950.176915:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826950.176928:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787826950.176933:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826950.176946:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9016 00000100:00080000:2.0:1787826950.176959:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826950.176967:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787826950.177014:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007fb80 x1874663043113600/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787826950.177043:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826950.177047:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787826950.177049:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826950.177053:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9016 00000100:00080000:2.0:1787826950.177056:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826950.177060:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826950.177071:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826950.177083:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ca80 x1874663043113728/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787826950.177090:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043113600:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787826950.177097:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826950.177102:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787826950.177103:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826950.177108:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9016 00000100:00080000:2.0:1787826950.177112:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826950.177116:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826950.177121:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826950.177133:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e680 x1874663043113856/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787826950.177163:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826950.177197:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826950/real 1787826950] req@ffff9181e007fb80 x1874663043113600/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826975 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826950.177216:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007fb80 x1874663043113600/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826975 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826950.177228:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826950.177232:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826950.177245:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043113600:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787826950.177280:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043113728:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787826950.177301:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826950.177310:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826950/real 1787826950] req@ffff9181e007ca80 x1874663043113728/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826975 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826950.177318:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ca80 x1874663043113728/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826975 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826950.177332:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826950.177336:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826950.177339:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043113728:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787826950.177345:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043113856:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787826950.177364:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787826950.177382:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826950/real 1787826950] req@ffff9181e007e680 x1874663043113856/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826975 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787826950.177393:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e680 x1874663043113856/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826975 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787826950.177399:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787826950.177401:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787826950.177403:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043113856:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787826950.753889:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826951.777665:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826952.800916:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826953.825050:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787826954.784956:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-cliir' failed (-108). 10000000:00000400:1.0:1787826954.784970:0:159735:0:(mgc_request.c:1900:mgc_process_log()) MGC192.168.203.147@tcp: IR log lustre-cliir failed, not fatal: rc = -108 10000000:01000000:1.0:1787826954.790350:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:1.0:1787826954.790359:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff9181c5620000 from 52 10000000:01000000:1.0:1787826954.790366:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:1.0:1787826954.790437:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f57d lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826954.790450:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f57d lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826954.790465:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f57d lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826954.790493:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215d240 going to sleep for 0 seconds 00000100:00100000:1.0:1787826954.790508:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043113984:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787826954.790528:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f57d lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826954.790536:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f57d lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826954.790556:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f57d lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826954.790565:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f57d lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826954.790579:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f57d lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826954.790587:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b5000 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787826954.790591:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f57d lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826954.790667:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f57d lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787826954.790675:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f57d lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826954.790681:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1787826954.790685:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787826954.790693:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f57d lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826954.790701:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f57d lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826954.790728:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787826954.790732:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:2.0:1787826954.790842:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826954.790850:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787826954.790854:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826954.790863:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9021 00000100:00080000:2.0:1787826954.790876:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826954.790884:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787826954.790911:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007dc00 x1874663043114112/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787826954.790936:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826954.790940:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787826954.790942:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826954.790947:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9021 00000100:00080000:2.0:1787826954.790951:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826954.790955:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826954.790966:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826954.790971:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043114112:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787826954.790981:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c000 x1874663043114240/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787826954.790992:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826954.790996:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787826954.791010:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826954.791014:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9021 00000100:00080000:2.0:1787826954.791019:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826954.791023:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826954.791029:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826954.791040:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c380 x1874663043114368/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787826954.791054:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826954.791092:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826954/real 1787826954] req@ffff9181e007dc00 x1874663043114112/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826979 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826954.791110:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007dc00 x1874663043114112/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826979 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826954.791119:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826954.791123:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826954.791136:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043114112:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787826954.791157:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043114240:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826954.791285:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826954.791307:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826954/real 1787826954] req@ffff9181e007c000 x1874663043114240/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826979 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826954.791321:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c000 x1874663043114240/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826979 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826954.791329:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826954.791333:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826954.791335:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043114240:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787826954.791342:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043114368:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826954.791364:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826954.791380:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826954/real 1787826954] req@ffff9181e007c380 x1874663043114368/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826979 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826954.791392:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c380 x1874663043114368/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826979 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826954.791400:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826954.791404:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826954.791408:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043114368:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787826954.848932:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826955.872913:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826956.896961:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826957.920878:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826958.944948:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787826959.904911:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826959.904921:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787826959.904928:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826959.904937:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9026 00000100:00080000:2.0:1787826959.904947:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826959.904954:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787826959.904988:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c380 x1874663043114496/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787826959.905022:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826959.905026:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787826959.905027:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826959.905032:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9026 00000100:00080000:2.0:1787826959.905035:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826959.905040:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826959.905048:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826959.905062:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c000 x1874663043114624/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787826959.905071:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826959.905073:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787826959.905075:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826959.905079:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9026 00000100:00080000:2.0:1787826959.905082:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826959.905086:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826959.905090:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826959.905100:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007dc00 x1874663043114752/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826959.905402:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043114496:192.168.203.147@tcp:250:kworker.0 00000800:00000100:0.0:1787826959.905459:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826959.905490:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826959/real 1787826959] req@ffff9181e007c380 x1874663043114496/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826984 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826959.905522:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c380 x1874663043114496/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826984 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826959.905531:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826959.905535:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826959.905553:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043114496:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787826959.905560:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043114624:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826959.905582:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826959.905594:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826959/real 1787826959] req@ffff9181e007c000 x1874663043114624/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826984 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826959.905606:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c000 x1874663043114624/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826984 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826959.905613:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826959.905616:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826959.905619:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043114624:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787826959.905625:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043114752:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826959.905643:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826959.905653:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826959/real 1787826959] req@ffff9181e007dc00 x1874663043114752/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826984 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826959.905663:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007dc00 x1874663043114752/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826984 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826959.905670:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826959.905673:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826959.905676:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043114752:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787826959.969048:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826960.993267:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826962.016955:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826963.040928:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826964.064929:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787826965.024848:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826965.024855:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787826965.024858:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826965.024867:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9031 00000100:00080000:2.0:1787826965.024875:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826965.024882:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787826965.024909:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e680 x1874663043114880/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787826965.024930:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826965.024933:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787826965.024934:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826965.024937:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9031 00000100:00080000:2.0:1787826965.024939:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826965.024942:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826965.024948:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826965.024956:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ca80 x1874663043115008/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787826965.024963:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826965.024965:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787826965.024967:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826965.024969:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9031 00000100:00080000:2.0:1787826965.024971:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826965.024973:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826965.024975:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826965.024984:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007fb80 x1874663043115136/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826965.024985:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043114880:192.168.203.147@tcp:250:kworker.0 00000800:00000100:0.0:1787826965.025046:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826965.025081:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826965/real 1787826965] req@ffff9181e007e680 x1874663043114880/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826990 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826965.025098:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e680 x1874663043114880/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826990 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826965.025113:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826965.025116:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826965.025135:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043114880:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787826965.025142:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043115008:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826965.025161:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826965.025172:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826965/real 1787826965] req@ffff9181e007ca80 x1874663043115008/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826990 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826965.025183:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ca80 x1874663043115008/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826990 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826965.025190:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826965.025193:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826965.025196:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043115008:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787826965.025216:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043115136:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826965.025232:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826965.025240:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826965/real 1787826965] req@ffff9181e007fb80 x1874663043115136/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826990 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826965.025251:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007fb80 x1874663043115136/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826990 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826965.025258:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826965.025261:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826965.025264:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043115136:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787826965.088918:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826966.112915:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826967.136924:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826968.161020:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826969.185015:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787826970.144930:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826970.144951:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787826970.144956:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826970.144970:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9036 00000100:00080000:2.0:1787826970.144983:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826970.144992:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787826970.145054:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ce00 x1874663043115264/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787826970.145091:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826970.145095:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787826970.145097:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826970.145102:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9036 00000100:00080000:2.0:1787826970.145105:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826970.145109:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826970.145128:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826970.145144:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f480 x1874663043115392/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787826970.145154:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826970.145158:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787826970.145160:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826970.145164:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9036 00000100:00080000:2.0:1787826970.145168:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826970.145171:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826970.145176:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826970.145202:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d880 x1874663043115520/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826970.145523:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043115264:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787826970.145625:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043115392:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787826970.145655:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043115520:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787826970.208992:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826971.232932:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787826972.256944:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1787826973.217046:0:2377:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:1.0:1787826973.217088:0:2377:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787826973.217105:0:2377:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1787826973.217140:0:2377:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1787826973.217147:0:2377:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:0.0:1787826973.217278:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826970/real 1787826973] req@ffff9181e007ce00 x1874663043115264/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826995 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826973.217315:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ce00 x1874663043115264/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826995 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826973.217326:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826973.217331:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826973.217347:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043115264:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787826973.217354:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826970/real 1787826973] req@ffff9181e007f480 x1874663043115392/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826995 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826973.217366:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f480 x1874663043115392/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826995 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826973.217373:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826973.217377:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826973.217380:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043115392:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787826973.217387:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826970/real 1787826973] req@ffff9181e007d880 x1874663043115520/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826995 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826973.217397:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d880 x1874663043115520/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826995 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826973.217405:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826973.217408:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826973.217411:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043115520:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787826973.280886:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826974.305883:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787826974.816882:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-client' failed (-108). 10000000:00020000:1.0:1787826974.816890:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:1.0:1787826974.816893:0:159735:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:1.0:1787826974.816894:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff9181c5620000 from 7 10000000:01000000:1.0:1787826974.816897:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00010000:00010000:1.0:1787826974.816933:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f584 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826974.816942:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f584 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826974.816950:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f584 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826974.816965:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215d240 going to sleep for 0 seconds 00000100:00100000:1.0:1787826974.816973:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043115648:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787826974.816983:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f584 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826974.816988:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f584 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826974.816995:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f584 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826974.817001:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f584 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826974.817005:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f584 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826974.817009:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4400 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787826974.817011:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f584 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826974.817016:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f584 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787826974.817020:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f584 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826974.817023:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:1.0:1787826974.817026:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787826974.817032:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f584 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826974.817039:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f584 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826974.817063:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787826974.817066:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:2.0:1787826974.817154:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826974.817162:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787826974.817166:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826974.817171:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9041 00000100:00080000:2.0:1787826974.817184:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826974.817190:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787826974.817212:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f100 x1874663043115776/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787826974.817239:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826974.817243:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787826974.817244:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826974.817249:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9041 00000100:00080000:2.0:1787826974.817253:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826974.817258:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826974.817267:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787826974.817270:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043115776:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787826974.817282:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f800 x1874663043115904/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787826974.817293:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826974.817295:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787826974.817297:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826974.817299:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9041 00000100:00080000:2.0:1787826974.817302:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826974.817305:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826974.817308:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826974.817316:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c700 x1874663043116032/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787826974.817339:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826974.817375:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826974/real 1787826974] req@ffff9181e007f100 x1874663043115776/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826999 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826974.817392:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f100 x1874663043115776/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787826999 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826974.817414:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826974.817418:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826974.817431:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043115776:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787826974.817448:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043115904:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826974.817471:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826974.817492:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826974/real 1787826974] req@ffff9181e007f800 x1874663043115904/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826999 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826974.817504:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f800 x1874663043115904/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826999 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826974.817511:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826974.817514:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826974.817517:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043115904:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787826974.817528:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043116032:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826974.817557:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826974.817568:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826974/real 1787826974] req@ffff9181e007c700 x1874663043116032/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826999 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826974.817579:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c700 x1874663043116032/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787826999 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826974.817586:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826974.817588:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826974.817591:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043116032:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787826975.328930:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826976.352918:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826977.376909:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826978.400906:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826979.424902:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787826979.872845:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826979.872856:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787826979.872860:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826979.872869:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9046 00000100:00080000:1.0:1787826979.872879:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787826979.872886:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787826979.872920:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3100 x1874663043116160/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787826979.872950:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826979.872954:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787826979.872956:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826979.872960:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9046 00000100:00080000:1.0:1787826979.872963:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787826979.872968:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787826979.872977:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787826979.872990:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1500 x1874663043116288/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826979.872995:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043116160:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787826979.873009:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826979.873012:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787826979.873014:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826979.873019:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9046 00000100:00080000:1.0:1787826979.873023:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787826979.873027:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787826979.873032:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787826979.873046:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0380 x1874663043116416/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787826979.873074:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826979.873117:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826979/real 1787826979] req@ffff9181c35b3100 x1874663043116160/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827004 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826979.873135:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3100 x1874663043116160/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827004 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826979.873144:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826979.873148:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826979.873160:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043116160:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787826979.873182:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043116288:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826979.873203:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826979.873214:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826979/real 1787826979] req@ffff9181c35b1500 x1874663043116288/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827004 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826979.873228:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1500 x1874663043116288/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827004 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826979.873236:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826979.873239:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826979.873243:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043116288:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787826979.873249:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043116416:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826979.873268:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826979.873279:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826979/real 1787826979] req@ffff9181c35b0380 x1874663043116416/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827004 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826979.873291:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0380 x1874663043116416/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827004 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826979.873299:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826979.873303:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826979.873306:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043116416:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787826980.449031:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787826981.472941:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826982.496933:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826983.521053:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826984.544940:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787826984.993022:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826984.993033:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787826984.993037:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826984.993046:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9051 00000100:00080000:1.0:1787826984.993056:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787826984.993064:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787826984.993097:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1c00 x1874663043116544/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787826984.993122:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826984.993126:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787826984.993128:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826984.993132:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9051 00000100:00080000:1.0:1787826984.993135:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787826984.993138:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787826984.993146:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787826984.993158:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0000 x1874663043116672/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787826984.993168:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787826984.993170:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787826984.993179:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787826984.993182:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9051 00000100:00080000:1.0:1787826984.993184:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787826984.993188:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787826984.993197:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787826984.993207:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1180 x1874663043116800/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826984.993241:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043116544:192.168.203.147@tcp:250:kworker.0 00000800:00000100:0.0:1787826984.993300:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826984.993347:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826984/real 1787826984] req@ffff9181c35b1c00 x1874663043116544/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827009 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826984.993367:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1c00 x1874663043116544/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827009 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826984.993384:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826984.993388:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826984.993401:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043116544:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787826984.993409:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043116672:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826984.993433:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826984.993446:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826984/real 1787826984] req@ffff9181c35b0000 x1874663043116672/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827009 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826984.993458:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0000 x1874663043116672/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827009 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826984.993465:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826984.993468:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826984.993471:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043116672:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787826984.993477:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043116800:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826984.993494:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826984.993504:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826984/real 1787826984] req@ffff9181c35b1180 x1874663043116800/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827009 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826984.993515:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1180 x1874663043116800/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827009 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826984.993523:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826984.993526:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826984.993529:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043116800:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787826985.568952:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826986.592953:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826987.617404:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826988.642374:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826989.664969:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787826990.113142:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826990.113156:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787826990.113161:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826990.113172:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9056 00000100:00080000:2.0:1787826990.113184:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826990.113192:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787826990.113237:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e300 x1874663043116928/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787826990.113268:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826990.113272:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787826990.113273:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826990.113277:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9056 00000100:00080000:2.0:1787826990.113281:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826990.113285:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826990.113307:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826990.113321:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ea00 x1874663043117056/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787826990.113332:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1787826990.113338:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043116928:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787826990.113338:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787826990.113340:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826990.113344:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9056 00000100:00080000:2.0:1787826990.113348:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826990.113354:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826990.113362:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826990.113374:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d180 x1874663043117184/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787826990.113418:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826990.113456:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826990/real 1787826990] req@ffff9181e007e300 x1874663043116928/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827015 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.113471:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e300 x1874663043116928/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827015 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.113478:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826990.113481:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826990.113496:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043116928:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787826990.113516:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043117056:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826990.113532:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826990.113540:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826990/real 1787826990] req@ffff9181e007ea00 x1874663043117056/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827015 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.113548:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ea00 x1874663043117056/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827015 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.113553:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826990.113555:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826990.113557:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043117056:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787826990.113560:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043117184:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826990.113569:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826990.113574:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826990/real 1787826990] req@ffff9181e007d180 x1874663043117184/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827015 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826990.113581:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d180 x1874663043117184/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827015 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826990.113586:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826990.113587:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826990.113589:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043117184:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787826990.689353:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826991.712994:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826992.736941:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826993.761094:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826994.786715:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787826995.233303:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'params' failed (-108). 10000000:00020000:1.0:1787826995.233311:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:1.0:1787826995.233316:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:1.0:1787826995.233318:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff9181c87b19b8 from 1 10000000:01000000:1.0:1787826995.233321:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00010000:00010000:1.0:1787826995.233362:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4000/0xbef42c0221d6f58b lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826995.233374:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b4000/0xbef42c0221d6f58b lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826995.233385:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b4000/0xbef42c0221d6f58b lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787826995.233405:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215d240 going to sleep for 0 seconds 00000100:00100000:1.0:1787826995.233415:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043117312:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787826995.233428:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4000/0xbef42c0221d6f58b lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826995.233436:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b4000/0xbef42c0221d6f58b lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826995.233448:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4000/0xbef42c0221d6f58b lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826995.233457:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b4000/0xbef42c0221d6f58b lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826995.233464:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b4000/0xbef42c0221d6f58b lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826995.233470:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4000 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787826995.233474:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4000/0xbef42c0221d6f58b lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826995.233482:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b4000/0xbef42c0221d6f58b lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787826995.233488:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4000/0xbef42c0221d6f58b lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826995.233494:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1787826995.233497:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787826995.233502:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b4000/0xbef42c0221d6f58b lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787826995.233509:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b4000/0xbef42c0221d6f58b lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787826995.233527:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787826995.233530:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:2.0:1787826995.234508:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826995.234515:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787826995.234518:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826995.234524:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9061 00000100:00080000:2.0:1787826995.234533:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826995.234539:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787826995.234563:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d500 x1874663043117440/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787826995.234591:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826995.234595:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787826995.234597:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826995.234601:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9061 00000100:00080000:2.0:1787826995.234604:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826995.234609:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826995.234617:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826995.234628:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007df80 x1874663043117568/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787826995.234637:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787826995.234640:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787826995.234641:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787826995.234644:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9061 00000100:00080000:2.0:1787826995.234648:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787826995.234651:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787826995.234655:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787826995.234664:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ed80 x1874663043117696/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787826995.234779:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043117440:192.168.203.147@tcp:250:kworker.0 00000800:00000100:0.0:1787826995.234837:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826995.234869:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826995/real 1787826995] req@ffff9181e007d500 x1874663043117440/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827020 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826995.234886:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d500 x1874663043117440/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827020 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826995.234896:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826995.234899:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826995.234912:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043117440:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787826995.234918:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043117568:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826995.234943:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826995.234955:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826995/real 1787826995] req@ffff9181e007df80 x1874663043117568/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827020 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826995.234967:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007df80 x1874663043117568/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827020 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826995.234981:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826995.234993:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826995.234996:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043117568:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787826995.235002:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043117696:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787826995.235021:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787826995.235032:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787826995/real 1787826995] req@ffff9181e007ed80 x1874663043117696/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827020 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787826995.235044:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ed80 x1874663043117696/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827020 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787826995.235052:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787826995.235055:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787826995.235058:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043117696:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787826995.809092:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826996.833977:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826997.856969:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826998.880948:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787826999.905618:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827000.352918:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827000.352932:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827000.352944:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827000.352958:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9066 00000100:00080000:1.0:1787827000.352978:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827000.352987:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787827000.353046:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2680 x1874663043117824/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827000.353332:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827000.353336:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827000.353338:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827000.353344:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9066 00000100:00080000:1.0:1787827000.353349:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827000.353362:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827000.353373:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827000.353388:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0e00 x1874663043117952/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827000.353399:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827000.353404:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827000.353406:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827000.353410:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9066 00000100:00080000:1.0:1787827000.353414:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827000.353419:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827000.353424:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827000.353449:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3480 x1874663043118080/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827000.353463:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043117824:192.168.203.147@tcp:250:kworker.0 00000800:00000100:0.0:1787827000.353538:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827000.353577:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827000/real 1787827000] req@ffff9181c35b2680 x1874663043117824/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827025 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827000.353596:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2680 x1874663043117824/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827025 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827000.353615:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827000.353620:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827000.353649:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043117824:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787827000.353657:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043117952:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827000.353678:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827000.353691:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827000/real 1787827000] req@ffff9181c35b0e00 x1874663043117952/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827025 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827000.353709:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0e00 x1874663043117952/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827025 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827000.353717:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827000.353720:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827000.353723:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043117952:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827000.353776:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043118080:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827000.353798:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827000.353809:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827000/real 1787827000] req@ffff9181c35b3480 x1874663043118080/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827025 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827000.353821:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3480 x1874663043118080/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827025 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827000.353830:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827000.353833:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827000.353836:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043118080:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1787827000.928940:0:164162:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1787827000.928986:0:159733:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1787827000.928998:0:159733:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480000/0xbef42c0221d6f488 lrc: 2/0,0 mode: PR/PR res: [0x200027cd1:0x7:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x878a5eea00ceefcf expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1787827000.929307:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827000.930525:0:159733:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480000/0xbef42c0221d6f488 lrc: 1/0,0 mode: --/PR res: [0x200027cd1:0x7:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ceefcf expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787827000.930542:0:159733:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00080000:2.0:1787827000.930694:0:2392:0:(client.c:1845:ptlrpc_send_new_req()) @@@ req waiting for recovery: (FULL != DISCONN) req@ffff9181c5634700 x1874663043118208/t0(0) o103->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:17/18 lens 328/224 e 0 to 0 dl 0 ref 1 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'ldlm_bl.0' uid:0 gid:0 projid:4294967295 00010000:00010000:3.0:1787827000.930735:0:159733:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9181c5620000 lock: ffff9181c3480000/0xbef42c0221d6f488 lrc: 0/0,0 mode: --/PR res: [0x200027cd1:0x7:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x878a5eea00ceefcf expref: -99 pid: 163667 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1787827000.930798:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827000.930800:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827000.930803:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827001.952956:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827002.976949:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827004.001178:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827005.024940:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827005.472901:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827005.472915:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827005.472920:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827005.472933:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9072 00000100:00080000:1.0:1787827005.472946:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827005.472956:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787827005.473005:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2a00 x1874663043118336/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827005.473040:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827005.473044:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827005.473046:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827005.473050:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9072 00000100:00080000:1.0:1787827005.473054:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827005.473058:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827005.473070:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827005.473085:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0a80 x1874663043118464/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827005.473095:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827005.473099:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827005.473101:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827005.473105:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9072 00000100:00080000:1.0:1787827005.473117:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827005.473124:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787827005.473131:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043118336:192.168.203.147@tcp:250:kworker.0 00000008:00100000:1.0:1787827005.473134:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827005.473158:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2d80 x1874663043118592/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827005.473224:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827005.473571:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827005/real 1787827005] req@ffff9181c35b2a00 x1874663043118336/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827030 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827005.473592:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2a00 x1874663043118336/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827030 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827005.473603:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827005.473607:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827005.473632:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043118336:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787827005.473641:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043118464:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827005.473667:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827005.473681:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827005/real 1787827005] req@ffff9181c35b0a80 x1874663043118464/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827030 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827005.473694:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0a80 x1874663043118464/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827030 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827005.473702:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827005.473706:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827005.473709:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043118464:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827005.473734:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043118592:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827005.473800:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827005.473813:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827005/real 1787827005] req@ffff9181c35b2d80 x1874663043118592/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827030 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827005.473831:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2d80 x1874663043118592/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827030 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827005.473839:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827005.473842:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827005.473845:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043118592:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827006.049010:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827007.072925:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827008.096949:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827009.120957:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827010.144934:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827010.592913:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827010.592927:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827010.592931:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827010.592944:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9077 00000100:00080000:1.0:1787827010.592957:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827010.592966:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787827010.593014:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0700 x1874663043118720/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827010.593046:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827010.593050:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827010.593052:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827010.593057:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9077 00000100:00080000:1.0:1787827010.593061:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827010.593065:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827010.593076:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827010.593093:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1f80 x1874663043118848/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827010.593103:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827010.593106:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827010.593108:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827010.593112:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9077 00000100:00080000:1.0:1787827010.593117:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827010.593121:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827010.593126:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827010.593143:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3800 x1874663043118976/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827010.593144:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043118720:192.168.203.147@tcp:250:kworker.0 00000800:00000100:0.0:1787827010.593243:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827010.593281:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827010/real 1787827010] req@ffff9181c35b0700 x1874663043118720/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827035 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827010.593315:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0700 x1874663043118720/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827035 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827010.593327:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827010.593331:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827010.593350:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043118720:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787827010.593358:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043118848:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827010.593400:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827010.593418:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827010/real 1787827010] req@ffff9181c35b1f80 x1874663043118848/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827035 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827010.593430:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1f80 x1874663043118848/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827035 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827010.593437:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827010.593441:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827010.593444:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043118848:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827010.593468:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043118976:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827010.593489:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827010.593499:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827010/real 1787827010] req@ffff9181c35b3800 x1874663043118976/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827035 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827010.593510:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3800 x1874663043118976/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827035 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827010.593522:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827010.593525:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827010.593529:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043118976:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1787827011.168936:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827012.192980:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827013.216862:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827014.240989:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827015.264935:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827015.712918:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827015.712934:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 10000000:01000000:1.0:1787827015.712938:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-sptlrpc' failed (-108). 00000100:00080000:2.0:1787827015.712939:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 10000000:00020000:1.0:1787827015.712953:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00080000:2.0:1787827015.712959:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9082 00000100:00080000:2.0:1787827015.712975:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827015.712985:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827015.713040:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ed80 x1874663043119104/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827015.713074:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827015.713078:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827015.713080:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827015.713084:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9082 00000100:00080000:2.0:1787827015.713088:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827015.713092:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827015.713103:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827015.713131:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043119104:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787827015.713132:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007df80 x1874663043119232/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827015.713147:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827015.713151:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827015.713153:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827015.713158:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9082 00000100:00080000:2.0:1787827015.713161:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827015.713166:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827015.713171:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827015.713184:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d500 x1874663043119360/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827015.713200:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827015.713235:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827015/real 1787827015] req@ffff9181e007ed80 x1874663043119104/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827040 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827015.713250:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ed80 x1874663043119104/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827040 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827015.713258:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827015.713261:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827015.713265:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043119104:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787827015.713285:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043119232:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827015.713298:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827015.713306:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827015/real 1787827015] req@ffff9181e007df80 x1874663043119232/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827040 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827015.713313:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007df80 x1874663043119232/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827040 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827015.713318:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827015.713320:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827015.713323:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043119232:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827015.713326:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043119360:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827015.713335:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827015.713340:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827015/real 1787827015] req@ffff9181e007d500 x1874663043119360/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827040 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827015.713347:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d500 x1874663043119360/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827040 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827015.713351:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827015.713353:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827015.713355:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043119360:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787827016.289304:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827017.312909:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787827017.376875:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:1.0:1787827017.376883:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff9181c5620000 from 341 10000000:01000000:1.0:1787827017.376888:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:1.0:1787827017.376938:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f592 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827017.376950:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f592 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827017.376962:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f592 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787827017.376981:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215d240 going to sleep for 0 seconds 00000100:00100000:1.0:1787827017.376992:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043119488:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787827017.377006:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f592 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827017.377014:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f592 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827017.377021:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f592 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827017.377029:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f592 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827017.377037:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f592 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827017.377043:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4200 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787827017.377046:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f592 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827017.377054:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f592 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787827017.377061:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f592 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787827017.377067:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:1.0:1787827017.377069:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787827017.377075:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f592 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827017.377086:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f592 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787827017.377117:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787827017.377121:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:3.0:1787827017.377192:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827017.377197:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827017.377200:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827017.377207:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9087 00000100:00080000:3.0:1787827017.377215:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827017.377221:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827017.377240:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043119616/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827017.377268:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827017.377271:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827017.377273:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827017.377277:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9087 00000100:00080000:3.0:1787827017.377280:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827017.377284:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827017.377297:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827017.377308:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043119744/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827017.377318:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827017.377321:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827017.377323:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827017.377326:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9087 00000100:00080000:3.0:1787827017.377329:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827017.377333:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827017.377337:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827017.377347:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043119872/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827017.377496:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043119616:192.168.203.147@tcp:250:kworker.0 00000800:00000100:0.0:1787827017.377564:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827017.377594:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827017/real 1787827017] req@ffff9181c5636a00 x1874663043119616/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827042 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827017.377612:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043119616/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827042 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827017.377622:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827017.377626:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827017.377642:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043119616:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787827017.377650:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043119744:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827017.377671:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827017.377687:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827017/real 1787827017] req@ffff9181c5637480 x1874663043119744/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827042 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827017.377699:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043119744/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827042 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827017.377707:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827017.377711:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827017.377713:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043119744:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827017.377719:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043119872:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827017.377739:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827017.377771:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827017/real 1787827017] req@ffff9181c5635500 x1874663043119872/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827042 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827017.377790:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043119872/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827042 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827017.377799:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827017.377803:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827017.377806:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043119872:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787827018.336942:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827019.360949:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827020.384927:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827021.408954:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827022.433262:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827022.880878:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827022.880888:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827022.880892:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827022.880902:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9089 00000100:00080000:2.0:1787827022.880911:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827022.880918:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827022.880951:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d180 x1874663043120000/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827022.880980:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827022.880984:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827022.880986:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827022.880989:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9089 00000100:00080000:2.0:1787827022.880992:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827022.880996:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827022.881006:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827022.881020:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ea00 x1874663043120128/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827022.881023:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043120000:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787827022.881035:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827022.881039:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827022.881040:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827022.881044:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9089 00000100:00080000:2.0:1787827022.881048:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827022.881051:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827022.881056:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827022.881081:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e300 x1874663043120256/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827022.881103:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827022.881139:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827022/real 1787827022] req@ffff9181e007d180 x1874663043120000/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827047 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827022.881156:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d180 x1874663043120000/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827047 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827022.881162:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827022.881165:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827022.881176:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043120000:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787827022.881196:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043120128:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827022.881220:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827022.881237:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827022/real 1787827022] req@ffff9181e007ea00 x1874663043120128/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827047 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827022.881249:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ea00 x1874663043120128/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827047 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827022.881262:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827022.881264:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827022.881266:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043120128:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827022.881270:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043120256:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827022.881284:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827022.881293:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827022/real 1787827022] req@ffff9181e007e300 x1874663043120256/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827047 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827022.881304:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e300 x1874663043120256/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827047 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827022.881310:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827022.881313:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827022.881316:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043120256:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827023.456980:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827024.481277:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827025.504925:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827026.528919:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827027.552970:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827028.000876:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827028.000885:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827028.000889:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827028.000900:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9094 00000100:00080000:1.0:1787827028.000912:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827028.000920:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787827028.000959:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3b80 x1874663043120384/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827028.000984:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827028.000987:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827028.000988:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827028.000991:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9094 00000100:00080000:1.0:1787827028.000993:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827028.000996:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827028.001004:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827028.001017:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3800 x1874663043120512/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827028.001024:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827028.001026:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827028.001027:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827028.001030:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9094 00000100:00080000:1.0:1787827028.001043:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827028.001048:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827028.001051:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827028.001061:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1f80 x1874663043120640/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827028.001064:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043120384:192.168.203.147@tcp:250:kworker.0 00000800:00000100:0.0:1787827028.001142:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827028.001179:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827028/real 1787827028] req@ffff9181c35b3b80 x1874663043120384/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827053 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827028.001207:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3b80 x1874663043120384/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827053 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827028.001217:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827028.001222:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827028.001237:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043120384:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787827028.001247:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043120512:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827028.001276:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827028.001290:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827028/real 1787827028] req@ffff9181c35b3800 x1874663043120512/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827053 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827028.001302:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3800 x1874663043120512/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827053 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827028.001310:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827028.001313:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827028.001317:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043120512:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827028.001340:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043120640:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827028.001360:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827028.001371:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827028/real 1787827028] req@ffff9181c35b1f80 x1874663043120640/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827053 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827028.001382:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1f80 x1874663043120640/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827053 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827028.001390:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827028.001393:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827028.001396:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043120640:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787827028.576958:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827029.601202:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827030.624930:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827031.648918:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827032.672933:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827033.120958:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827033.120968:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827033.120972:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827033.120981:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9099 00000100:00080000:3.0:1787827033.120992:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827033.121013:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827033.121047:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043120768/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827033.121075:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827033.121079:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827033.121081:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827033.121086:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9099 00000100:00080000:3.0:1787827033.121089:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827033.121094:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827033.121103:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827033.121115:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043120768:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827033.121117:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043120896/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827033.121136:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827033.121139:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827033.121141:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827033.121146:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9099 00000100:00080000:3.0:1787827033.121149:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827033.121153:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827033.121158:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827033.121168:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043121024/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827033.121226:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043120896:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827033.121259:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043121024:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827033.697157:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827034.720929:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827035.746719:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1787827036.193093:0:2378:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:0.0:1787827036.193135:0:2378:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1787827036.193152:0:2378:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1787827036.193191:0:2378:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1787827036.193200:0:2378:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:3.0:1787827036.193296:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827033/real 1787827036] req@ffff9181c5636d80 x1874663043120896/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827058 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827036.193320:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043120896/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827058 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827036.193331:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827036.193336:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827036.193339:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043120896:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827036.193345:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827033/real 1787827036] req@ffff9181c5637800 x1874663043121024/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827058 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827036.193355:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043121024/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827058 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827036.193368:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827036.193382:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827036.193386:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043121024:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827036.193392:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827033/real 1787827036] req@ffff9181c5637100 x1874663043120768/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827058 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827036.193403:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043120768/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827058 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827036.193410:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827036.193412:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827036.193427:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043120768:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787827036.768948:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787827037.408906:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-cliir' failed (-108). 10000000:00000400:1.0:1787827037.408914:0:159735:0:(mgc_request.c:1900:mgc_process_log()) MGC192.168.203.147@tcp: IR log lustre-cliir failed, not fatal: rc = -108 10000000:01000000:1.0:1787827037.412550:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:1.0:1787827037.412554:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff9181c5620000 from 52 10000000:01000000:1.0:1787827037.412558:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:1.0:1787827037.412598:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f599 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827037.412606:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f599 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827037.412616:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f599 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787827037.412633:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215d240 going to sleep for 0 seconds 00000100:00100000:1.0:1787827037.412643:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043121152:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787827037.412656:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f599 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827037.412663:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f599 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827037.412670:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f599 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827037.412677:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f599 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827037.412685:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f599 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827037.412691:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b5000 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787827037.412694:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f599 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827037.412702:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f599 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787827037.412710:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f599 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787827037.412716:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1787827037.412720:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787827037.412726:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f599 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827037.412731:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f599 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787827037.412786:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787827037.412789:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:2.0:1787827037.412875:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827037.412883:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827037.412886:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827037.412894:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9104 00000100:00080000:2.0:1787827037.412904:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827037.412911:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827037.412944:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c700 x1874663043121280/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827037.412985:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827037.412989:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827037.412992:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827037.413001:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9104 00000100:00080000:2.0:1787827037.413007:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827037.413012:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787827037.413020:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043121280:192.168.203.147@tcp:250:kworker.0 00000008:00100000:2.0:1787827037.413022:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827037.413041:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f800 x1874663043121408/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827037.413052:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827037.413055:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827037.413057:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827037.413062:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9104 00000100:00080000:2.0:1787827037.413065:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827037.413068:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827037.413073:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000800:00000100:0.0:1787827037.413086:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787827037.413086:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f100 x1874663043121536/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827037.413129:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827037/real 1787827037] req@ffff9181e007c700 x1874663043121280/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827062 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827037.413144:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c700 x1874663043121280/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827062 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827037.413152:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827037.413155:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827037.413168:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043121280:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787827037.413200:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043121408:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827037.413217:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827037.413229:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827037/real 1787827037] req@ffff9181e007f800 x1874663043121408/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827062 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827037.413237:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f800 x1874663043121408/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827062 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827037.413243:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827037.413246:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827037.413248:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043121408:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827037.413252:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043121536:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827037.413262:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827037.413271:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827037/real 1787827037] req@ffff9181e007f100 x1874663043121536/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827062 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827037.413279:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f100 x1874663043121536/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827062 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827037.413283:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827037.413285:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827037.413287:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043121536:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827037.792937:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827038.816948:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827039.840979:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827040.864884:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827041.888965:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827042.848886:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827042.848896:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827042.848900:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827042.848910:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9109 00000100:00080000:3.0:1787827042.848922:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827042.848940:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827042.848981:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043121664/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827042.849016:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827042.849021:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827042.849023:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827042.849027:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9109 00000100:00080000:3.0:1787827042.849030:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827042.849034:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827042.849043:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827042.849060:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043121792/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827042.849069:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1787827042.849074:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043121664:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787827042.849075:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827042.849078:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827042.849089:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9109 00000100:00080000:3.0:1787827042.849093:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827042.849098:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827042.849104:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827042.849122:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043121920/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827042.849151:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827042.849183:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827042/real 1787827042] req@ffff9181c5637100 x1874663043121664/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827067 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827042.849196:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043121664/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827067 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827042.849204:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827042.849206:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827042.849215:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043121664:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787827042.849234:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043121792:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827042.849251:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827042.849260:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827042/real 1787827042] req@ffff9181c5637800 x1874663043121792/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827067 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827042.849268:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043121792/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827067 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827042.849274:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827042.849276:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827042.849278:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043121792:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827042.849281:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043121920:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827042.849293:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827042.849302:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827042/real 1787827042] req@ffff9181c5636d80 x1874663043121920/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827067 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827042.849313:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043121920/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827067 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827042.849320:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827042.849323:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827042.849326:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043121920:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827042.912990:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827043.936891:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827044.960908:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827045.984918:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827047.009137:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827047.968851:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827047.968860:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827047.968864:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827047.968873:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9114 00000100:00080000:3.0:1787827047.968883:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827047.968889:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827047.968940:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043122048/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827047.968966:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827047.968971:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827047.968973:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827047.968977:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9114 00000100:00080000:3.0:1787827047.968981:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827047.968994:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827047.969006:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827047.969019:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043122176/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827047.969023:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043122048:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787827047.969046:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827047.969049:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827047.969051:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827047.969056:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9114 00000100:00080000:3.0:1787827047.969060:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827047.969064:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827047.969069:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827047.969081:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634a80 x1874663043122304/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827047.969088:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827047.969128:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827047/real 1787827047] req@ffff9181c5635c00 x1874663043122048/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827072 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827047.969147:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043122048/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827072 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827047.969154:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827047.969156:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827047.969168:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043122048:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787827047.969189:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043122176:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827047.969215:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827047.969231:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827047/real 1787827047] req@ffff9181c5634000 x1874663043122176/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827072 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827047.969243:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043122176/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827072 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827047.969251:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827047.969254:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827047.969257:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043122176:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827047.969262:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043122304:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827047.969282:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827047.969293:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827047/real 1787827047] req@ffff9181c5634a80 x1874663043122304/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827072 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827047.969306:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634a80 x1874663043122304/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827072 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827047.969318:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827047.969321:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827047.969324:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043122304:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1787827048.032919:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827049.057000:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827050.080896:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827051.104896:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827052.128855:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827053.088901:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827053.088916:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827053.088922:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827053.088936:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9119 00000100:00080000:3.0:1787827053.088949:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827053.088962:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827053.089010:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637b80 x1874663043122432/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827053.089055:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827053.089060:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827053.089062:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827053.089067:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9119 00000100:00080000:3.0:1787827053.089071:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827053.089078:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827053.089091:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827053.089110:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634a80 x1874663043122560/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827053.089118:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043122432:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787827053.089125:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827053.089129:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827053.089131:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827053.089137:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9119 00000100:00080000:3.0:1787827053.089141:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827053.089148:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827053.089154:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827053.089169:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043122688/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827053.089209:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827053.089258:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827053/real 1787827053] req@ffff9181c5637b80 x1874663043122432/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827078 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827053.089281:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637b80 x1874663043122432/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827078 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827053.089300:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827053.089316:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827053.089336:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043122432:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827053.089368:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043122560:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827053.089402:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827053.089412:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827053/real 1787827053] req@ffff9181c5634a80 x1874663043122560/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827078 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827053.089421:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634a80 x1874663043122560/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827078 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827053.089427:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827053.089429:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827053.089431:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043122560:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827053.089435:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043122688:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827053.089448:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827053.089459:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827053/real 1787827053] req@ffff9181c5634000 x1874663043122688/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827078 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827053.089466:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043122688/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827078 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827053.089476:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827053.089479:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827053.089484:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043122688:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827053.152948:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827054.176869:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827055.200940:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827056.224922:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827057.248965:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:3.0:1787827057.696998:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-client' failed (-108). 10000000:00020000:3.0:1787827057.697014:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:3.0:1787827057.697020:0:159735:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:3.0:1787827057.697023:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff9181c5620000 from 7 10000000:01000000:3.0:1787827057.697027:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00010000:00010000:3.0:1787827057.697086:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5a0 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827057.697099:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5a0 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827057.697114:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5a0 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787827057.697138:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d225ff00 going to sleep for 0 seconds 00000100:00100000:3.0:1787827057.697153:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043122816:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:3.0:1787827057.697170:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5a0 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827057.697179:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5a0 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827057.697186:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5a0 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827057.697194:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5a0 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827057.697202:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5a0 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827057.697209:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3480000 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:3.0:1787827057.697214:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5a0 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827057.697222:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5a0 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1787827057.697230:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5a0 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787827057.697236:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:3.0:1787827057.697240:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787827057.697247:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5a0 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827057.697255:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5a0 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787827057.697281:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:3.0:1787827057.697284:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:2.0:1787827057.697327:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827057.697337:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827057.697339:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827057.697349:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9124 00000100:00080000:2.0:1787827057.697359:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827057.697366:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827057.697406:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d880 x1874663043122944/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827057.697435:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827057.697439:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827057.697441:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827057.697445:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9124 00000100:00080000:2.0:1787827057.697449:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827057.697453:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827057.697462:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827057.697477:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f480 x1874663043123072/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827057.697483:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043122944:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787827057.697493:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827057.697497:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827057.697498:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827057.697503:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9124 00000100:00080000:2.0:1787827057.697507:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827057.697512:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827057.697517:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827057.697529:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ce00 x1874663043123200/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827057.697544:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827057.697578:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827057/real 1787827057] req@ffff9181e007d880 x1874663043122944/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827082 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827057.697593:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d880 x1874663043122944/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827082 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827057.697603:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827057.697606:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827057.697626:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043122944:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827057.697647:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043123072:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827057.697666:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827057.697676:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827057/real 1787827057] req@ffff9181e007f480 x1874663043123072/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827082 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827057.697687:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f480 x1874663043123072/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827082 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827057.697694:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827057.697697:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827057.697701:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043123072:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827057.697706:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043123200:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827057.697720:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827057.697729:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827057/real 1787827057] req@ffff9181e007ce00 x1874663043123200/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827082 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827057.697773:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ce00 x1874663043123200/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827082 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827057.697783:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827057.697787:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827057.697790:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043123200:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787827058.273586:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827059.298345:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827060.320965:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827061.345150:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827062.368996:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827062.816901:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827062.816913:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827062.816918:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827062.816929:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9129 00000100:00080000:2.0:1787827062.816943:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827062.816950:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827062.816999:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007fb80 x1874663043123328/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827062.817033:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827062.817037:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827062.817039:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827062.817050:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9129 00000100:00080000:2.0:1787827062.817054:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827062.817059:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827062.817070:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827062.817085:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ca80 x1874663043123456/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827062.817095:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827062.817098:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827062.817100:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827062.817108:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9129 00000100:00100000:1.0:1787827062.817110:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043123328:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787827062.817112:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827062.817121:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827062.817127:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827062.817140:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e680 x1874663043123584/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827062.817195:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827062.817237:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827062/real 1787827062] req@ffff9181e007fb80 x1874663043123328/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827087 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827062.817255:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007fb80 x1874663043123328/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827087 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827062.817266:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827062.817270:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827062.817288:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043123328:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827062.817310:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043123456:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827062.817337:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827062.817362:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827062/real 1787827062] req@ffff9181e007ca80 x1874663043123456/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827087 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827062.817370:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ca80 x1874663043123456/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827087 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827062.817376:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827062.817378:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827062.817380:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043123456:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827062.817384:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043123584:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827062.817397:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827062.817403:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827062/real 1787827062] req@ffff9181e007e680 x1874663043123584/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827087 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827062.817410:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e680 x1874663043123584/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827087 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827062.817414:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827062.817416:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827062.817418:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043123584:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827063.393024:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827064.416953:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827065.441828:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827066.464900:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827067.488975:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827067.936867:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827067.936878:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827067.936881:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827067.936898:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9134 00000100:00080000:1.0:1787827067.936909:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827067.936917:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787827067.936952:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0700 x1874663043123712/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827067.936974:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827067.936977:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827067.936978:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827067.936981:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9134 00000100:00080000:1.0:1787827067.936984:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827067.936987:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827067.936996:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827067.937009:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2d80 x1874663043123840/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827067.937020:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1787827067.937022:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043123712:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787827067.937028:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827067.937032:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827067.937037:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9134 00000100:00080000:1.0:1787827067.937041:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827067.937046:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827067.937052:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827067.937066:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0a80 x1874663043123968/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827067.937077:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827067.937109:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827067/real 1787827067] req@ffff9181c35b0700 x1874663043123712/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827092 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827067.937125:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0700 x1874663043123712/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827092 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827067.937135:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827067.937139:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827067.937158:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043123712:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787827067.937182:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043123840:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827067.937207:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827067.937219:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827067/real 1787827067] req@ffff9181c35b2d80 x1874663043123840/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827092 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827067.937228:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2d80 x1874663043123840/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827092 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827067.937233:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827067.937235:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827067.937238:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043123840:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827067.937242:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043123968:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827067.937254:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827067.937260:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827067/real 1787827067] req@ffff9181c35b0a80 x1874663043123968/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827092 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827067.937271:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0a80 x1874663043123968/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827092 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827067.937278:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827067.937281:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827067.937284:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043123968:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787827068.513017:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827069.537650:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827070.560947:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827071.584938:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827072.608871:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827073.056924:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827073.056934:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827073.056938:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827073.056948:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9139 00000100:00080000:0.0:1787827073.056959:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827073.056966:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787827073.057002:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5c00 x1874663043124096/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827073.057035:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827073.057039:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827073.057041:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827073.057046:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9139 00000100:00080000:0.0:1787827073.057049:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827073.057054:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787827073.057063:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827073.057076:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7480 x1874663043124224/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827073.057088:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827073.057092:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827073.057094:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1787827073.057104:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043124096:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787827073.057105:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9139 00000100:00080000:0.0:1787827073.057111:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827073.057119:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787827073.057135:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827073.057151:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7b80 x1874663043124352/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827073.057224:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827073.057284:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827073/real 1787827073] req@ffff9181c55a5c00 x1874663043124096/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827098 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827073.057303:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5c00 x1874663043124096/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827098 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827073.057314:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827073.057319:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827073.057337:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043124096:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827073.057373:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043124224:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827073.057405:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827073.057419:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827073/real 1787827073] req@ffff9181c55a7480 x1874663043124224/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827098 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827073.057430:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7480 x1874663043124224/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827098 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827073.057438:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827073.057442:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827073.057446:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043124224:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827073.057452:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043124352:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827073.057477:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827073.057487:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827073/real 1787827073] req@ffff9181c55a7b80 x1874663043124352/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827098 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827073.057504:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7b80 x1874663043124352/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827098 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827073.057509:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827073.057511:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827073.057513:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043124352:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827073.632905:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827074.656928:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827075.680885:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827076.704871:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827077.728885:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:3.0:1787827078.176908:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'params' failed (-108). 10000000:00020000:3.0:1787827078.176917:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:3.0:1787827078.176923:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:3.0:1787827078.176926:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff9181c87b19b8 from 1 10000000:01000000:3.0:1787827078.176929:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00000100:00080000:0.0:1787827078.176935:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827078.176941:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827078.176945:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827078.176955:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9144 00000100:00080000:0.0:1787827078.176964:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827078.176976:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:3.0:1787827078.177002:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5a7 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827078.177011:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7100 x1874663043124608/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:3.0:1787827078.177017:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5a7 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827078.177034:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5a7 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787827078.177037:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827078.177043:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827078.177046:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827078.177052:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9144 00000100:00080000:0.0:1787827078.177055:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827078.177059:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827078.177064:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d225ff00 going to sleep for 0 seconds 00000008:00100000:0.0:1787827078.177067:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827078.177074:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043124480:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00100000:0.0:1787827078.177086:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6680 x1874663043124736/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827078.177093:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043124608:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1787827078.177095:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5a7 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787827078.177099:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827078.177108:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827078.177112:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00010000:00010000:3.0:1787827078.177114:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5a7 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787827078.177119:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9144 00000100:00080000:0.0:1787827078.177126:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00010000:00010000:3.0:1787827078.177127:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5a7 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827078.177137:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5a7 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787827078.177145:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:3.0:1787827078.177147:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5a7 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000008:00100000:0.0:1787827078.177152:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:3.0:1787827078.177161:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3480000 already unused, calling callback (ffffffffc1273c20) 00000800:00000100:1.0:1787827078.177163:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 10000000:00010000:3.0:1787827078.177165:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5a7 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827078.177171:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7b80 x1874663043124864/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:3.0:1787827078.177176:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5a7 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1787827078.177185:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5a7 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787827078.177198:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827078/real 1787827078] req@ffff9181c55a7100 x1874663043124608/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827103 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 10000000:01000000:3.0:1787827078.177202:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:3.0:1787827078.177210:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787827078.177218:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5a7 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787827078.177223:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7100 x1874663043124608/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827103 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:3.0:1787827078.177230:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5a7 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787827078.177237:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827078.177244:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827078.177249:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043124608:192.168.203.147@tcp:250:kworker.0 10000000:01000000:3.0:1787827078.177255:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:3.0:1787827078.177259:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00100000:1.0:1787827078.177274:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043124736:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827078.177309:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827078.177320:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827078/real 1787827078] req@ffff9181c55a6680 x1874663043124736/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827103 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827078.177336:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6680 x1874663043124736/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827103 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827078.177343:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827078.177346:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827078.177349:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043124736:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827078.177355:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043124864:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827078.177381:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827078.177393:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827078/real 1787827078] req@ffff9181c55a7b80 x1874663043124864/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827103 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827078.177403:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7b80 x1874663043124864/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827103 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827078.177410:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827078.177413:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827078.177417:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043124864:192.168.203.147@tcp:38:kworker.0 00000100:00080000:0.0:1787827078.177447:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827078.177452:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827078.177454:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827078.177459:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9149 00000100:00080000:0.0:1787827078.177463:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827078.177467:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787827078.177480:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7480 x1874663043124992/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827078.177696:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043124992:192.168.203.147@tcp:250:kworker.0 00000800:00000100:1.0:1787827078.177718:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827078.177728:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827078/real 1787827078] req@ffff9181c55a7480 x1874663043124992/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827103 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827078.177739:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7480 x1874663043124992/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827103 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827078.177772:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827078.177775:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827078.177792:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043124992:192.168.203.147@tcp:250:kworker.0 00010000:00010000:2.0:1787827078.752895:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827079.776907:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827080.800899:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827081.824955:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827082.848909:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827083.296862:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827083.296870:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827083.296873:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827083.296882:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9149 00000100:00080000:3.0:1787827083.296891:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827083.296897:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827083.296929:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043125120/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827083.296955:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827083.296957:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827083.296959:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827083.296962:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9149 00000100:00080000:3.0:1787827083.296964:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827083.296967:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827083.296974:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827083.296995:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043125248/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827083.297015:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043125120:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787827083.297016:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827083.297024:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827083.297027:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827083.297031:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9149 00000100:00080000:3.0:1787827083.297033:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827083.297036:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827083.297040:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827083.297050:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043125376/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827083.297081:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827083.297115:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827083/real 1787827083] req@ffff9181c5635c00 x1874663043125120/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827108 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827083.297131:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043125120/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827108 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827083.297141:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827083.297145:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827083.297165:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043125120:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827083.297191:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043125248:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827083.297213:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827083.297225:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827083/real 1787827083] req@ffff9181c5636d80 x1874663043125248/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827108 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827083.297237:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043125248/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827108 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827083.297245:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827083.297248:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827083.297251:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043125248:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827083.297266:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043125376:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827083.297286:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827083.297296:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827083/real 1787827083] req@ffff9181c5637800 x1874663043125376/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827108 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827083.297307:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043125376/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827108 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827083.297315:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827083.297318:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827083.297321:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043125376:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827083.872893:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827084.896908:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827085.920895:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827086.945904:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827087.968909:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827088.416856:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827088.416865:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827088.416869:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827088.416879:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9155 00000100:00080000:3.0:1787827088.416888:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827088.416895:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827088.416928:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043125504/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827088.416955:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827088.416958:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827088.416960:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827088.416964:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9155 00000100:00080000:3.0:1787827088.416968:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827088.416972:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827088.416981:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827088.416995:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043125632/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827088.417005:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827088.417010:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:1.0:1787827088.417011:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043125504:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787827088.417012:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827088.417022:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9155 00000100:00080000:3.0:1787827088.417025:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827088.417030:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827088.417035:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827088.417047:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043125760/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827088.417066:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827088.417093:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827088/real 1787827088] req@ffff9181c5637100 x1874663043125504/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827113 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827088.417104:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043125504/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827113 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827088.417110:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827088.417112:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827088.417123:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043125504:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827088.417150:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043125632:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827088.417172:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827088.417183:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827088/real 1787827088] req@ffff9181c5635500 x1874663043125632/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827113 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827088.417194:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043125632/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827113 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827088.417201:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827088.417204:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827088.417208:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043125632:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827088.417214:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043125760:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827088.417240:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827088.417252:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827088/real 1787827088] req@ffff9181c5637480 x1874663043125760/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827113 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827088.417259:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043125760/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827113 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827088.417265:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827088.417267:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827088.417270:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043125760:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787827088.992891:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827090.016900:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827091.040887:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827092.064871:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827093.088927:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827093.536820:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827093.536827:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827093.536830:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827093.536838:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9160 00000100:00080000:3.0:1787827093.536847:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827093.536852:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827093.536883:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043125888/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827093.536905:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827093.536909:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827093.536911:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827093.536915:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9160 00000100:00080000:3.0:1787827093.536918:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827093.536923:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827093.536932:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827093.536945:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043126016/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827093.536956:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827093.536959:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:1.0:1787827093.536960:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043125888:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787827093.536961:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827093.536967:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9160 00000100:00080000:3.0:1787827093.536971:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827093.536974:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827093.536977:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827093.536987:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635f80 x1874663043126144/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827093.537025:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827093.537057:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827093/real 1787827093] req@ffff9181c5636a00 x1874663043125888/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827118 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827093.537074:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043125888/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827118 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827093.537085:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827093.537088:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827093.537104:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043125888:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827093.537128:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043126016:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827093.537155:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827093.537167:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827093/real 1787827093] req@ffff9181c5634380 x1874663043126016/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827118 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827093.537179:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043126016/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827118 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827093.537186:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827093.537190:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827093.537193:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043126016:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827093.537199:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043126144:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827093.537216:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827093.537226:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827093/real 1787827093] req@ffff9181c5635f80 x1874663043126144/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827118 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827093.537242:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635f80 x1874663043126144/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827118 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827093.537249:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827093.537253:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827093.537255:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043126144:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827094.112949:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827095.137297:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827096.162151:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827097.184930:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827098.208890:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827098.656877:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827098.656886:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827098.656888:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827098.656896:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9165 00000100:00080000:0.0:1787827098.656906:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 10000000:01000000:3.0:1787827098.656906:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-sptlrpc' failed (-108). 10000000:00020000:3.0:1787827098.656913:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00080000:0.0:1787827098.656916:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787827098.656945:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5c00 x1874663043126272/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827098.656970:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827098.656974:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827098.656975:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827098.656979:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9165 00000100:00080000:0.0:1787827098.656983:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827098.656987:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787827098.656997:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827098.657010:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6d80 x1874663043126400/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827098.657018:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043126272:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787827098.657027:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827098.657031:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827098.657033:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827098.657037:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9165 00000100:00080000:0.0:1787827098.657039:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827098.657042:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787827098.657047:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827098.657057:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5880 x1874663043126528/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827098.657097:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043126400:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827098.657115:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043126528:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787827099.232905:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:3.0:1787827100.064840:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:3.0:1787827100.064848:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff9181c5620000 from 341 10000000:01000000:3.0:1787827100.064853:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:3.0:1787827100.064895:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5ae lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827100.064903:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5ae lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827100.064912:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5ae lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787827100.064929:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d225ff00 going to sleep for 0 seconds 00000100:00100000:3.0:1787827100.064939:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043126656:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:3.0:1787827100.064953:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5ae lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827100.064961:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5ae lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827100.064967:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5ae lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827100.064972:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5ae lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827100.064976:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5ae lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827100.064980:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3480000 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:3.0:1787827100.064982:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5ae lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827100.064987:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5ae lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1787827100.064991:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5ae lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787827100.064997:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:3.0:1787827100.065000:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787827100.065006:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5ae lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827100.065012:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5ae lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787827100.065032:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:3.0:1787827100.065035:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:CONNECTING 00010000:00010000:1.0:1787827100.256887:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827101.280989:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1787827101.729042:0:2379:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:0.0:1787827101.729086:0:2379:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1787827101.729102:0:2379:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1787827101.729138:0:2379:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1787827101.729148:0:2379:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:1.0:1787827101.729189:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827098/real 1787827101] req@ffff9181c55a6d80 x1874663043126400/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827123 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827101.729212:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6d80 x1874663043126400/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827123 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827101.729224:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827101.729229:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827101.729233:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043126400:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827101.729240:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827098/real 1787827101] req@ffff9181c55a5880 x1874663043126528/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827123 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827101.729258:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5880 x1874663043126528/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827123 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827101.729265:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827101.729268:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827101.729271:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043126528:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827101.729279:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827098/real 1787827101] req@ffff9181c55a5c00 x1874663043126272/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827123 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827101.729290:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5c00 x1874663043126272/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827123 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827101.729297:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827101.729321:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827101.729337:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043126272:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787827101.729479:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827101.729485:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827101.729488:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827101.729493:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9170 00000100:00080000:2.0:1787827101.729500:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827101.729507:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827101.729551:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007dc00 x1874663043126784/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827101.729575:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827101.729579:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827101.729581:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827101.729584:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9170 00000100:00080000:2.0:1787827101.729588:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827101.729593:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827101.729601:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827101.729613:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c000 x1874663043126912/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827101.729621:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827101.729624:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827101.729626:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827101.729629:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9170 00000100:00080000:2.0:1787827101.729632:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827101.729635:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827101.729639:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827101.729652:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c380 x1874663043127040/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827101.730222:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043126784:192.168.203.147@tcp:250:kworker.0 00000800:00000100:1.0:1787827101.730294:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827101.730335:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827101/real 1787827101] req@ffff9181e007dc00 x1874663043126784/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827126 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827101.730366:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007dc00 x1874663043126784/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827126 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827101.730375:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827101.730378:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827101.730394:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043126784:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827101.730402:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043126912:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827101.730426:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827101.730438:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827101/real 1787827101] req@ffff9181e007c000 x1874663043126912/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827126 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827101.730450:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c000 x1874663043126912/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827126 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827101.730457:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827101.730461:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827101.730464:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043126912:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827101.730469:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043127040:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827101.730488:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827101.730498:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827101/real 1787827101] req@ffff9181e007c380 x1874663043127040/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827126 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827101.730507:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c380 x1874663043127040/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827126 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827101.730514:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827101.730517:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827101.730530:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043127040:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827102.305621:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827103.328935:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827104.352886:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827105.376928:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827106.400948:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827106.848878:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827106.848889:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827106.848893:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827106.848903:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9173 00000100:00080000:3.0:1787827106.848912:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827106.848919:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827106.848952:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634e00 x1874663043127168/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827106.848975:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827106.848978:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827106.848980:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827106.848984:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9173 00000100:00080000:3.0:1787827106.848988:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827106.848992:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827106.849001:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827106.849020:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043127168:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827106.849025:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635f80 x1874663043127296/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827106.849037:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827106.849040:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827106.849042:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827106.849046:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9173 00000100:00080000:3.0:1787827106.849049:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827106.849053:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827106.849059:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827106.849068:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043127424/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827106.849082:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827106.849126:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827106/real 1787827106] req@ffff9181c5634e00 x1874663043127168/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827131 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827106.849147:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634e00 x1874663043127168/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827131 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827106.849157:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827106.849162:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827106.849172:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043127168:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827106.849197:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043127296:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827106.849221:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827106.849233:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827106/real 1787827106] req@ffff9181c5635f80 x1874663043127296/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827131 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827106.849244:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635f80 x1874663043127296/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827131 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827106.849252:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827106.849255:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827106.849258:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043127296:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827106.849263:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043127424:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827106.849280:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827106.849286:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827106/real 1787827106] req@ffff9181c5634380 x1874663043127424/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827131 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827106.849297:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043127424/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827131 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827106.849304:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827106.849307:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827106.849310:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043127424:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787827107.424976:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827108.449313:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827109.472948:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827110.496893:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827111.520905:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827111.969006:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827111.969019:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827111.969029:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827111.969041:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9178 00000100:00080000:2.0:1787827111.969054:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827111.969063:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827111.969112:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c380 x1874663043127552/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827111.969144:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827111.969147:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827111.969148:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827111.969152:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9178 00000100:00080000:2.0:1787827111.969155:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827111.969159:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827111.969173:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827111.969184:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c000 x1874663043127680/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827111.969191:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827111.969194:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827111.969196:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827111.969203:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9178 00000100:00100000:1.0:1787827111.969204:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043127552:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787827111.969207:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827111.969215:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827111.969221:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827111.969233:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007dc00 x1874663043127808/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827111.969289:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827111.969330:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827111/real 1787827111] req@ffff9181e007c380 x1874663043127552/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827136 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827111.969345:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c380 x1874663043127552/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827136 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827111.969354:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827111.969357:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827111.969374:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043127552:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827111.969397:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043127680:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827111.969424:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827111.969439:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827111/real 1787827111] req@ffff9181e007c000 x1874663043127680/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827136 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827111.969461:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c000 x1874663043127680/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827136 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827111.969468:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827111.969470:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827111.969472:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043127680:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827111.969476:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043127808:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827111.969492:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827111.969502:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827111/real 1787827111] req@ffff9181e007dc00 x1874663043127808/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827136 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827111.969513:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007dc00 x1874663043127808/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827136 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827111.969521:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827111.969525:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827111.969528:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043127808:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827112.544943:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827113.568961:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827114.593003:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827115.618601:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827116.642851:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827117.088872:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827117.088882:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827117.088886:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827117.088897:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9183 00000100:00080000:2.0:1787827117.088907:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827117.088914:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827117.088957:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e680 x1874663043127936/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827117.088989:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827117.088993:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827117.088994:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827117.088998:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9183 00000100:00080000:2.0:1787827117.089002:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827117.089005:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827117.089014:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827117.089026:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ca80 x1874663043128064/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827117.089035:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827117.089038:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827117.089040:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827117.089043:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9183 00000100:00080000:2.0:1787827117.089046:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827117.089050:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827117.089054:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827117.089064:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007fb80 x1874663043128192/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827117.089911:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043127936:192.168.203.147@tcp:250:kworker.0 00000800:00000100:1.0:1787827117.089968:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827117.090001:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827117/real 1787827117] req@ffff9181e007e680 x1874663043127936/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827142 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827117.090017:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e680 x1874663043127936/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827142 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827117.090094:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827117.090099:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827117.090173:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043127936:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827117.090306:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043128064:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827117.090331:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827117.090354:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827117/real 1787827117] req@ffff9181e007ca80 x1874663043128064/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827142 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827117.090366:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ca80 x1874663043128064/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827142 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827117.090373:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827117.090376:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827117.090380:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043128064:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827117.090385:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043128192:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827117.090402:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827117.090411:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827117/real 1787827117] req@ffff9181e007fb80 x1874663043128192/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827142 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827117.090435:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007fb80 x1874663043128192/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827142 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827117.090442:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827117.090445:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827117.090448:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043128192:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787827117.664971:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827118.688974:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827119.713075:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:3.0:1787827120.096961:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-cliir' failed (-108). 10000000:00000400:3.0:1787827120.096971:0:159735:0:(mgc_request.c:1900:mgc_process_log()) MGC192.168.203.147@tcp: IR log lustre-cliir failed, not fatal: rc = -108 10000000:01000000:3.0:1787827120.096980:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:3.0:1787827120.096982:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff9181c5620000 from 52 10000000:01000000:3.0:1787827120.096986:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:3.0:1787827120.097061:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5b5 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827120.097073:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5b5 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827120.097085:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5b5 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787827120.097106:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d225ff00 going to sleep for 0 seconds 00000100:00100000:3.0:1787827120.097118:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043128320:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:3.0:1787827120.097132:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5b5 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827120.097138:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5b5 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827120.097142:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5b5 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827120.097147:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5b5 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827120.097152:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5b5 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827120.097165:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3480000 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:3.0:1787827120.097168:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5b5 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827120.097175:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5b5 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1787827120.097181:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5b5 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787827120.097186:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:3.0:1787827120.097190:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787827120.097198:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5b5 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827120.097211:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5b5 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787827120.097236:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:3.0:1787827120.097239:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:0.0:1787827120.097435:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827120.097440:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827120.097443:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827120.097449:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9188 00000100:00080000:0.0:1787827120.097458:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827120.097464:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787827120.097484:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5f80 x1874663043128448/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827120.097502:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827120.097505:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827120.097506:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827120.097509:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9188 00000100:00080000:0.0:1787827120.097511:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827120.097513:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787827120.097521:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827120.097528:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4700 x1874663043128576/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827120.097535:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827120.097537:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827120.097539:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827120.097541:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9188 00000100:00080000:0.0:1787827120.097543:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827120.097546:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787827120.097548:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827120.097554:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4380 x1874663043128704/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827120.097604:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043128448:192.168.203.147@tcp:250:kworker.0 00000800:00000100:1.0:1787827120.097683:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827120.097722:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827120/real 1787827120] req@ffff9181c55a5f80 x1874663043128448/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827145 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827120.097737:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5f80 x1874663043128448/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827145 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827120.097782:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827120.097788:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827120.097806:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043128448:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827120.097818:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043128576:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827120.097858:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827120.097872:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827120/real 1787827120] req@ffff9181c55a4700 x1874663043128576/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827145 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827120.097884:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4700 x1874663043128576/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827145 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827120.097892:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827120.097895:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827120.097899:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043128576:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827120.097905:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043128704:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827120.097925:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827120.097936:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827120/real 1787827120] req@ffff9181c55a4380 x1874663043128704/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827145 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827120.097947:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4380 x1874663043128704/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827145 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827120.097954:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827120.097957:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827120.097960:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043128704:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827120.736905:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827121.760952:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827122.785108:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827123.808889:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827124.832938:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827125.280896:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827125.280906:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827125.280910:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827125.280921:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9191 00000100:00080000:0.0:1787827125.280932:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827125.280939:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787827125.280973:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5180 x1874663043128832/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827125.281004:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827125.281008:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827125.281010:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827125.281015:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9191 00000100:00080000:0.0:1787827125.281018:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827125.281022:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787827125.281031:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827125.281045:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5500 x1874663043128960/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827125.281055:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827125.281058:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827125.281059:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827125.281063:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9191 00000100:00100000:1.0:1787827125.281065:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043128832:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787827125.281066:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827125.281071:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787827125.281077:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827125.281086:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4e00 x1874663043129088/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827125.281130:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827125.281168:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827125/real 1787827125] req@ffff9181c55a5180 x1874663043128832/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827150 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827125.281183:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5180 x1874663043128832/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827150 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827125.281194:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827125.281197:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827125.281214:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043128832:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827125.281238:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043128960:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827125.281262:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827125.281276:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827125/real 1787827125] req@ffff9181c55a5500 x1874663043128960/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827150 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827125.281294:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5500 x1874663043128960/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827150 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827125.281302:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827125.281305:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827125.281308:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043128960:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827125.281314:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043129088:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827125.281330:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827125.281339:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827125/real 1787827125] req@ffff9181c55a4e00 x1874663043129088/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827150 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827125.281349:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4e00 x1874663043129088/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827150 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827125.281370:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827125.281373:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827125.281377:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043129088:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787827125.856941:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827126.880894:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827127.904927:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827128.928978:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827129.952952:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827130.400863:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827130.400872:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827130.400877:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827130.400886:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9197 00000100:00080000:0.0:1787827130.400897:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827130.400904:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787827130.400940:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7800 x1874663043129216/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827130.400967:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827130.400970:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827130.400971:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827130.400974:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9197 00000100:00080000:0.0:1787827130.400977:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827130.400980:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787827130.400988:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827130.400999:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4e00 x1874663043129344/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827130.401006:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827130.401008:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827130.401009:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827130.401012:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9197 00000100:00080000:0.0:1787827130.401014:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827130.401016:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787827130.401019:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827130.401022:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043129216:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787827130.401028:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5500 x1874663043129472/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827130.401080:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827130.401121:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827130/real 1787827130] req@ffff9181c55a7800 x1874663043129216/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827155 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827130.401139:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7800 x1874663043129216/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827155 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827130.401148:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827130.401152:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827130.401163:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043129216:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827130.401170:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043129344:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827130.401189:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827130.401202:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827130/real 1787827130] req@ffff9181c55a4e00 x1874663043129344/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827155 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827130.401214:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4e00 x1874663043129344/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827155 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827130.401221:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827130.401224:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827130.401227:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043129344:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827130.401248:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043129472:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827130.401263:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827130.401272:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827130/real 1787827130] req@ffff9181c55a5500 x1874663043129472/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827155 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827130.401282:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5500 x1874663043129472/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827155 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827130.401288:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827130.401291:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827130.401294:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043129472:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827130.976901:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827132.000932:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827133.024939:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827134.048923:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827135.072957:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827135.520873:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827135.520884:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827135.520888:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827135.520898:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9202 00000100:00080000:0.0:1787827135.520908:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827135.520923:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787827135.520953:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5180 x1874663043129600/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827135.520972:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827135.520975:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827135.520976:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827135.520979:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9202 00000100:00080000:0.0:1787827135.520981:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827135.520984:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787827135.520990:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827135.520999:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4380 x1874663043129728/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827135.521005:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827135.521007:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827135.521008:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1787827135.521013:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043129600:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787827135.521018:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9202 00000100:00080000:0.0:1787827135.521022:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827135.521025:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787827135.521028:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827135.521035:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4700 x1874663043129856/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827135.521067:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827135.521093:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827135/real 1787827135] req@ffff9181c55a5180 x1874663043129600/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827160 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827135.521104:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5180 x1874663043129600/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827160 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827135.521111:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827135.521114:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827135.521125:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043129600:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827135.521145:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043129728:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827135.521160:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827135.521170:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827135/real 1787827135] req@ffff9181c55a4380 x1874663043129728/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827160 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827135.521178:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4380 x1874663043129728/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827160 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827135.521183:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827135.521185:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827135.521187:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043129728:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827135.521190:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043129856:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827135.521202:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827135.521211:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827135/real 1787827135] req@ffff9181c55a4700 x1874663043129856/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827160 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827135.521223:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4700 x1874663043129856/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827160 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827135.521229:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827135.521232:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827135.521234:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043129856:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787827136.096938:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827137.120949:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827138.144941:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827139.168955:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:3.0:1787827140.128928:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-client' failed (-108). 10000000:00020000:3.0:1787827140.128938:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:3.0:1787827140.128942:0:159735:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:3.0:1787827140.128943:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff9181c5620000 from 7 10000000:01000000:3.0:1787827140.128946:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00010000:00010000:3.0:1787827140.128984:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5bc lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827140.128991:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5bc lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827140.128999:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5bc lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787827140.129014:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d225ff00 going to sleep for 0 seconds 00000100:00100000:3.0:1787827140.129023:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043129984:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:3.0:1787827140.129035:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5bc lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827140.129043:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5bc lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827140.129050:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5bc lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827140.129057:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5bc lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827140.129063:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5bc lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827140.129069:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3480000 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:3.0:1787827140.129073:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5bc lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827140.129080:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5bc lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1787827140.129086:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5bc lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787827140.129092:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:3.0:1787827140.129096:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787827140.129102:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5bc lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827140.129109:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5bc lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787827140.129130:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:3.0:1787827140.129133:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:1.0:1787827140.129303:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827140.129307:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827140.129310:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827140.129315:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9207 00000100:00080000:1.0:1787827140.129322:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827140.129327:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787827140.129358:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2a00 x1874663043130112/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827140.129385:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827140.129389:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827140.129391:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827140.129395:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9207 00000100:00080000:1.0:1787827140.129399:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827140.129403:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827140.129413:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827140.129424:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3480 x1874663043130240/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827140.129434:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827140.129437:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827140.129439:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827140.129442:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9207 00000100:00080000:1.0:1787827140.129445:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827140.129447:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827140.129451:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827140.129457:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0e00 x1874663043130368/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827140.130309:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043130112:192.168.203.147@tcp:250:kworker.0 00000800:00000100:0.0:1787827140.130366:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827140.130397:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827140/real 1787827140] req@ffff9181c35b2a00 x1874663043130112/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827165 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827140.130413:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2a00 x1874663043130112/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827165 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827140.130428:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827140.130431:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827140.130443:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043130112:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787827140.130450:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043130240:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827140.130471:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827140.130481:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827140/real 1787827140] req@ffff9181c35b3480 x1874663043130240/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827165 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827140.130492:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3480 x1874663043130240/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827165 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827140.130499:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827140.130502:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827140.130505:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043130240:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827140.130510:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043130368:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827140.130524:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827140.130533:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827140/real 1787827140] req@ffff9181c35b0e00 x1874663043130368/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827165 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827140.130543:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0e00 x1874663043130368/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827165 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827140.130550:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827140.130553:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827140.130555:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043130368:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787827140.193043:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827141.217133:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827142.242389:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827143.264906:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827144.288959:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827145.248889:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827145.248898:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827145.248902:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827145.248911:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9211 00000100:00080000:3.0:1787827145.248921:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827145.248928:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827145.248960:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043130496/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827145.248988:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827145.248991:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827145.248993:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827145.248999:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9211 00000100:00080000:3.0:1787827145.249002:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827145.249006:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827145.249014:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827145.249026:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043130624/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827145.249034:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827145.249037:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827145.249039:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827145.249042:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9211 00000100:00080000:3.0:1787827145.249044:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827145.249048:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827145.249053:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827145.249061:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043130752/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827145.249077:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043130496:192.168.203.147@tcp:250:kworker.0 00000800:00000100:0.0:1787827145.249138:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827145.249166:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827145/real 1787827145] req@ffff9181c5636a00 x1874663043130496/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827170 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827145.249179:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043130496/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827170 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827145.249186:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827145.249189:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827145.249201:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043130496:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787827145.249208:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043130624:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827145.249245:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827145.249258:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827145/real 1787827145] req@ffff9181c5637480 x1874663043130624/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827170 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827145.249269:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043130624/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827170 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827145.249277:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827145.249280:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827145.249283:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043130624:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827145.249318:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043130752:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827145.249337:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827145.249348:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827145/real 1787827145] req@ffff9181c5635500 x1874663043130752/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827170 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827145.249361:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043130752/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827170 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827145.249369:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827145.249372:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827145.249376:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043130752:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827145.312912:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827146.336909:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827147.360888:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827148.384914:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827149.408952:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827150.368919:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827150.368934:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827150.368938:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827150.368952:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9216 00000100:00080000:3.0:1787827150.368964:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827150.368978:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827150.369042:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043130880/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827150.369082:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827150.369086:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827150.369088:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827150.369093:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9216 00000100:00080000:3.0:1787827150.369097:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827150.369101:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827150.369112:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827150.369124:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043131008/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827150.369134:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827150.369137:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827150.369139:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827150.369144:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9216 00000100:00080000:3.0:1787827150.369147:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827150.369151:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827150.369156:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827150.369165:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043131136/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827150.369286:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043130880:192.168.203.147@tcp:250:kworker.0 00000800:00000100:0.0:1787827150.369373:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827150.369415:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827150/real 1787827150] req@ffff9181c5637100 x1874663043130880/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827175 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827150.369434:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043130880/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827175 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827150.369444:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827150.369449:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827150.369512:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043130880:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787827150.369562:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043131008:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827150.369601:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827150.369616:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827150/real 1787827150] req@ffff9181c5637800 x1874663043131008/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827175 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827150.369629:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043131008/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827175 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827150.369643:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827150.369646:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827150.369650:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043131008:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827150.369656:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043131136:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827150.369675:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827150.369685:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827150/real 1787827150] req@ffff9181c5636d80 x1874663043131136/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827175 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827150.369696:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043131136/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827175 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827150.369703:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827150.369712:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827150.369715:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043131136:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827150.432968:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827151.456948:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827152.480866:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827153.504882:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827154.528895:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827155.488955:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827155.488969:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827155.488973:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827155.488987:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9222 00000100:00080000:0.0:1787827155.488998:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827155.489006:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787827155.489047:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5f80 x1874663043131264/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827155.489082:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827155.489087:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827155.489088:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827155.489093:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9222 00000100:00080000:0.0:1787827155.489097:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827155.489101:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787827155.489112:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827155.489127:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5c00 x1874663043131392/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827155.489138:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827155.489141:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827155.489143:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827155.489159:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9222 00000100:00080000:0.0:1787827155.489163:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827155.489171:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787827155.489175:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043131264:192.168.203.147@tcp:250:kworker.0 00000008:00100000:0.0:1787827155.489185:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827155.489205:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5880 x1874663043131520/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827155.489258:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827155.489308:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827155/real 1787827155] req@ffff9181c55a5f80 x1874663043131264/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827180 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827155.489326:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5f80 x1874663043131264/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827180 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827155.489337:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827155.489355:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827155.489373:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043131264:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827155.489381:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043131392:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827155.489403:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827155.489415:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827155/real 1787827155] req@ffff9181c55a5c00 x1874663043131392/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827180 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827155.489426:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5c00 x1874663043131392/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827180 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827155.489434:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827155.489437:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827155.489439:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043131392:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827155.489469:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043131520:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827155.489489:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827155.489514:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827155/real 1787827155] req@ffff9181c55a5880 x1874663043131520/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827180 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827155.489525:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5880 x1874663043131520/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827180 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827155.489539:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827155.489543:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827155.489546:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043131520:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787827155.552990:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827156.578272:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827157.601362:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827158.625067:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827159.649023:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:3.0:1787827160.608907:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'params' failed (-108). 10000000:00020000:3.0:1787827160.608921:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:3.0:1787827160.608930:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:3.0:1787827160.608933:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff9181c87b19b8 from 1 10000000:01000000:3.0:1787827160.608937:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00010000:00010000:3.0:1787827160.608997:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5c3 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827160.609012:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5c3 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827160.609026:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5c3 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787827160.609056:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d225ff00 going to sleep for 0 seconds 00000100:00100000:3.0:1787827160.609144:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043131648:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:3.0:1787827160.609177:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5c3 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827160.609188:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5c3 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827160.609201:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5c3 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827160.609210:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5c3 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827160.609217:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5c3 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827160.609225:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3480000 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:3.0:1787827160.609229:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5c3 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827160.609237:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5c3 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1787827160.609245:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5c3 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787827160.609252:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:3.0:1787827160.609256:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787827160.609263:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5c3 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827160.609270:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5c3 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787827160.609302:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:3.0:1787827160.609305:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:2.0:1787827160.609392:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827160.609400:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827160.609403:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827160.609413:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9227 00000100:00080000:2.0:1787827160.609423:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827160.609430:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827160.609456:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ce00 x1874663043131776/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827160.609483:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827160.609487:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827160.609489:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827160.609493:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9227 00000100:00080000:2.0:1787827160.609497:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827160.609503:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827160.609512:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827160.609527:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f480 x1874663043131904/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827160.609534:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827160.609536:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827160.609537:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827160.609539:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9227 00000100:00080000:2.0:1787827160.609542:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827160.609545:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827160.609548:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827160.609554:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d880 x1874663043132032/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827160.609624:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043131776:192.168.203.147@tcp:250:kworker.0 00000800:00000100:1.0:1787827160.609675:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827160.609715:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827160/real 1787827160] req@ffff9181e007ce00 x1874663043131776/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827185 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827160.609891:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ce00 x1874663043131776/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827185 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827160.609900:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827160.609904:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827160.609916:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043131776:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827160.609924:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043131904:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827160.609952:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827160.609970:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827160/real 1787827160] req@ffff9181e007f480 x1874663043131904/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827185 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827160.609983:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f480 x1874663043131904/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827185 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827160.609992:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827160.609998:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827160.610003:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043131904:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827160.610010:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043132032:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827160.610035:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827160.610049:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827160/real 1787827160] req@ffff9181e007d880 x1874663043132032/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827185 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827160.610067:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d880 x1874663043132032/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827185 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827160.610076:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827160.610079:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827160.610082:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043132032:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827160.673077:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827161.696926:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827162.720942:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827163.744919:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827164.768904:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827165.728837:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827165.728844:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827165.728847:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827165.728856:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9232 00000100:00080000:2.0:1787827165.728864:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827165.728869:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827165.728898:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f100 x1874663043132160/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827165.728917:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827165.728920:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827165.728922:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827165.728925:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9232 00000100:00080000:2.0:1787827165.728927:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827165.728929:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827165.728937:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827165.728945:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f800 x1874663043132288/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827165.728952:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827165.728956:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:1.0:1787827165.728957:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043132160:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787827165.728957:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827165.728962:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9232 00000100:00080000:2.0:1787827165.728965:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827165.728969:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827165.728971:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827165.728977:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c700 x1874663043132416/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827165.729064:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043132288:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827165.729104:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043132416:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787827165.792868:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827166.816940:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827167.840875:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1787827168.801081:0:2376:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:0.0:1787827168.801127:0:2376:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1787827168.801144:0:2376:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1787827168.801180:0:2376:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1787827168.801189:0:2376:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:1.0:1787827168.801242:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827165/real 1787827168] req@ffff9181e007f800 x1874663043132288/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827190 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827168.801268:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f800 x1874663043132288/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827190 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827168.801287:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827168.801293:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827168.801297:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043132288:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827168.801311:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827165/real 1787827168] req@ffff9181e007c700 x1874663043132416/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827190 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827168.801323:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c700 x1874663043132416/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827190 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827168.801331:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827168.801334:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827168.801337:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043132416:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827168.801343:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827165/real 1787827168] req@ffff9181e007f100 x1874663043132160/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827190 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827168.801354:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f100 x1874663043132160/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827190 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827168.801361:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827168.801364:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827168.801385:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043132160:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787827168.864898:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827169.888881:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827170.848881:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827170.848893:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827170.848896:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827170.848914:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9237 00000100:00080000:1.0:1787827170.848927:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827170.848934:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787827170.848966:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2680 x1874663043132544/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827170.848991:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827170.848995:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827170.848997:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827170.849001:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9237 00000100:00080000:1.0:1787827170.849004:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827170.849008:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827170.849016:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827170.849029:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1180 x1874663043132672/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827170.849034:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043132544:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787827170.849058:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827170.849061:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827170.849063:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827170.849067:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9237 00000100:00080000:1.0:1787827170.849071:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827170.849074:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827170.849079:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827170.849090:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0000 x1874663043132800/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827170.849111:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827170.849145:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827170/real 1787827170] req@ffff9181c35b2680 x1874663043132544/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827195 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827170.849163:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2680 x1874663043132544/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827195 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827170.849172:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827170.849176:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827170.849194:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043132544:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787827170.849226:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043132672:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827170.849248:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827170.849260:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827170/real 1787827170] req@ffff9181c35b1180 x1874663043132672/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827195 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827170.849271:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1180 x1874663043132672/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827195 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827170.849279:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827170.849282:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827170.849285:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043132672:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827170.849291:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043132800:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827170.849308:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827170.849318:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827170/real 1787827170] req@ffff9181c35b0000 x1874663043132800/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827195 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827170.849333:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0000 x1874663043132800/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827195 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827170.849341:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827170.849344:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827170.849347:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043132800:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827170.912899:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827171.936874:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827172.961050:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827173.984991:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827175.008893:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827175.968871:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827175.968881:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827175.968885:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827175.968894:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9242 00000100:00080000:0.0:1787827175.968902:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827175.968918:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787827175.968951:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6d80 x1874663043132928/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827175.968970:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827175.968972:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827175.968974:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827175.968976:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9242 00000100:00080000:0.0:1787827175.968979:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827175.968983:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787827175.968990:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827175.968998:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043132928:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787827175.969001:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7480 x1874663043133056/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827175.969026:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827175.969029:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827175.969030:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827175.969033:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9242 00000100:00080000:0.0:1787827175.969036:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827175.969039:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787827175.969043:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827175.969050:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7b80 x1874663043133184/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827175.969063:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827175.969089:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827175/real 1787827175] req@ffff9181c55a6d80 x1874663043132928/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827200 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827175.969101:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6d80 x1874663043132928/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827200 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827175.969108:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827175.969110:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827175.969122:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043132928:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827175.969140:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043133056:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827175.969163:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827175.969175:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827175/real 1787827175] req@ffff9181c55a7480 x1874663043133056/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827200 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827175.969187:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7480 x1874663043133056/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827200 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827175.969196:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827175.969199:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827175.969201:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043133056:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827175.969207:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043133184:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827175.969222:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827175.969229:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827175/real 1787827175] req@ffff9181c55a7b80 x1874663043133184/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827200 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827175.969236:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7b80 x1874663043133184/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827200 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827175.969241:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827175.969243:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827175.969245:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043133184:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787827176.032888:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827177.056893:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827178.080887:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827179.104915:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827180.128872:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827181.088858:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827181.088865:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827181.088868:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827181.088877:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9247 00000100:00080000:0.0:1787827181.088886:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827181.088891:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 10000000:01000000:3.0:1787827181.088919:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-sptlrpc' failed (-108). 10000000:00020000:3.0:1787827181.088927:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00100000:0.0:1787827181.088929:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6680 x1874663043133312/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827181.088959:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827181.088963:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827181.088964:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827181.088967:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9247 00000100:00080000:0.0:1787827181.088982:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827181.088985:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787827181.088993:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827181.089003:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7100 x1874663043133440/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827181.089011:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827181.089013:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827181.089014:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827181.089018:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9247 00000100:00100000:1.0:1787827181.089020:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043133312:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787827181.089021:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827181.089026:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787827181.089032:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827181.089040:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7b80 x1874663043133568/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827181.089080:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827181.089108:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827181/real 1787827181] req@ffff9181c55a6680 x1874663043133312/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827206 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827181.089120:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6680 x1874663043133312/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827206 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827181.089127:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827181.089130:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827181.089132:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043133312:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827181.089158:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043133440:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827181.089175:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827181.089183:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827181/real 1787827181] req@ffff9181c55a7100 x1874663043133440/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827206 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827181.089190:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7100 x1874663043133440/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827206 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827181.089195:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827181.089197:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827181.089199:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043133440:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827181.089202:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043133568:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827181.089211:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827181.089216:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827181/real 1787827181] req@ffff9181c55a7b80 x1874663043133568/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827206 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827181.089223:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7b80 x1874663043133568/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827206 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827181.089227:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827181.089229:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827181.089230:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043133568:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827181.152841:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:2.0:1787827182.176859:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:2.0:1787827182.176866:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff9181c5620000 from 341 00010000:00010000:1.0:1787827182.176869:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:2.0:1787827182.176870:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:2.0:1787827182.176919:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f5ca lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827182.176928:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f5ca lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827182.176937:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f5ca lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827182.176952:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5c94a80 going to sleep for 0 seconds 00000100:00100000:2.0:1787827182.176970:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043133696:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:2.0:1787827182.176985:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f5ca lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827182.176993:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f5ca lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827182.177000:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f5ca lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827182.177007:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f5ca lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827182.177014:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f5ca lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827182.177021:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181d1842c00 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:2.0:1787827182.177024:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f5ca lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827182.177032:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f5ca lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1787827182.177040:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f5ca lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787827182.177046:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:2.0:1787827182.177049:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787827182.177055:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f5ca lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827182.177062:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f5ca lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787827182.177082:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:2.0:1787827182.177084:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:3.0:1787827182.177116:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827182.177121:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827182.177123:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827182.177128:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9252 00000100:00080000:3.0:1787827182.177134:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827182.177138:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827182.177153:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043133824/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827182.177168:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827182.177170:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827182.177172:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827182.177174:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9252 00000100:00080000:3.0:1787827182.177176:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827182.177179:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827182.177185:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827182.177192:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043133952/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827182.177197:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043133824:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787827182.177200:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827182.177207:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827182.177210:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827182.177213:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9252 00000100:00080000:3.0:1787827182.177215:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827182.177217:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827182.177221:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827182.177228:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634a80 x1874663043134080/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827182.177259:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827182.177284:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827182/real 1787827182] req@ffff9181c5635c00 x1874663043133824/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827207 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827182.177298:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043133824/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827207 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827182.177306:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827182.177309:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827182.177320:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043133824:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827182.177338:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043133952:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827182.177364:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827182.177373:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827182/real 1787827182] req@ffff9181c5634000 x1874663043133952/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827207 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827182.177382:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043133952/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827207 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827182.177389:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827182.177391:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827182.177394:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043133952:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827182.177399:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043134080:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827182.177412:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827182.177418:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827182/real 1787827182] req@ffff9181c5634a80 x1874663043134080/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827207 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827182.177432:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634a80 x1874663043134080/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827207 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827182.177438:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827182.177440:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827182.177443:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043134080:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827183.200862:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827184.224908:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827185.248876:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827186.272878:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827187.232867:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827187.232874:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827187.232877:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827187.232885:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9253 00000100:00080000:1.0:1787827187.232893:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827187.232899:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787827187.232939:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1c00 x1874663043134208/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827187.232958:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827187.232960:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827187.232962:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827187.232964:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9253 00000100:00080000:1.0:1787827187.232967:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827187.232971:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827187.232979:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827187.232983:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043134208:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827187.232991:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0380 x1874663043134336/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827187.232999:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827187.233001:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827187.233002:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827187.233005:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9253 00000100:00080000:1.0:1787827187.233008:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827187.233010:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827187.233012:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827187.233019:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1500 x1874663043134464/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827187.233033:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827187.233061:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827187/real 1787827187] req@ffff9181c35b1c00 x1874663043134208/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827212 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827187.233073:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1c00 x1874663043134208/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827212 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827187.233079:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827187.233082:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827187.233092:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043134208:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787827187.233108:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043134336:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827187.233124:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827187.233141:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827187/real 1787827187] req@ffff9181c35b0380 x1874663043134336/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827212 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827187.233154:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0380 x1874663043134336/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827212 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827187.233159:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827187.233161:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827187.233163:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043134336:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827187.233168:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043134464:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827187.233179:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827187.233184:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827187/real 1787827187] req@ffff9181c35b1500 x1874663043134464/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827212 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827187.233191:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1500 x1874663043134464/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827212 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827187.233195:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827187.233197:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827187.233198:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043134464:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827187.296905:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827188.320853:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827189.344863:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827190.368866:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827191.392991:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827192.352833:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827192.352841:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827192.352845:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827192.352853:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9258 00000100:00080000:3.0:1787827192.352863:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827192.352868:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827192.352923:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637b80 x1874663043134592/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827192.352952:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827192.352955:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827192.352956:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827192.352960:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9258 00000100:00080000:3.0:1787827192.352963:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827192.352966:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827192.352979:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827192.352993:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634a80 x1874663043134720/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827192.353003:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043134592:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787827192.353004:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827192.353012:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827192.353013:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827192.353017:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9258 00000100:00080000:3.0:1787827192.353020:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827192.353023:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827192.353027:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827192.353036:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043134848/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827192.353066:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827192.353096:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827192/real 1787827192] req@ffff9181c5637b80 x1874663043134592/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827217 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827192.353115:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637b80 x1874663043134592/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827217 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827192.353123:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827192.353126:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827192.353140:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043134592:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787827192.353159:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043134720:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827192.353175:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827192.353187:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827192/real 1787827192] req@ffff9181c5634a80 x1874663043134720/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827217 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827192.353196:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634a80 x1874663043134720/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827217 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827192.353202:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827192.353204:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827192.353206:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043134720:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827192.353210:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043134848:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827192.353221:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827192.353227:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827192/real 1787827192] req@ffff9181c5634000 x1874663043134848/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827217 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827192.353240:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043134848/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827217 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827192.353254:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827192.353257:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827192.353261:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043134848:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827192.416882:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827193.440977:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827194.464926:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827195.488904:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827196.512931:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827197.472869:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827197.472879:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827197.472882:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827197.472897:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9264 00000100:00080000:0.0:1787827197.472908:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827197.472915:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787827197.472953:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7480 x1874663043134976/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827197.472980:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827197.472992:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827197.472994:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827197.472999:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9264 00000100:00080000:0.0:1787827197.473003:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827197.473007:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787827197.473017:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827197.473027:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043134976:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787827197.473032:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6d80 x1874663043135104/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827197.473045:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827197.473048:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827197.473051:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827197.473055:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9264 00000100:00080000:0.0:1787827197.473059:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827197.473063:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787827197.473068:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827197.473079:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5880 x1874663043135232/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827197.473088:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827197.473121:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827197/real 1787827197] req@ffff9181c55a7480 x1874663043134976/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827222 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827197.473139:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7480 x1874663043134976/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827222 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827197.473146:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827197.473149:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827197.473162:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043134976:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827197.473188:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043135104:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827197.473221:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827197.473232:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827197/real 1787827197] req@ffff9181c55a6d80 x1874663043135104/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827222 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827197.473241:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6d80 x1874663043135104/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827222 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827197.473247:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827197.473249:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827197.473252:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043135104:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827197.473256:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043135232:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827197.473273:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827197.473284:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827197/real 1787827197] req@ffff9181c55a5880 x1874663043135232/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827222 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827197.473296:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5880 x1874663043135232/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827222 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827197.473311:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827197.473315:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827197.473318:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043135232:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827197.536918:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827198.560892:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827199.584903:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827200.608882:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827201.632901:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827202.592854:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827202.592862:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827202.592865:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827202.592874:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9269 00000100:00080000:2.0:1787827202.592883:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827202.592889:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 10000000:01000000:0.0:1787827202.592911:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-cliir' failed (-108). 10000000:00000400:0.0:1787827202.592915:0:159735:0:(mgc_request.c:1900:mgc_process_log()) MGC192.168.203.147@tcp: IR log lustre-cliir failed, not fatal: rc = -108 00000100:00100000:2.0:1787827202.592926:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e300 x1874663043135360/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827202.592949:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827202.592952:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827202.592953:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827202.592957:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9269 00000100:00080000:2.0:1787827202.592959:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827202.592962:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827202.592970:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827202.592978:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ea00 x1874663043135488/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827202.592986:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827202.592989:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827202.592990:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827202.592993:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9269 00000100:00080000:2.0:1787827202.592995:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827202.592997:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827202.593000:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827202.593007:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d180 x1874663043135616/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827202.593014:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043135360:192.168.203.147@tcp:250:kworker.0 00000800:00000100:1.0:1787827202.593081:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827202.593117:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827202/real 1787827202] req@ffff9181e007e300 x1874663043135360/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827227 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827202.593152:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e300 x1874663043135360/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827227 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827202.593162:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827202.593166:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827202.593170:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043135360:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827202.593177:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043135488:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827202.593201:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827202.593215:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827202/real 1787827202] req@ffff9181e007ea00 x1874663043135488/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827227 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827202.593227:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ea00 x1874663043135488/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827227 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827202.593237:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827202.593241:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827202.593244:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043135488:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827202.593267:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043135616:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827202.593294:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827202.593303:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827202/real 1787827202] req@ffff9181e007d180 x1874663043135616/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827227 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827202.593311:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d180 x1874663043135616/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827227 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827202.593317:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827202.593319:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827202.593321:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043135616:192.168.203.147@tcp:38:kworker.0 10000000:01000000:0.0:1787827202.598637:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:0.0:1787827202.598641:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff9181c5620000 from 52 10000000:01000000:0.0:1787827202.598644:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:0.0:1787827202.598678:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0513200/0xbef42c0221d6f5d1 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827202.598687:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e0513200/0xbef42c0221d6f5d1 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827202.598695:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e0513200/0xbef42c0221d6f5d1 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827202.598709:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d01a5300 going to sleep for 0 seconds 00000100:00100000:0.0:1787827202.598715:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043135744:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:0.0:1787827202.598726:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e0513200/0xbef42c0221d6f5d1 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827202.598732:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e0513200/0xbef42c0221d6f5d1 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827202.598741:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0513200/0xbef42c0221d6f5d1 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827202.598761:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e0513200/0xbef42c0221d6f5d1 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827202.598767:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e0513200/0xbef42c0221d6f5d1 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827202.598772:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0513200 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:0.0:1787827202.598775:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e0513200/0xbef42c0221d6f5d1 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827202.598780:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e0513200/0xbef42c0221d6f5d1 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1787827202.598786:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e0513200/0xbef42c0221d6f5d1 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787827202.598790:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:0.0:1787827202.598793:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787827202.598797:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e0513200/0xbef42c0221d6f5d1 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827202.598802:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e0513200/0xbef42c0221d6f5d1 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787827202.598817:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:0.0:1787827202.598832:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:3.0:1787827202.598894:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827202.598907:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827202.598914:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827202.598921:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9274 00000100:00080000:3.0:1787827202.598928:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827202.598933:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827202.598953:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043135872/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827202.599009:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043135872:192.168.203.147@tcp:250:kworker.0 00000800:00000100:1.0:1787827202.599048:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827202.599066:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827202/real 1787827202] req@ffff9181c5635c00 x1874663043135872/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827227 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827202.599077:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043135872/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827227 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827202.599084:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827202.599087:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827202.599097:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043135872:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787827202.656887:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827203.680902:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827204.704907:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827205.728900:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827206.752906:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827207.712940:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827207.712950:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827207.712954:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827207.712965:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9274 00000100:00080000:2.0:1787827207.712976:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827207.712983:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827207.713020:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d500 x1874663043136000/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827207.713048:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827207.713051:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827207.713053:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827207.713056:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9274 00000100:00080000:2.0:1787827207.713059:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827207.713062:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827207.713071:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827207.713081:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007df80 x1874663043136128/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827207.713088:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827207.713090:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827207.713091:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827207.713095:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9274 00000100:00080000:2.0:1787827207.713096:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827207.713098:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827207.713105:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827207.713114:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043136000:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787827207.713116:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ed80 x1874663043136256/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827207.713202:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827207.713240:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827207/real 1787827207] req@ffff9181e007d500 x1874663043136000/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827232 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827207.713255:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d500 x1874663043136000/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827232 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827207.713264:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827207.713268:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827207.713282:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043136000:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827207.713288:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043136128:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827207.713306:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827207.713319:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827207/real 1787827207] req@ffff9181e007df80 x1874663043136128/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827232 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827207.713337:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007df80 x1874663043136128/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827232 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827207.713344:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827207.713347:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827207.713350:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043136128:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827207.713374:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043136256:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827207.713387:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827207.713393:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827207/real 1787827207] req@ffff9181e007ed80 x1874663043136256/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827232 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827207.713401:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ed80 x1874663043136256/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827232 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827207.713406:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827207.713408:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827207.713410:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043136256:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827207.776958:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827208.800936:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827209.824911:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827210.848898:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827211.872907:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827212.832871:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827212.832879:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827212.832882:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827212.832890:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9279 00000100:00080000:1.0:1787827212.832900:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827212.832906:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787827212.832944:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3100 x1874663043136384/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827212.832977:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827212.832982:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827212.832984:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827212.832988:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9279 00000100:00080000:1.0:1787827212.832993:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827212.832998:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827212.833008:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827212.833022:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043136384:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827212.833028:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1500 x1874663043136512/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827212.833037:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827212.833040:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827212.833041:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827212.833044:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9279 00000100:00080000:1.0:1787827212.833047:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827212.833050:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827212.833054:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827212.833063:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0380 x1874663043136640/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827212.833083:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827212.833115:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827212/real 1787827212] req@ffff9181c35b3100 x1874663043136384/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827237 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827212.833129:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3100 x1874663043136384/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827237 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827212.833137:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827212.833140:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827212.833156:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043136384:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787827212.833182:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043136512:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827212.833211:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827212.833227:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827212/real 1787827212] req@ffff9181c35b1500 x1874663043136512/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827237 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827212.833242:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1500 x1874663043136512/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827237 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827212.833252:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827212.833255:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827212.833259:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043136512:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827212.833266:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043136640:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827212.833286:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827212.833295:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827212/real 1787827212] req@ffff9181c35b0380 x1874663043136640/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827237 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827212.833309:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0380 x1874663043136640/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827237 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827212.833315:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827212.833317:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827212.833319:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043136640:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827212.896931:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827213.920880:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827214.944892:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827215.968893:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827216.992923:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827217.952871:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827217.952883:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827217.952887:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827217.952898:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9284 00000100:00080000:1.0:1787827217.952908:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827217.952915:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787827217.952952:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1c00 x1874663043136768/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827217.952981:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827217.952985:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827217.952987:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827217.952992:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9284 00000100:00080000:1.0:1787827217.952996:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827217.953002:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827217.953011:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827217.953020:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043136768:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827217.953029:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0000 x1874663043136896/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827217.953043:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827217.953047:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827217.953049:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827217.953054:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9284 00000100:00080000:1.0:1787827217.953068:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000800:00000100:0.0:1787827217.953074:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1787827217.953076:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827217.953084:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827217.953101:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1180 x1874663043137024/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827217.953109:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827217/real 1787827217] req@ffff9181c35b1c00 x1874663043136768/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827242 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827217.953139:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1c00 x1874663043136768/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827242 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827217.953148:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827217.953151:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827217.953170:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043136768:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787827217.953197:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043136896:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827217.953224:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827217.953240:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827217/real 1787827217] req@ffff9181c35b0000 x1874663043136896/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827242 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827217.953255:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0000 x1874663043136896/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827242 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827217.953267:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827217.953270:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827217.953272:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043136896:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827217.953277:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043137024:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827217.953295:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827217.953304:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827217/real 1787827217] req@ffff9181c35b1180 x1874663043137024/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827242 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827217.953313:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1180 x1874663043137024/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827242 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827217.953319:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827217.953322:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827217.953324:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043137024:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827218.016895:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827219.040943:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827220.064889:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827221.088951:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827222.112914:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827223.072873:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827223.072884:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827223.072888:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827223.072899:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9289 00000100:00080000:1.0:1787827223.072910:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827223.072918:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 10000000:01000000:3.0:1787827223.072949:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-client' failed (-108). 10000000:00020000:3.0:1787827223.072956:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00100000:1.0:1787827223.072958:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2680 x1874663043137152/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 10000000:01000000:3.0:1787827223.072963:0:159735:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:3.0:1787827223.072969:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff9181c5620000 from 7 10000000:01000000:3.0:1787827223.072975:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00000100:00080000:1.0:1787827223.072990:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827223.072996:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827223.072999:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827223.073004:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9289 00000100:00080000:1.0:1787827223.073009:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827223.073015:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:3.0:1787827223.073018:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5d8 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000008:00100000:1.0:1787827223.073028:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:3.0:1787827223.073032:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5d8 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787827223.073051:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0e00 x1874663043137408/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:3.0:1787827223.073051:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5d8 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787827223.073075:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827223.073079:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827223.073085:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787827223.073086:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d225ff00 going to sleep for 0 seconds 00000100:00100000:0.0:1787827223.073087:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043137152:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787827223.073098:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9289 00000100:00100000:3.0:1787827223.073104:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043137280:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00080000:1.0:1787827223.073109:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827223.073117:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:3.0:1787827223.073124:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5d8 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000008:00100000:1.0:1787827223.073125:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:3.0:1787827223.073140:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5d8 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787827223.073145:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3480 x1874663043137536/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:3.0:1787827223.073158:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5d8 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827223.073169:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5d8 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827223.073180:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5d8 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000800:00000100:0.0:1787827223.073183:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00010000:00010000:3.0:1787827223.073195:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3480000 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:3.0:1787827223.073200:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5d8 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827223.073208:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5d8 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1787827223.073214:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5d8 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787827223.073220:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:3.0:1787827223.073225:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787827223.073234:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5d8 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827223.073237:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827223/real 1787827223] req@ffff9181c35b2680 x1874663043137152/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827248 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:3.0:1787827223.073285:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f5d8 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827223.073294:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2680 x1874663043137152/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827248 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827223.073315:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827223.073322:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 10000000:01000000:3.0:1787827223.073330:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00100000:0.0:1787827223.073332:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043137152:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787827223.073334:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00100000:0.0:1787827223.073371:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043137408:192.168.203.147@tcp:38:kworker.0 00000100:00080000:1.0:1787827223.073388:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827223.073394:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827223.073397:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000800:00000100:0.0:1787827223.073403:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1787827223.073409:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9294 00000100:00100000:0.0:1787827223.073423:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827223/real 1787827223] req@ffff9181c35b0e00 x1874663043137408/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827248 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827223.073423:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827223.073430:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787827223.073459:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2a00 x1874663043137664/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827223.073460:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0e00 x1874663043137408/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827248 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827223.073491:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827223.073496:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827223.073501:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043137408:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827223.073509:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043137536:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827223.073544:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827223.073562:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827223/real 1787827223] req@ffff9181c35b3480 x1874663043137536/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827248 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827223.073576:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3480 x1874663043137536/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827248 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827223.073584:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827223.073588:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827223.073592:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043137536:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827223.073608:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043137664:192.168.203.147@tcp:250:kworker.0 00000800:00000100:0.0:1787827223.073630:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827223.073642:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827223/real 1787827223] req@ffff9181c35b2a00 x1874663043137664/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827248 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827223.073655:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2a00 x1874663043137664/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827248 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827223.073663:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827223.073667:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827223.073700:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043137664:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787827223.136962:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827224.160917:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827225.184916:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827226.208873:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827227.232910:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827228.192885:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827228.192906:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827228.192910:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827228.192924:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9294 00000100:00080000:3.0:1787827228.192936:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827228.192943:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827228.192974:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043137792/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827228.192996:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827228.192999:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827228.193001:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827228.193004:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9294 00000100:00080000:3.0:1787827228.193006:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827228.193009:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827228.193016:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827228.193025:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043137920/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827228.193032:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827228.193035:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827228.193037:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827228.193040:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9294 00000100:00080000:3.0:1787827228.193042:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1787827228.193046:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043137792:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787827228.193047:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827228.193054:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827228.193063:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043138048/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827228.193105:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827228.193135:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827228/real 1787827228] req@ffff9181c5636d80 x1874663043137792/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827253 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827228.193149:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043137792/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827253 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827228.193157:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827228.193160:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827228.193170:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043137792:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787827228.193176:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043137920:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827228.193193:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827228.193204:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827228/real 1787827228] req@ffff9181c5637800 x1874663043137920/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827253 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827228.193213:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043137920/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827253 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827228.193223:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827228.193226:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827228.193228:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043137920:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827228.193246:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043138048:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827228.193257:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827228.193263:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827228/real 1787827228] req@ffff9181c5637100 x1874663043138048/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827253 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827228.193270:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043138048/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827253 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827228.193275:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827228.193277:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827228.193279:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043138048:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827228.256888:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827229.280869:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827230.304911:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827231.328926:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827232.352909:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827233.312808:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827233.312816:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827233.312819:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827233.312827:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9299 00000100:00080000:3.0:1787827233.312837:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827233.312843:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827233.312873:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043138176/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827233.312897:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827233.312900:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827233.312901:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827233.312904:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9299 00000100:00080000:3.0:1787827233.312906:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827233.312909:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827233.312916:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827233.312925:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043138304/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827233.312939:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1787827233.312942:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043138176:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787827233.312943:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827233.312945:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827233.312953:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9299 00000100:00080000:3.0:1787827233.312956:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827233.312958:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827233.312961:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827233.312970:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043138432/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827233.313046:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043138304:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827233.313071:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043138432:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827233.376891:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827234.400882:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827235.424885:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1787827236.385023:0:2377:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:1.0:1787827236.385068:0:2377:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787827236.385086:0:2377:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1787827236.385122:0:2377:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1787827236.385132:0:2377:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:0.0:1787827236.385217:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827233/real 1787827236] req@ffff9181c5637480 x1874663043138304/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827258 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827236.385268:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043138304/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827258 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827236.385292:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827236.385298:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827236.385304:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043138304:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827236.385315:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827233/real 1787827236] req@ffff9181c5636a00 x1874663043138432/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827258 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827236.385331:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043138432/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827258 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827236.385341:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827236.385346:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827236.385350:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043138432:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827236.385360:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827233/real 1787827236] req@ffff9181c5635500 x1874663043138176/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827258 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827236.385377:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043138176/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827258 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827236.385386:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827236.385391:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827236.385419:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043138176:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787827236.448877:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827237.472930:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827238.432881:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827238.432893:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827238.432897:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827238.432910:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9305 00000100:00080000:2.0:1787827238.432920:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827238.432927:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827238.432963:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ed80 x1874663043138560/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827238.432994:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827238.432998:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827238.433000:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827238.433005:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9305 00000100:00080000:2.0:1787827238.433009:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827238.433014:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827238.433026:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827238.433041:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007df80 x1874663043138688/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827238.433053:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043138560:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787827238.433055:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827238.433064:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827238.433067:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827238.433073:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9305 00000100:00080000:2.0:1787827238.433077:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827238.433081:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827238.433087:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827238.433104:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d500 x1874663043138816/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827238.433147:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827238.433185:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827238/real 1787827238] req@ffff9181e007ed80 x1874663043138560/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827263 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827238.433206:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ed80 x1874663043138560/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827263 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827238.433219:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827238.433224:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827238.433246:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043138560:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787827238.433282:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043138688:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827238.433309:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827238.433325:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827238/real 1787827238] req@ffff9181e007df80 x1874663043138688/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827263 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827238.433340:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007df80 x1874663043138688/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827263 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827238.433350:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827238.433354:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827238.433358:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043138688:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827238.433367:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043138816:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827238.433392:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827238.433405:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827238/real 1787827238] req@ffff9181e007d500 x1874663043138816/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827263 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827238.433419:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d500 x1874663043138816/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827263 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827238.433436:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827238.433440:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827238.433444:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043138816:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827238.496877:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827239.522319:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827240.544884:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827241.568887:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827242.592881:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827243.552839:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827243.552847:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827243.552850:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827243.552859:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9310 00000100:00080000:3.0:1787827243.552869:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827243.552875:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 10000000:01000000:2.0:1787827243.552889:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'params' failed (-108). 10000000:00020000:2.0:1787827243.552894:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:2.0:1787827243.552898:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:2.0:1787827243.552900:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff9181c87b19b8 from 1 00000100:00100000:3.0:1787827243.552905:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043138944/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 10000000:01000000:2.0:1787827243.552910:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00000100:00080000:3.0:1787827243.552930:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827243.552934:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827243.552935:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827243.552939:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9310 00000100:00080000:3.0:1787827243.552941:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827243.552946:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:2.0:1787827243.552947:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1843000/0xbef42c0221d6f5df lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000008:00100000:3.0:1787827243.552954:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:2.0:1787827243.552966:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181d1843000/0xbef42c0221d6f5df lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787827243.552970:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635f80 x1874663043139200/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827243.552985:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043138944:192.168.203.147@tcp:250:kworker.0 00010000:00010000:2.0:1787827243.552993:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181d1843000/0xbef42c0221d6f5df lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787827243.552994:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827243.553002:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827243.553005:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1787827243.553019:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5c94a80 going to sleep for 0 seconds 00000100:00080000:3.0:1787827243.553021:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9310 00000100:00100000:2.0:1787827243.553029:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043139072:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00080000:3.0:1787827243.553033:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00010000:00010000:2.0:1787827243.553042:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181d1843000/0xbef42c0221d6f5df lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787827243.553042:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:0.0:1787827243.553047:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:3.0:1787827243.553054:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:2.0:1787827243.553055:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181d1843000/0xbef42c0221d6f5df lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827243.553064:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1843000/0xbef42c0221d6f5df lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827243.553070:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181d1843000/0xbef42c0221d6f5df lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787827243.553075:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634e00 x1874663043139328/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:2.0:1787827243.553079:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181d1843000/0xbef42c0221d6f5df lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827243.553086:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827243/real 1787827243] req@ffff9181c5634380 x1874663043138944/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827268 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:2.0:1787827243.553100:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181d1843000 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:2.0:1787827243.553106:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181d1843000/0xbef42c0221d6f5df lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827243.553120:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181d1843000/0xbef42c0221d6f5df lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827243.553123:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043138944/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827268 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 10000000:00010000:2.0:1787827243.553137:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181d1843000/0xbef42c0221d6f5df lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787827243.553146:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:2.0:1787827243.553151:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00000100:00080000:0.0:1787827243.553153:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00010000:00010000:2.0:1787827243.553159:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181d1843000/0xbef42c0221d6f5df lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787827243.553160:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827243.553168:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043138944:192.168.203.147@tcp:250:kworker.0 00010000:00010000:2.0:1787827243.553171:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181d1843000/0xbef42c0221d6f5df lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787827243.553190:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:2.0:1787827243.553192:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00100000:0.0:1787827243.553193:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043139200:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827243.553217:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827243.553230:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827243/real 1787827243] req@ffff9181c5635f80 x1874663043139200/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827268 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827243.553237:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1787827243.553241:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635f80 x1874663043139200/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827268 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827243.553244:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827243.553254:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827243.553256:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00080000:3.0:1787827243.553257:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1787827243.553260:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043139200:192.168.203.147@tcp:38:kworker.0 00000100:00080000:3.0:1787827243.553266:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9315 00000100:00100000:0.0:1787827243.553270:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043139328:192.168.203.147@tcp:38:kworker.0 00000100:00080000:3.0:1787827243.553275:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827243.553283:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:0.0:1787827243.553288:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827243.553300:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043139456/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827243.553301:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827243/real 1787827243] req@ffff9181c5634e00 x1874663043139328/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827268 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827243.553329:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634e00 x1874663043139328/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827268 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827243.553335:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827243.553337:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827243.553340:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043139328:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827243.553350:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043139456:192.168.203.147@tcp:250:kworker.0 00000800:00000100:0.0:1787827243.553362:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827243.553369:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827243/real 1787827243] req@ffff9181c5634380 x1874663043139456/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827268 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827243.553377:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043139456/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827268 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827243.553382:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827243.553384:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827243.553394:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043139456:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787827243.616892:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827244.640883:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827245.664864:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827246.688895:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827247.712845:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827248.672878:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827248.672888:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827248.672892:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827248.672903:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9315 00000100:00080000:1.0:1787827248.672914:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827248.672920:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787827248.672961:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0a80 x1874663043139584/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827248.672991:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827248.672994:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827248.672996:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827248.673000:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9315 00000100:00080000:1.0:1787827248.673002:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827248.673005:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827248.673014:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827248.673024:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2d80 x1874663043139712/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827248.673031:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827248.673034:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827248.673035:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827248.673038:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9315 00000100:00080000:1.0:1787827248.673040:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827248.673042:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827248.673046:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827248.673051:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043139584:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827248.673058:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0700 x1874663043139840/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827248.673126:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827248.673164:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827248/real 1787827248] req@ffff9181c35b0a80 x1874663043139584/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827273 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827248.673180:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0a80 x1874663043139584/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827273 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827248.673198:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827248.673201:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827248.673212:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043139584:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787827248.673218:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043139712:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827248.673242:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827248.673255:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827248/real 1787827248] req@ffff9181c35b2d80 x1874663043139712/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827273 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827248.673269:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2d80 x1874663043139712/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827273 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827248.673278:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827248.673281:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827248.673285:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043139712:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827248.673313:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043139840:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827248.673330:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827248.673338:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827248/real 1787827248] req@ffff9181c35b0700 x1874663043139840/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827273 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827248.673346:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0700 x1874663043139840/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827273 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827248.673356:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827248.673358:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827248.673361:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043139840:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827248.736920:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827249.760943:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827250.784862:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827251.808918:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827252.832891:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827253.792858:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827253.792866:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827253.792869:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827253.792879:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9320 00000100:00080000:3.0:1787827253.792889:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827253.792895:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827253.792928:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043139968/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827253.792963:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827253.792968:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827253.792970:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827253.792974:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9320 00000100:00080000:3.0:1787827253.792979:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827253.792983:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827253.792991:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827253.793002:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043139968:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827253.793007:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043140096/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827253.793019:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827253.793022:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827253.793024:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827253.793028:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9320 00000100:00080000:3.0:1787827253.793031:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827253.793034:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827253.793038:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827253.793049:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043140224/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827253.793069:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827253.793104:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827253/real 1787827253] req@ffff9181c5635500 x1874663043139968/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827278 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827253.793126:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043139968/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827278 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827253.793133:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827253.793137:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827253.793150:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043139968:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827253.793213:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043140096:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827253.793233:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827253.793243:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827253/real 1787827253] req@ffff9181c5636a00 x1874663043140096/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827278 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827253.793252:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043140096/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827278 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827253.793257:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827253.793260:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827253.793262:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043140096:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827253.793266:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043140224:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827253.793277:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827253.793283:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827253/real 1787827253] req@ffff9181c5637480 x1874663043140224/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827278 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827253.793291:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043140224/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827278 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827253.793301:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827253.793303:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827253.793305:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043140224:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827253.856927:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827254.880901:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827255.904870:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827256.928927:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827257.952893:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827258.912838:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827258.912846:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827258.912850:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827258.912858:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9325 00000100:00080000:1.0:1787827258.912874:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827258.912880:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787827258.912912:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1f80 x1874663043140352/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827258.912938:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827258.912941:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827258.912942:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827258.912945:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9325 00000100:00080000:1.0:1787827258.912947:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827258.912950:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827258.912957:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827258.912966:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3800 x1874663043140480/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827258.912975:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827258.912979:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:0.0:1787827258.912980:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043140352:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787827258.912981:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827258.912989:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9325 00000100:00080000:1.0:1787827258.912992:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827258.912994:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827258.912998:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827258.913006:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3b80 x1874663043140608/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827258.913069:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827258.913115:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827258/real 1787827258] req@ffff9181c35b1f80 x1874663043140352/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827283 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827258.913133:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1f80 x1874663043140352/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827283 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827258.913143:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827258.913147:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827258.913165:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043140352:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787827258.913189:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043140480:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827258.913217:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827258.913229:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827258/real 1787827258] req@ffff9181c35b3800 x1874663043140480/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827283 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827258.913238:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3800 x1874663043140480/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827283 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827258.913243:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827258.913246:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827258.913248:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043140480:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827258.913253:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043140608:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827258.913266:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827258.913272:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827258/real 1787827258] req@ffff9181c35b3b80 x1874663043140608/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827283 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827258.913280:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3b80 x1874663043140608/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827283 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827258.913285:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827258.913287:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827258.913289:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043140608:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827258.976923:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827260.000895:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827261.024893:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827262.048936:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827263.072940:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827264.032875:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827264.032883:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827264.032886:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827264.032895:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9330 10000000:01000000:2.0:1787827264.032896:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-sptlrpc' failed (-108). 10000000:00020000:2.0:1787827264.032903:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00080000:3.0:1787827264.032912:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827264.032919:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827264.032948:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043140736/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827264.032969:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827264.032972:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827264.032973:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827264.032976:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9330 00000100:00080000:3.0:1787827264.032979:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827264.032981:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827264.032988:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827264.032997:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043140864/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827264.033005:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827264.033008:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827264.033011:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1787827264.033012:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043140736:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787827264.033016:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9330 00000100:00080000:3.0:1787827264.033023:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827264.033027:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827264.033032:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827264.033044:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043140992/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827264.033072:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827264.033102:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827264/real 1787827264] req@ffff9181c5637100 x1874663043140736/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827289 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827264.033116:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043140736/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827289 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827264.033124:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827264.033126:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827264.033129:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043140736:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787827264.033146:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043140864:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827264.033160:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827264.033168:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827264/real 1787827264] req@ffff9181c5637800 x1874663043140864/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827289 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827264.033177:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043140864/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827289 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827264.033183:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827264.033185:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827264.033187:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043140864:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827264.033190:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043140992:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827264.033200:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827264.033206:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827264/real 1787827264] req@ffff9181c5636d80 x1874663043140992/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827289 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827264.033213:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043140992/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827289 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827264.033218:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827264.033220:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827264.033222:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043140992:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827264.096912:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827265.120907:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:2.0:1787827265.568824:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:2.0:1787827265.568832:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff9181c5620000 from 341 10000000:01000000:2.0:1787827265.568837:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:2.0:1787827265.568893:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1842200/0xbef42c0221d6f5e6 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827265.568906:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181d1842200/0xbef42c0221d6f5e6 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827265.568917:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181d1842200/0xbef42c0221d6f5e6 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827265.568937:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5c94a80 going to sleep for 0 seconds 00000100:00100000:2.0:1787827265.568950:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043141120:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:2.0:1787827265.568964:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181d1842200/0xbef42c0221d6f5e6 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827265.568973:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181d1842200/0xbef42c0221d6f5e6 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827265.568980:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1842200/0xbef42c0221d6f5e6 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827265.568987:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181d1842200/0xbef42c0221d6f5e6 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827265.568994:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181d1842200/0xbef42c0221d6f5e6 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827265.569000:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181d1842200 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:2.0:1787827265.569004:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181d1842200/0xbef42c0221d6f5e6 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827265.569011:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181d1842200/0xbef42c0221d6f5e6 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1787827265.569018:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181d1842200/0xbef42c0221d6f5e6 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787827265.569024:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:2.0:1787827265.569027:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787827265.569033:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181d1842200/0xbef42c0221d6f5e6 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827265.569044:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181d1842200/0xbef42c0221d6f5e6 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787827265.569063:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:2.0:1787827265.569066:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:3.0:1787827265.569122:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827265.569127:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827265.569130:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827265.569137:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9335 00000100:00080000:3.0:1787827265.569147:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827265.569153:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827265.569187:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043141248/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827265.569212:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827265.569216:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827265.569218:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827265.569222:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9335 00000100:00080000:3.0:1787827265.569227:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827265.569233:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827265.569243:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827265.569251:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043141248:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827265.569260:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043141376/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827265.569271:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827265.569273:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827265.569275:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827265.569278:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9335 00000100:00080000:3.0:1787827265.569281:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827265.569284:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827265.569290:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827265.569298:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634a80 x1874663043141504/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827265.569310:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827265.569339:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827265/real 1787827265] req@ffff9181c5635c00 x1874663043141248/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827290 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827265.569352:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043141248/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827290 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827265.569365:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827265.569367:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827265.569378:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043141248:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787827265.569409:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043141376:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827265.569428:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827265.569437:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827265/real 1787827265] req@ffff9181c5634000 x1874663043141376/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827290 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827265.569446:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043141376/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827290 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827265.569451:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827265.569453:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827265.569456:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043141376:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827265.569459:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043141504:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827265.569469:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827265.569475:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827265/real 1787827265] req@ffff9181c5634a80 x1874663043141504/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827290 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827265.569483:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634a80 x1874663043141504/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827290 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827265.569488:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827265.569490:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827265.569492:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043141504:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827266.144955:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827267.168904:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827268.192911:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827269.216905:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827270.240837:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827270.688830:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827270.688841:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827270.688845:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827270.688853:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9337 00000100:00080000:2.0:1787827270.688863:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827270.688869:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827270.688908:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d180 x1874663043141632/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827270.688934:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827270.688938:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827270.688939:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827270.688942:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9337 00000100:00080000:2.0:1787827270.688945:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827270.688948:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827270.688956:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827270.688967:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ea00 x1874663043141760/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827270.688975:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827270.688978:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827270.688981:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827270.688986:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9337 00000100:00080000:2.0:1787827270.688990:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827270.688995:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827270.689001:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827270.689011:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043141632:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787827270.689015:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e300 x1874663043141888/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827270.689070:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827270.689101:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827270/real 1787827270] req@ffff9181e007d180 x1874663043141632/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827295 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827270.689115:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d180 x1874663043141632/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827295 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827270.689123:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827270.689126:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827270.689144:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043141632:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787827270.689150:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043141760:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827270.689168:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827270.689177:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827270/real 1787827270] req@ffff9181e007ea00 x1874663043141760/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827295 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827270.689185:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ea00 x1874663043141760/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827295 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827270.689191:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827270.689193:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827270.689195:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043141760:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827270.689211:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043141888:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827270.689225:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827270.689230:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827270/real 1787827270] req@ffff9181e007e300 x1874663043141888/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827295 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827270.689238:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e300 x1874663043141888/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827295 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827270.689243:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827270.689245:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827270.689246:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043141888:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827271.264929:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827272.288862:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827273.312909:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827274.336860:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827275.360884:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827275.808853:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827275.808861:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827275.808865:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827275.808873:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9342 00000100:00080000:3.0:1787827275.808882:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827275.808888:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827275.808920:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637b80 x1874663043142016/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827275.808940:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827275.808943:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827275.808944:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827275.808947:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9342 00000100:00080000:3.0:1787827275.808950:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827275.808952:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827275.808959:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827275.808969:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634a80 x1874663043142144/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827275.808977:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827275.808979:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827275.808981:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827275.808985:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9342 00000100:00080000:3.0:1787827275.808987:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827275.808989:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827275.808993:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827275.808998:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043142016:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827275.809002:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043142272/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827275.809062:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827275.809098:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827275/real 1787827275] req@ffff9181c5637b80 x1874663043142016/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827300 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827275.809115:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637b80 x1874663043142016/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827300 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827275.809125:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827275.809129:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827275.809143:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043142016:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787827275.809150:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043142144:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827275.809173:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827275.809185:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827275/real 1787827275] req@ffff9181c5634a80 x1874663043142144/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827300 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827275.809203:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634a80 x1874663043142144/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827300 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827275.809211:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827275.809214:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827275.809218:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043142144:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827275.809240:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043142272:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827275.809258:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827275.809268:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827275/real 1787827275] req@ffff9181c5634000 x1874663043142272/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827300 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827275.809279:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043142272/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827300 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827275.809286:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827275.809289:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827275.809292:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043142272:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787827276.384878:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827277.408880:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827278.432926:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827279.456890:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827280.480911:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827280.928840:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827280.928848:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827280.928853:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827280.928863:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9347 00000100:00080000:3.0:1787827280.928874:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827280.928882:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827280.928920:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043142400/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827280.928943:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827280.928946:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827280.928947:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827280.928950:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9347 00000100:00080000:3.0:1787827280.928953:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827280.928956:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827280.928964:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827280.928973:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043142528/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827280.928983:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043142400:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787827280.928990:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827280.928993:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827280.928994:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827280.928997:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9347 00000100:00080000:3.0:1787827280.928999:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827280.929002:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827280.929005:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827280.929012:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043142656/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827280.929042:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827280.929074:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827280/real 1787827280] req@ffff9181c5635c00 x1874663043142400/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827305 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827280.929088:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043142400/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827305 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827280.929100:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827280.929103:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827280.929119:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043142400:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787827280.929138:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043142528:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827280.929155:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827280.929170:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827280/real 1787827280] req@ffff9181c5636d80 x1874663043142528/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827305 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827280.929179:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043142528/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827305 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827280.929184:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827280.929187:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827280.929189:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043142528:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827280.929193:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043142656:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827280.929203:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827280.929209:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827280/real 1787827280] req@ffff9181c5637800 x1874663043142656/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827305 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827280.929216:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043142656/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827305 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827280.929221:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827280.929223:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827280.929225:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043142656:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827281.504959:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827282.528980:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827283.552890:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827284.576871:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827285.600920:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827286.048852:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 10000000:01000000:2.0:1787827286.048852:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-cliir' failed (-108). 10000000:00000400:2.0:1787827286.048861:0:159735:0:(mgc_request.c:1900:mgc_process_log()) MGC192.168.203.147@tcp: IR log lustre-cliir failed, not fatal: rc = -108 00000100:00080000:1.0:1787827286.048863:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827286.048867:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 10000000:01000000:2.0:1787827286.048870:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:2.0:1787827286.048874:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff9181c5620000 from 52 00000100:00080000:1.0:1787827286.048875:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9352 10000000:01000000:2.0:1787827286.048882:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00000100:00080000:1.0:1787827286.048886:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827286.048894:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787827286.048919:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3b80 x1874663043142912/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:2.0:1787827286.048935:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1842200/0xbef42c0221d6f5ed lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787827286.048942:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827286.048946:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00010000:00010000:2.0:1787827286.048946:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181d1842200/0xbef42c0221d6f5ed lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787827286.048950:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827286.048959:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9352 00000100:00080000:1.0:1787827286.048962:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00010000:00010000:2.0:1787827286.048965:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181d1842200/0xbef42c0221d6f5ed lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787827286.048967:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827286.048978:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827286.048979:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043142912:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787827286.048986:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5c94a80 going to sleep for 0 seconds 00000100:00100000:1.0:1787827286.048993:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3800 x1874663043143040/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827286.048996:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043142784:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00080000:1.0:1787827286.049016:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:2.0:1787827286.049019:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181d1842200/0xbef42c0221d6f5ed lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787827286.049020:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827286.049024:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827286.049031:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9352 00000100:00080000:1.0:1787827286.049037:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00010000:00010000:2.0:1787827286.049038:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181d1842200/0xbef42c0221d6f5ed lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787827286.049041:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:0.0:1787827286.049047:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:1.0:1787827286.049050:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:2.0:1787827286.049059:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1842200/0xbef42c0221d6f5ed lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787827286.049066:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1f80 x1874663043143168/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:2.0:1787827286.049077:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181d1842200/0xbef42c0221d6f5ed lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827286.049082:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827286/real 1787827286] req@ffff9181c35b3b80 x1874663043142912/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827311 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:2.0:1787827286.049089:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181d1842200/0xbef42c0221d6f5ed lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827286.049100:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181d1842200 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:2.0:1787827286.049103:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181d1842200/0xbef42c0221d6f5ed lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827286.049106:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3b80 x1874663043142912/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827311 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:2.0:1787827286.049113:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181d1842200/0xbef42c0221d6f5ed lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787827286.049119:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 10000000:00010000:2.0:1787827286.049121:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181d1842200/0xbef42c0221d6f5ed lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787827286.049125:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 10000000:01000000:2.0:1787827286.049130:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00000100:00100000:0.0:1787827286.049133:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043142912:192.168.203.147@tcp:250:kworker.0 00010000:00010000:2.0:1787827286.049135:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787827286.049142:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181d1842200/0xbef42c0221d6f5ed lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827286.049151:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181d1842200/0xbef42c0221d6f5ed lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827286.049154:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043143040:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827286.049173:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 10000000:01000000:2.0:1787827286.049179:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:2.0:1787827286.049183:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00100000:0.0:1787827286.049187:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827286/real 1787827286] req@ffff9181c35b3800 x1874663043143040/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827311 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827286.049200:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3800 x1874663043143040/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827311 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827286.049206:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827286.049209:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827286.049212:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043143040:192.168.203.147@tcp:38:kworker.0 00000100:00080000:1.0:1787827286.049217:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1787827286.049218:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043143168:192.168.203.147@tcp:38:kworker.0 00000100:00080000:1.0:1787827286.049223:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827286.049226:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827286.049231:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9357 00000100:00080000:1.0:1787827286.049235:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000800:00000100:0.0:1787827286.049255:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1787827286.049258:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787827286.049270:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827286/real 1787827286] req@ffff9181c35b1f80 x1874663043143168/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827311 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827286.049277:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0700 x1874663043143296/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827286.049288:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1f80 x1874663043143168/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827311 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827286.049298:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827286.049301:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827286.049305:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043143168:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827286.049316:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043143296:192.168.203.147@tcp:250:kworker.0 00000800:00000100:0.0:1787827286.049329:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827286.049336:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827286/real 1787827286] req@ffff9181c35b0700 x1874663043143296/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827311 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827286.049344:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0700 x1874663043143296/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827311 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827286.049349:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827286.049351:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827286.049361:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043143296:192.168.203.147@tcp:250:kworker.0 00010000:00010000:2.0:1787827286.624859:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827287.648885:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827288.672893:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827289.696943:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827290.720909:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827291.168863:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827291.168874:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827291.168878:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827291.168888:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9357 00000100:00080000:3.0:1787827291.168898:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827291.168905:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827291.168938:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043143424/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827291.168965:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827291.168969:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827291.168971:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827291.168976:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9357 00000100:00080000:3.0:1787827291.168979:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827291.168984:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827291.168994:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827291.169010:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043143552/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827291.169017:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043143424:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787827291.169029:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827291.169034:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827291.169036:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827291.169042:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9357 00000100:00080000:3.0:1787827291.169046:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827291.169051:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827291.169056:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827291.169069:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043143680/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827291.169078:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827291.169114:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827291/real 1787827291] req@ffff9181c5637100 x1874663043143424/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827316 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827291.169127:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043143424/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827316 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827291.169140:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827291.169143:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827291.169156:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043143424:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787827291.169173:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043143552:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827291.169188:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827291.169200:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827291/real 1787827291] req@ffff9181c5637480 x1874663043143552/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827316 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827291.169235:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043143552/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827316 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827291.169241:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827291.169243:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827291.169246:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043143552:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827291.169255:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043143680:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827291.169270:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827291.169277:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827291/real 1787827291] req@ffff9181c5636a00 x1874663043143680/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827316 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827291.169285:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043143680/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827316 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827291.169290:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827291.169292:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827291.169294:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043143680:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827291.744856:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827292.768984:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827293.792898:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827294.816912:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827295.840926:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827296.288849:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827296.288857:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827296.288861:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827296.288880:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9362 00000100:00080000:3.0:1787827296.288890:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827296.288896:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827296.288926:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043143808/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827296.288959:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827296.288962:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827296.288963:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827296.288967:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9362 00000100:00080000:3.0:1787827296.288978:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827296.288985:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827296.288992:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827296.288999:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043143808:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827296.289002:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043143936/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827296.289013:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827296.289015:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827296.289016:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827296.289019:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9362 00000100:00080000:3.0:1787827296.289022:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827296.289024:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827296.289027:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827296.289033:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634e00 x1874663043144064/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827296.289107:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043143936:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827296.289153:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043144064:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787827296.864913:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827297.888917:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827298.912893:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1787827299.360985:0:2378:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:2.0:1787827299.361021:0:2378:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1787827299.361035:0:2378:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:2.0:1787827299.361065:0:2378:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:2.0:1787827299.361072:0:2378:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:0.0:1787827299.361160:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827296/real 1787827299] req@ffff9181c5634380 x1874663043143936/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827321 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827299.361195:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043143936/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827321 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827299.361206:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827299.361211:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827299.361214:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043143936:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827299.361222:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827296/real 1787827299] req@ffff9181c5634e00 x1874663043144064/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827321 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827299.361230:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634e00 x1874663043144064/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827321 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827299.361236:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827299.361238:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827299.361240:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043144064:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827299.361245:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827296/real 1787827299] req@ffff9181c5635500 x1874663043143808/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827321 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827299.361252:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043143808/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827321 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827299.361259:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827299.361261:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827299.361281:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043143808:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787827299.936885:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827300.960927:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827301.408964:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827301.408978:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827301.408983:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827301.408997:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9368 00000100:00080000:2.0:1787827301.409009:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827301.409018:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827301.409065:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f100 x1874663043144192/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827301.409102:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827301.409107:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827301.409109:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827301.409114:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9368 00000100:00080000:2.0:1787827301.409118:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827301.409123:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827301.409147:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827301.409166:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043144192:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787827301.409167:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c700 x1874663043144320/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827301.409185:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827301.409189:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827301.409192:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827301.409197:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9368 00000100:00080000:2.0:1787827301.409201:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827301.409207:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827301.409214:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827301.409228:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f800 x1874663043144448/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827301.409248:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827301.409289:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827301/real 1787827301] req@ffff9181e007f100 x1874663043144192/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827326 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827301.409307:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f100 x1874663043144192/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827326 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827301.409317:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827301.409320:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827301.409336:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043144192:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787827301.409373:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043144320:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827301.409401:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827301.409415:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827301/real 1787827301] req@ffff9181e007c700 x1874663043144320/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827326 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827301.409429:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c700 x1874663043144320/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827326 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827301.409438:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827301.409441:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827301.409445:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043144320:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827301.409451:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043144448:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827301.409472:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827301.409484:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827301/real 1787827301] req@ffff9181e007f800 x1874663043144448/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827326 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827301.409495:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f800 x1874663043144448/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827326 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827301.409502:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827301.409505:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827301.409508:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043144448:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827301.984937:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827303.008883:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827304.032901:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827305.056855:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827306.080883:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827306.528865:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827306.528877:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827306.528881:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 10000000:01000000:2.0:1787827306.528885:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-client' failed (-108). 10000000:00020000:2.0:1787827306.528891:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00080000:3.0:1787827306.528893:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9373 10000000:01000000:2.0:1787827306.528897:0:159735:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:2.0:1787827306.528900:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff9181c5620000 from 7 10000000:01000000:2.0:1787827306.528904:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00000100:00080000:3.0:1787827306.528905:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827306.528914:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827306.528951:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635f80 x1874663043144704/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:2.0:1787827306.528963:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f5f4 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787827306.528972:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:2.0:1787827306.528975:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f5f4 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787827306.528978:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827306.528982:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827306.528988:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9373 00010000:00010000:2.0:1787827306.528991:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f5f4 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787827306.528991:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827306.528996:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827306.529009:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827306.529015:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5c94a80 going to sleep for 0 seconds 00000100:00100000:0.0:1787827306.529021:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043144704:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827306.529025:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043144832/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827306.529027:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043144576:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00080000:3.0:1787827306.529043:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:2.0:1787827306.529046:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f5f4 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787827306.529047:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827306.529051:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827306.529058:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9373 00010000:00010000:2.0:1787827306.529060:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f5f4 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787827306.529062:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827306.529069:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:2.0:1787827306.529071:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f5f4 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000008:00100000:3.0:1787827306.529073:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:2.0:1787827306.529082:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f5f4 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787827306.529087:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634e00 x1874663043144960/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827306.529112:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00010000:00010000:2.0:1787827306.529113:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f5f4 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827306.529125:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181d1843400 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:2.0:1787827306.529130:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f5f4 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827306.529141:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f5f4 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1787827306.529151:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f5f4 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827306.529157:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827306/real 1787827306] req@ffff9181c5635f80 x1874663043144704/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827331 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 10000000:01000000:2.0:1787827306.529158:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:2.0:1787827306.529165:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787827306.529174:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f5f4 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827306.529181:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635f80 x1874663043144704/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827331 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:2.0:1787827306.529183:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f5f4 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787827306.529204:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827306.529209:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827306.529213:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043144704:192.168.203.147@tcp:250:kworker.0 10000000:01000000:2.0:1787827306.529221:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:2.0:1787827306.529224:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00100000:0.0:1787827306.529231:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043144832:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827306.529252:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1787827306.529260:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1787827306.529264:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827306/real 1787827306] req@ffff9181c5635500 x1874663043144832/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827331 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827306.529266:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827306.529269:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827306.529278:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9378 00000100:00080000:3.0:1787827306.529283:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1787827306.529284:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043144832/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827331 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827306.529289:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00080000:0.0:1787827306.529300:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827306.529304:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827306.529308:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043145088/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827306.529311:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043144832:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827306.529323:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043144960:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827306.529341:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827306.529348:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827306/real 1787827306] req@ffff9181c5634e00 x1874663043144960/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827331 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827306.529356:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634e00 x1874663043144960/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827331 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827306.529362:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827306.529364:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827306.529370:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043144960:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827306.529380:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043145088:192.168.203.147@tcp:250:kworker.0 00000800:00000100:0.0:1787827306.529391:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827306.529397:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827306/real 1787827306] req@ffff9181c5634380 x1874663043145088/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827331 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827306.529405:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043145088/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827331 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827306.529415:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827306.529417:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827306.529429:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043145088:192.168.203.147@tcp:250:kworker.0 00010000:00010000:2.0:1787827307.104900:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827308.129010:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827309.152914:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827310.176953:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827311.200919:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827311.648830:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827311.648842:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827311.648846:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827311.648857:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9378 00000100:00080000:3.0:1787827311.648868:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827311.648876:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827311.648913:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043145216/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827311.648943:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827311.648947:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827311.648949:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827311.648954:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9378 00000100:00080000:3.0:1787827311.648959:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827311.648964:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827311.648974:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827311.648976:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043145216:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827311.648990:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043145344/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827311.649004:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827311.649008:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827311.649010:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827311.649015:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9378 00000100:00080000:3.0:1787827311.649021:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827311.649025:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827311.649031:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000800:00000100:0.0:1787827311.649043:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827311.649046:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043145472/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827311.649095:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827311/real 1787827311] req@ffff9181c5636a00 x1874663043145216/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827336 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827311.649111:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043145216/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827336 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827311.649124:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827311.649128:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827311.649138:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043145216:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787827311.649158:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043145344:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827311.649181:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827311.649194:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827311/real 1787827311] req@ffff9181c5637480 x1874663043145344/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827336 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827311.649205:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043145344/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827336 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827311.649214:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827311.649217:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827311.649220:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043145344:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827311.649225:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043145472:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827311.649237:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827311.649243:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827311/real 1787827311] req@ffff9181c5637100 x1874663043145472/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827336 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827311.649251:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043145472/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827336 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827311.649257:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827311.649258:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827311.649260:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043145472:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827312.224931:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827313.248926:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827314.272912:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827315.296899:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827316.320850:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827316.768861:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827316.768872:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827316.768876:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827316.768888:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9383 00000100:00080000:3.0:1787827316.768900:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827316.768907:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827316.768950:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043145600/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827316.768974:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827316.768977:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827316.768978:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827316.768982:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9383 00000100:00080000:3.0:1787827316.768984:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827316.768987:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827316.768996:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827316.769005:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043145728/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827316.769015:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1787827316.769016:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043145600:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787827316.769019:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827316.769023:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827316.769041:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9383 00000100:00080000:3.0:1787827316.769045:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827316.769048:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827316.769053:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827316.769063:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043145856/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827316.769104:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827316.769141:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827316/real 1787827316] req@ffff9181c5637800 x1874663043145600/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827341 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827316.769159:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043145600/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827341 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827316.769167:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827316.769171:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827316.769183:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043145600:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787827316.769206:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043145728:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827316.769223:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827316.769236:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827316/real 1787827316] req@ffff9181c5636d80 x1874663043145728/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827341 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827316.769251:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043145728/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827341 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827316.769262:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827316.769266:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827316.769270:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043145728:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827316.769281:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043145856:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827316.769303:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827316.769312:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827316/real 1787827316] req@ffff9181c5635c00 x1874663043145856/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827341 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827316.769321:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043145856/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827341 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827316.769327:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827316.769330:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827316.769332:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043145856:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787827317.344868:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827318.368941:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827319.392899:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827320.416914:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827321.440870:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827321.888832:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827321.888843:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827321.888848:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827321.888858:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9388 00000100:00080000:3.0:1787827321.888867:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827321.888876:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827321.888912:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043145984/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827321.888939:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827321.888943:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827321.888945:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827321.888950:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9388 00000100:00080000:3.0:1787827321.888954:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827321.888958:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827321.888968:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827321.888972:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043145984:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827321.888984:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634a80 x1874663043146112/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827321.889009:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827321.889014:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827321.889016:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827321.889021:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9388 00000100:00080000:3.0:1787827321.889026:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827321.889031:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827321.889036:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000800:00000100:0.0:1787827321.889049:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827321.889058:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637b80 x1874663043146240/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827321.889092:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827321/real 1787827321] req@ffff9181c5634000 x1874663043145984/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827346 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827321.889107:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043145984/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827346 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827321.889120:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827321.889123:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827321.889133:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043145984:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787827321.889152:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043146112:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827321.889170:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827321.889179:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827321/real 1787827321] req@ffff9181c5634a80 x1874663043146112/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827346 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827321.889190:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634a80 x1874663043146112/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827346 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827321.889196:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827321.889198:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827321.889201:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043146112:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827321.889208:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043146240:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827321.889221:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827321.889228:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827321/real 1787827321] req@ffff9181c5637b80 x1874663043146240/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827346 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827321.889235:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637b80 x1874663043146240/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827346 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827321.889241:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827321.889242:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827321.889244:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043146240:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827322.464894:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827323.488887:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827324.512897:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827325.536897:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827326.560918:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827327.008832:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827327.008841:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 10000000:01000000:2.0:1787827327.008842:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'params' failed (-108). 00000100:00080000:1.0:1787827327.008845:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 10000000:00020000:2.0:1787827327.008849:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00080000:1.0:1787827327.008855:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9393 10000000:01000000:2.0:1787827327.008855:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:2.0:1787827327.008858:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff9181c87b19b8 from 1 10000000:01000000:2.0:1787827327.008862:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00000100:00080000:1.0:1787827327.008867:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827327.008874:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787827327.008898:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2d80 x1874663043146496/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:2.0:1787827327.008910:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f5fb lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787827327.008920:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827327.008925:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827327.008927:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00010000:00010000:2.0:1787827327.008927:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f5fb lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787827327.008935:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9393 00000100:00080000:1.0:1787827327.008940:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827327.008943:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:2.0:1787827327.008944:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f5fb lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000008:00100000:1.0:1787827327.008951:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827327.008973:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5c94a80 going to sleep for 0 seconds 00000100:00100000:1.0:1787827327.008979:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0a80 x1874663043146624/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827327.008983:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043146496:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787827327.008983:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043146368:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:2.0:1787827327.009001:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f5fb lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787827327.009006:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827327.009012:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827327.009015:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00010000:00010000:2.0:1787827327.009020:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f5fb lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787827327.009021:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9393 00010000:00010000:2.0:1787827327.009033:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f5fb lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787827327.009040:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00010000:00010000:2.0:1787827327.009043:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f5fb lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787827327.009047:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:0.0:1787827327.009052:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:1.0:1787827327.009058:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:2.0:1787827327.009063:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f5fb lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787827327.009072:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2a00 x1874663043146752/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:2.0:1787827327.009075:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181d1843400 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:2.0:1787827327.009084:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f5fb lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827327.009089:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827327/real 1787827327] req@ffff9181c35b2d80 x1874663043146496/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827352 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:2.0:1787827327.009101:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f5fb lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827327.009110:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2d80 x1874663043146496/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827352 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 10000000:00010000:2.0:1787827327.009114:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f5fb lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787827327.009120:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827327.009126:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 10000000:01000000:2.0:1787827327.009129:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00000100:00100000:0.0:1787827327.009131:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043146496:192.168.203.147@tcp:250:kworker.0 00010000:00010000:2.0:1787827327.009134:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787827327.009142:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f5fb lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827327.009148:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f5fb lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827327.009153:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043146624:192.168.203.147@tcp:38:kworker.0 10000000:01000000:2.0:1787827327.009164:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:2.0:1787827327.009168:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000800:00000100:0.0:1787827327.009181:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827327.009195:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827327/real 1787827327] req@ffff9181c35b0a80 x1874663043146624/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827352 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827327.009202:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1787827327.009207:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0a80 x1874663043146624/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827352 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827327.009208:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827327.009211:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827327.009219:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9398 00000100:00080000:0.0:1787827327.009223:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827327.009225:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827327.009230:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00080000:1.0:1787827327.009233:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787827327.009238:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043146624:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827327.009244:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043146752:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827327.009249:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3480 x1874663043146880/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827327.009262:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827327.009274:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827327/real 1787827327] req@ffff9181c35b2a00 x1874663043146752/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827352 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827327.009283:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2a00 x1874663043146752/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827352 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827327.009288:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827327.009290:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827327.009292:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043146752:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827327.009301:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043146880:192.168.203.147@tcp:250:kworker.0 00000800:00000100:0.0:1787827327.009311:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827327.009316:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827327/real 1787827327] req@ffff9181c35b3480 x1874663043146880/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827352 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827327.009324:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3480 x1874663043146880/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827352 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827327.009329:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827327.009331:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827327.009341:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043146880:192.168.203.147@tcp:250:kworker.0 00010000:00010000:2.0:1787827327.584864:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827328.608906:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827329.632893:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827330.656950:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827331.680895:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827332.128955:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827332.128966:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827332.128970:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827332.128982:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9398 00000100:00080000:3.0:1787827332.128995:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827332.129013:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827332.129071:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637b80 x1874663043147008/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827332.129099:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827332.129102:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827332.129104:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827332.129107:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9398 00000100:00080000:3.0:1787827332.129110:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827332.129114:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827332.129124:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827332.129136:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634a80 x1874663043147136/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827332.129145:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827332.129147:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827332.129150:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827332.129154:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9398 00000100:00080000:3.0:1787827332.129157:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827332.129161:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787827332.129169:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043147008:192.168.203.147@tcp:250:kworker.0 00000008:00100000:3.0:1787827332.129169:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827332.129187:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043147264/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827332.129244:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827332.129277:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827332/real 1787827332] req@ffff9181c5637b80 x1874663043147008/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827357 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827332.129293:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637b80 x1874663043147008/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827357 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827332.129306:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827332.129309:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827332.129329:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043147008:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787827332.129342:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043147136:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827332.129372:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827332.129386:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827332/real 1787827332] req@ffff9181c5634a80 x1874663043147136/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827357 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827332.129399:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634a80 x1874663043147136/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827357 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827332.129405:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827332.129407:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827332.129409:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043147136:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827332.129430:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043147264:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827332.129445:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827332.129452:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827332/real 1787827332] req@ffff9181c5634000 x1874663043147264/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827357 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827332.129460:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043147264/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827357 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827332.129465:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827332.129468:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827332.129470:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043147264:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827332.704911:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827333.728927:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827334.752893:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827335.776905:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827336.800909:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827337.248843:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827337.248855:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827337.248858:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827337.248867:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9403 00000100:00080000:3.0:1787827337.248876:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827337.248892:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827337.248925:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043147392/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827337.248949:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827337.248952:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827337.248953:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827337.248956:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9403 00000100:00080000:3.0:1787827337.248959:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827337.248961:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827337.248969:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827337.248978:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043147520/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827337.248987:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827337.248989:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827337.248991:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1787827337.248994:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043147392:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787827337.248996:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9403 00000100:00080000:3.0:1787827337.248999:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827337.249003:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827337.249008:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827337.249015:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043147648/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827337.249069:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827337.249104:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827337/real 1787827337] req@ffff9181c5635c00 x1874663043147392/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827362 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827337.249119:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043147392/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827362 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827337.249132:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827337.249135:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827337.249146:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043147392:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787827337.249167:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043147520:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827337.249187:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827337.249200:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827337/real 1787827337] req@ffff9181c5636d80 x1874663043147520/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827362 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827337.249214:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043147520/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827362 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827337.249223:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827337.249226:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827337.249230:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043147520:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827337.249236:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043147648:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827337.249254:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827337.249264:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827337/real 1787827337] req@ffff9181c5637800 x1874663043147648/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827362 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827337.249277:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043147648/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827362 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827337.249282:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827337.249284:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827337.249286:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043147648:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787827337.824914:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827338.848946:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827339.872902:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827340.896873:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827341.921003:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827342.368827:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827342.368835:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827342.368838:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827342.368846:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9408 00000100:00080000:3.0:1787827342.368870:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827342.368877:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827342.368921:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043147776/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827342.368947:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827342.368949:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827342.368951:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827342.368954:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9408 00000100:00080000:3.0:1787827342.368957:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827342.368960:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827342.368966:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827342.368974:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043147904/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827342.368982:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827342.368984:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827342.368986:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827342.368988:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9408 00000100:00080000:3.0:1787827342.368990:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827342.368993:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827342.368996:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827342.369007:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043147776:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827342.369007:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043148032/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827342.369068:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827342.369099:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827342/real 1787827342] req@ffff9181c5637100 x1874663043147776/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827367 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827342.369113:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043147776/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827367 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827342.369121:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827342.369124:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827342.369138:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043147776:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787827342.369143:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043147904:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827342.369158:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827342.369168:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827342/real 1787827342] req@ffff9181c5637480 x1874663043147904/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827367 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827342.369176:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043147904/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827367 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827342.369187:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827342.369190:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827342.369193:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043147904:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827342.369210:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043148032:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827342.369221:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827342.369238:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827342/real 1787827342] req@ffff9181c5636a00 x1874663043148032/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827367 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827342.369245:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043148032/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827367 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827342.369251:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827342.369253:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827342.369255:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043148032:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827342.944861:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827343.968919:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827344.992868:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827346.016909:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827347.040859:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827347.488850:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 10000000:01000000:2.0:1787827347.488855:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-sptlrpc' failed (-108). 00000100:00080000:0.0:1787827347.488859:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 10000000:00020000:2.0:1787827347.488861:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00080000:0.0:1787827347.488864:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827347.488873:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9414 00000100:00080000:0.0:1787827347.488882:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827347.488888:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787827347.488917:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5c00 x1874663043148160/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827347.488940:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827347.488944:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827347.488945:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827347.488948:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9414 00000100:00080000:0.0:1787827347.488951:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827347.488953:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787827347.488960:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827347.488969:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5f80 x1874663043148288/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827347.488977:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827347.488979:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827347.488981:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827347.488983:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9414 00000100:00080000:0.0:1787827347.488985:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827347.488988:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787827347.488991:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827347.488994:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043148160:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787827347.488999:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4700 x1874663043148416/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827347.489053:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827347.489080:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827347/real 1787827347] req@ffff9181c55a5c00 x1874663043148160/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827372 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827347.489093:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5c00 x1874663043148160/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827372 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827347.489101:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827347.489104:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827347.489107:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043148160:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827347.489112:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043148288:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827347.489126:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827347.489134:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827347/real 1787827347] req@ffff9181c55a5f80 x1874663043148288/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827372 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827347.489142:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5f80 x1874663043148288/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827372 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827347.489148:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827347.489150:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827347.489153:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043148288:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827347.489175:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043148416:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827347.489196:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827347.489208:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827347/real 1787827347] req@ffff9181c55a4700 x1874663043148416/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827372 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827347.489216:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4700 x1874663043148416/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827372 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827347.489222:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827347.489224:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827347.489226:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043148416:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787827348.064888:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827349.088900:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:2.0:1787827349.344845:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:2.0:1787827349.344852:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff9181c5620000 from 341 10000000:01000000:2.0:1787827349.344856:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:2.0:1787827349.344914:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f602 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827349.344924:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f602 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827349.344933:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f602 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827349.344950:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5c94a80 going to sleep for 0 seconds 00000100:00100000:2.0:1787827349.344959:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043148544:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:2.0:1787827349.344971:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f602 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827349.344977:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f602 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827349.344982:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f602 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827349.344995:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f602 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827349.345000:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f602 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827349.345004:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181d1843400 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:2.0:1787827349.345007:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f602 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827349.345012:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f602 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1787827349.345018:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f602 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787827349.345022:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:2.0:1787827349.345024:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787827349.345029:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f602 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827349.345033:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f602 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787827349.345048:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:2.0:1787827349.345050:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:1.0:1787827349.345164:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827349.345169:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827349.345172:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827349.345177:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9419 00000100:00080000:1.0:1787827349.345185:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827349.345190:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787827349.345218:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0e00 x1874663043148672/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827349.345241:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827349.345244:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827349.345245:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827349.345248:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9419 00000100:00080000:1.0:1787827349.345250:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827349.345253:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827349.345261:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827349.345269:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2680 x1874663043148800/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827349.345286:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827349.345288:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827349.345290:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827349.345292:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9419 00000100:00080000:1.0:1787827349.345294:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1787827349.345299:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043148672:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787827349.345299:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827349.345306:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827349.345314:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1180 x1874663043148928/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827349.345387:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827349.345423:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827349/real 1787827349] req@ffff9181c35b0e00 x1874663043148672/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827374 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827349.345453:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0e00 x1874663043148672/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827374 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827349.345470:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827349.345475:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827349.345487:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043148672:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787827349.345495:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043148800:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827349.345532:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827349.345546:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827349/real 1787827349] req@ffff9181c35b2680 x1874663043148800/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827374 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827349.345568:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2680 x1874663043148800/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827374 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827349.345577:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827349.345581:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827349.345585:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043148800:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827349.345605:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043148928:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827349.345625:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827349.345644:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827349/real 1787827349] req@ffff9181c35b1180 x1874663043148928/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827374 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827349.345657:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1180 x1874663043148928/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827374 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827349.345666:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827349.345669:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827349.345672:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043148928:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787827350.112868:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827351.136916:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827352.160868:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827353.184933:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827354.208918:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827354.656869:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827354.656880:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827354.656884:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827354.656894:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9421 00000100:00080000:1.0:1787827354.656904:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827354.656912:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787827354.656949:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0000 x1874663043149056/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827354.656988:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827354.656992:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827354.656994:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827354.656999:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9421 00000100:00080000:1.0:1787827354.657003:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827354.657007:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827354.657017:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827354.657034:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1c00 x1874663043149184/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827354.657043:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043149056:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787827354.657051:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827354.657056:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827354.657059:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827354.657065:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9421 00000100:00080000:1.0:1787827354.657077:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827354.657082:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827354.657088:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827354.657101:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0380 x1874663043149312/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827354.657114:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827354.657146:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827354/real 1787827354] req@ffff9181c35b0000 x1874663043149056/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827379 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827354.657160:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0000 x1874663043149056/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827379 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827354.657168:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827354.657171:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827354.657184:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043149056:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787827354.657202:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043149184:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827354.657226:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827354.657241:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827354/real 1787827354] req@ffff9181c35b1c00 x1874663043149184/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827379 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827354.657254:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1c00 x1874663043149184/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827379 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827354.657264:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827354.657267:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827354.657271:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043149184:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827354.657277:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043149312:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827354.657292:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827354.657300:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827354/real 1787827354] req@ffff9181c35b0380 x1874663043149312/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827379 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827354.657309:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0380 x1874663043149312/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827379 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827354.657314:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827354.657316:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827354.657318:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043149312:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827355.232915:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827356.256887:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827357.280882:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827358.304876:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827359.328872:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827359.776839:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827359.776846:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827359.776849:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827359.776858:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9426 00000100:00080000:2.0:1787827359.776867:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827359.776872:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827359.776904:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d880 x1874663043149440/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827359.776924:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827359.776930:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827359.776932:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827359.776937:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9426 00000100:00080000:2.0:1787827359.776941:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827359.776946:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827359.776955:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827359.776975:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043149440:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787827359.776982:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f480 x1874663043149568/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827359.776995:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827359.776997:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827359.776998:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827359.777002:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9426 00000100:00080000:2.0:1787827359.777005:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827359.777008:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827359.777012:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827359.777021:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ce00 x1874663043149696/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827359.777069:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043149568:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827359.777104:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043149696:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787827360.352918:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827361.376902:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827362.400856:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1787827362.848991:0:2379:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:1.0:1787827362.849059:0:2379:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787827362.849076:0:2379:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1787827362.849111:0:2379:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1787827362.849119:0:2379:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:0.0:1787827362.849172:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827359/real 1787827362] req@ffff9181e007f480 x1874663043149568/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827384 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827362.849202:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f480 x1874663043149568/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827384 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827362.849215:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827362.849220:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827362.849225:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043149568:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827362.849234:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827359/real 1787827362] req@ffff9181e007ce00 x1874663043149696/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827384 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827362.849247:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ce00 x1874663043149696/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827384 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827362.849262:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827362.849266:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827362.849269:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043149696:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827362.849276:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827359/real 1787827362] req@ffff9181e007d880 x1874663043149440/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827384 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827362.849288:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d880 x1874663043149440/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827384 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827362.849295:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827362.849299:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827362.849318:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043149440:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787827363.424876:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827364.448856:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827364.896813:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827364.896821:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827364.896824:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827364.896834:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9431 00000100:00080000:2.0:1787827364.896845:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827364.896851:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827364.896880:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007fb80 x1874663043149824/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827364.896903:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827364.896917:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827364.896918:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827364.896921:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9431 00000100:00080000:2.0:1787827364.896923:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827364.896926:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827364.896933:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827364.896942:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ca80 x1874663043149952/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827364.896949:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827364.896952:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827364.896954:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827364.896956:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9431 00000100:00080000:2.0:1787827364.896958:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827364.896962:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787827364.896967:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043149824:192.168.203.147@tcp:250:kworker.0 00000008:00100000:2.0:1787827364.896967:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827364.896976:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e680 x1874663043150080/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827364.897037:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827364.897075:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827364/real 1787827364] req@ffff9181e007fb80 x1874663043149824/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827389 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827364.897095:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007fb80 x1874663043149824/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827389 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827364.897106:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827364.897109:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827364.897133:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043149824:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787827364.897142:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043149952:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827364.897173:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827364.897188:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827364/real 1787827364] req@ffff9181e007ca80 x1874663043149952/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827389 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827364.897202:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ca80 x1874663043149952/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827389 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827364.897211:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827364.897214:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827364.897218:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043149952:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827364.897254:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043150080:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827364.897275:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827364.897288:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827364/real 1787827364] req@ffff9181e007e680 x1874663043150080/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827389 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827364.897301:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e680 x1874663043150080/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827389 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827364.897310:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827364.897313:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827364.897316:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043150080:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827365.472920:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827366.496872:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827367.520937:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827368.544846:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:2.0:1787827369.504923:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-cliir' failed (-108). 10000000:00000400:2.0:1787827369.504934:0:159735:0:(mgc_request.c:1900:mgc_process_log()) MGC192.168.203.147@tcp: IR log lustre-cliir failed, not fatal: rc = -108 10000000:01000000:2.0:1787827369.504945:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:2.0:1787827369.504948:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff9181c5620000 from 52 10000000:01000000:2.0:1787827369.504953:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:2.0:1787827369.505033:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f609 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827369.505050:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f609 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827369.505067:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f609 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827369.505094:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5c94a80 going to sleep for 0 seconds 00000100:00100000:2.0:1787827369.505119:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043150208:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:2.0:1787827369.505135:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f609 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827369.505145:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f609 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827369.505154:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f609 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827369.505163:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f609 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827369.505173:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f609 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827369.505184:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181d1843400 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:2.0:1787827369.505188:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f609 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827369.505198:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f609 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1787827369.505207:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f609 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787827369.505214:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:2.0:1787827369.505230:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787827369.505248:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f609 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827369.505257:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f609 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787827369.505282:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:2.0:1787827369.505285:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:1.0:1787827369.505391:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827369.505401:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827369.505405:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827369.505415:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9436 00000100:00080000:1.0:1787827369.505428:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827369.505436:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787827369.505476:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1500 x1874663043150336/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827369.505509:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827369.505514:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827369.505516:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827369.505522:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9436 00000100:00080000:1.0:1787827369.505526:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827369.505531:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827369.505542:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827369.505570:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3100 x1874663043150464/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827369.505587:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043150336:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787827369.505591:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827369.505602:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827369.505609:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827369.505616:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9436 00000100:00080000:1.0:1787827369.505619:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827369.505624:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827369.505630:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827369.505646:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0380 x1874663043150592/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827369.505735:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827369.505835:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827369/real 1787827369] req@ffff9181c35b1500 x1874663043150336/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827394 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827369.505891:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1500 x1874663043150336/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827394 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827369.505912:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827369.505918:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827369.505939:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043150336:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787827369.505970:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043150464:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827369.506022:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827369.506049:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827369/real 1787827369] req@ffff9181c35b3100 x1874663043150464/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827394 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827369.506096:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3100 x1874663043150464/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827394 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827369.506124:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827369.506129:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827369.506134:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043150464:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827369.506142:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043150592:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827369.506174:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827369.506192:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827369/real 1787827369] req@ffff9181c35b0380 x1874663043150592/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827394 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827369.506233:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0380 x1874663043150592/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827394 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827369.506252:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827369.506256:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827369.506260:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043150592:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827369.568927:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827370.593015:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827371.616884:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827372.640968:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827373.664962:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827374.624869:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827374.624878:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827374.624883:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827374.624892:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9441 00000100:00080000:1.0:1787827374.624902:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827374.624909:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787827374.624944:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1c00 x1874663043150720/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827374.624965:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827374.624968:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827374.624969:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827374.624973:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9441 00000100:00080000:1.0:1787827374.624975:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827374.624978:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827374.624985:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827374.624995:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0000 x1874663043150848/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827374.625004:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827374.625007:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827374.625010:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1787827374.625013:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043150720:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787827374.625015:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9441 00000100:00080000:1.0:1787827374.625031:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827374.625038:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827374.625045:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827374.625061:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1180 x1874663043150976/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827374.625085:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827374.625120:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827374/real 1787827374] req@ffff9181c35b1c00 x1874663043150720/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827399 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827374.625136:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1c00 x1874663043150720/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827399 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827374.625145:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827374.625148:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827374.625161:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043150720:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787827374.625182:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043150848:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827374.625209:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827374.625220:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827374/real 1787827374] req@ffff9181c35b0000 x1874663043150848/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827399 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827374.625230:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0000 x1874663043150848/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827399 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827374.625237:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827374.625240:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827374.625242:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043150848:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827374.625248:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043150976:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827374.625260:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827374.625267:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827374/real 1787827374] req@ffff9181c35b1180 x1874663043150976/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827399 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827374.625276:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1180 x1874663043150976/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827399 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827374.625282:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827374.625284:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827374.625286:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043150976:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787827374.688884:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827375.712858:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827376.736890:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827377.761009:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827378.784862:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827379.744890:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827379.744901:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827379.744905:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827379.744915:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9446 00000100:00080000:2.0:1787827379.744928:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827379.744935:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827379.744974:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007dc00 x1874663043151104/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827379.745022:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827379.745026:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827379.745029:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827379.745033:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9446 00000100:00080000:2.0:1787827379.745037:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827379.745042:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827379.745051:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827379.745064:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043151104:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787827379.745070:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c000 x1874663043151232/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827379.745091:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827379.745095:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827379.745097:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827379.745102:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9446 00000100:00080000:2.0:1787827379.745106:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827379.745111:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827379.745116:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827379.745130:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c380 x1874663043151360/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827379.745142:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827379.745177:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827379/real 1787827379] req@ffff9181e007dc00 x1874663043151104/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827404 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827379.745191:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007dc00 x1874663043151104/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827404 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827379.745200:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827379.745204:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827379.745219:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043151104:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787827379.745248:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043151232:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827379.745275:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827379.745290:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827379/real 1787827379] req@ffff9181e007c000 x1874663043151232/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827404 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827379.745314:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c000 x1874663043151232/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827404 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827379.745324:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827379.745327:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827379.745331:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043151232:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827379.745337:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043151360:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827379.745355:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827379.745363:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827379/real 1787827379] req@ffff9181e007c380 x1874663043151360/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827404 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827379.745371:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c380 x1874663043151360/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827404 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827379.745387:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827379.745390:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827379.745392:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043151360:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827379.808865:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827380.832996:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827381.856897:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827382.880910:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827383.904896:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827384.864941:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827384.864954:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827384.864960:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827384.864987:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9451 00000100:00080000:0.0:1787827384.865002:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827384.865011:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787827384.865060:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4380 x1874663043151488/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827384.865109:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827384.865114:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827384.865117:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827384.865122:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9451 00000100:00080000:0.0:1787827384.865127:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827384.865135:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787827384.865151:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827384.865179:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5180 x1874663043151616/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827384.865202:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043151488:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787827384.865206:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827384.865219:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827384.865223:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827384.865234:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9451 00000100:00080000:0.0:1787827384.865242:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827384.865251:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787827384.865260:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827384.865280:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5500 x1874663043151744/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827384.865299:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827384.865352:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827384/real 1787827384] req@ffff9181c55a4380 x1874663043151488/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827409 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827384.865376:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4380 x1874663043151488/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827409 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827384.865389:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827384.865408:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827384.865429:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043151488:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827384.865459:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043151616:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827384.865494:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827384.865528:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827384/real 1787827384] req@ffff9181c55a5180 x1874663043151616/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827409 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827384.865546:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5180 x1874663043151616/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827409 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827384.865585:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827384.865590:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827384.865596:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043151616:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827384.865605:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043151744:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827384.865634:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827384.865649:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827384/real 1787827384] req@ffff9181c55a5500 x1874663043151744/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827409 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827384.865664:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5500 x1874663043151744/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827409 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827384.865675:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827384.865678:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827384.865683:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043151744:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787827384.928967:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827385.952955:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827386.976885:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827388.000949:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827389.024885:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:2.0:1787827389.984900:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-client' failed (-108). 10000000:00020000:2.0:1787827389.984908:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:2.0:1787827389.984913:0:159735:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:2.0:1787827389.984915:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff9181c5620000 from 7 10000000:01000000:2.0:1787827389.984919:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00010000:00010000:2.0:1787827389.984964:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f610 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827389.984976:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f610 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787827389.984983:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:2.0:1787827389.984987:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f610 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787827389.984990:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827389.985012:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827389.985019:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9456 00000100:00080000:0.0:1787827389.985029:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827389.985036:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827389.985043:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5c94a80 going to sleep for 0 seconds 00000100:00100000:2.0:1787827389.985052:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c380 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043151872:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00100000:0.0:1787827389.985081:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4e00 x1874663043152000/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:2.0:1787827389.985085:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f610 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827389.985106:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f610 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827389.985113:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f610 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787827389.985116:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827389.985123:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00010000:00010000:2.0:1787827389.985124:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f610 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787827389.985125:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827389.985135:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9456 00000100:00080000:0.0:1787827389.985142:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00010000:00010000:2.0:1787827389.985148:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f610 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787827389.985150:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:2.0:1787827389.985160:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181d1843400 already unused, calling callback (ffffffffc1273c20) 00000008:00100000:0.0:1787827389.985163:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 10000000:00010000:2.0:1787827389.985164:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f610 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827389.985175:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f610 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827389.985181:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7800 x1874663043152128/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 10000000:00010000:2.0:1787827389.985185:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f610 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787827389.985192:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:2.0:1787827389.985196:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787827389.985214:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f610 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787827389.985222:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:2.0:1787827389.985223:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f610 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787827389.985230:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043152000:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787827389.985235:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827389.985242:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827389.985253:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9456 00000100:00080000:0.0:1787827389.985259:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827389.985266:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 10000000:01000000:2.0:1787827389.985267:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:2.0:1787827389.985274:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:CONNECTING 00000008:00100000:0.0:1787827389.985276:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827389.985292:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5500 x1874663043152256/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827389.985301:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827389.985355:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827389/real 1787827389] req@ffff9181c55a4e00 x1874663043152000/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827414 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827389.985373:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4e00 x1874663043152000/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827414 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827389.985396:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827389.985407:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00080000:0.0:1787827389.985417:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1787827389.985422:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043152000:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787827389.985424:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827389.985430:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827389.985444:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9461 00000100:00080000:0.0:1787827389.985449:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827389.985453:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787827389.985468:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5180 x1874663043152384/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827389.985475:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043152128:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827389.985507:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827389.985540:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827389/real 1787827389] req@ffff9181c55a7800 x1874663043152128/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827414 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827389.985552:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7800 x1874663043152128/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827414 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827389.985559:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827389.985562:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827389.985564:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043152128:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827389.985582:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043152256:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827389.985619:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827389.985629:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827389/real 1787827389] req@ffff9181c55a5500 x1874663043152256/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827414 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827389.985641:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5500 x1874663043152256/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827414 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827389.985648:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827389.985666:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827389.985669:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043152256:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827389.985687:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043152384:192.168.203.147@tcp:250:kworker.0 00000800:00000100:1.0:1787827389.985704:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827389.985715:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827389/real 1787827389] req@ffff9181c55a5180 x1874663043152384/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827414 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827389.985736:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5180 x1874663043152384/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827414 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827389.985743:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827389.985761:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827389.985806:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043152384:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787827390.048888:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827391.073057:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827392.096940:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827393.120921:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827394.145024:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827395.104925:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827395.104944:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827395.104952:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827395.104971:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9461 00000100:00080000:2.0:1787827395.104988:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827395.104999:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827395.105060:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c380 x1874663043152512/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827395.105099:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827395.105104:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827395.105124:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827395.105142:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9461 00000100:00080000:2.0:1787827395.105148:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827395.105156:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827395.105171:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827395.105188:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043152512:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787827395.105195:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c000 x1874663043152640/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827395.105221:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827395.105228:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827395.105232:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827395.105241:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9461 00000100:00080000:2.0:1787827395.105250:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827395.105260:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827395.105271:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827395.105297:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007dc00 x1874663043152768/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827395.105300:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827395.105381:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827395/real 1787827395] req@ffff9181e007c380 x1874663043152512/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827420 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827395.105407:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c380 x1874663043152512/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827420 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827395.105430:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827395.105436:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827395.105458:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043152512:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827395.105552:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043152640:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827395.105603:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827395.105645:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827395/real 1787827395] req@ffff9181e007c000 x1874663043152640/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827420 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827395.105669:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c000 x1874663043152640/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827420 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827395.105680:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827395.105685:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827395.105707:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043152640:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827395.105715:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043152768:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827395.105744:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827395.105758:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827395/real 1787827395] req@ffff9181e007dc00 x1874663043152768/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827420 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827395.105773:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007dc00 x1874663043152768/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827420 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827395.105784:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827395.105787:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827395.105823:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043152768:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787827395.169015:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827396.192852:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827397.216900:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827398.240886:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827399.264967:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827400.224837:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827400.224849:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827400.224854:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827400.224866:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9466 00000100:00080000:2.0:1787827400.224880:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827400.224889:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827400.224933:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e680 x1874663043152896/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827400.224969:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827400.224976:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827400.224979:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827400.224986:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9466 00000100:00080000:2.0:1787827400.224992:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827400.225002:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827400.225016:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827400.225039:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043152896:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787827400.225043:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ca80 x1874663043153024/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827400.225081:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827400.225086:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827400.225089:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827400.225095:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9466 00000100:00080000:2.0:1787827400.225102:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827400.225109:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827400.225119:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827400.225156:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007fb80 x1874663043153152/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827400.225171:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827400.225221:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827400/real 1787827400] req@ffff9181e007e680 x1874663043152896/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827425 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827400.225243:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e680 x1874663043152896/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827425 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827400.225257:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827400.225262:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827400.225298:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043152896:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827400.225327:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043153024:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827400.225379:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827400.225399:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827400/real 1787827400] req@ffff9181e007ca80 x1874663043153024/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827425 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827400.225414:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ca80 x1874663043153024/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827425 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827400.225424:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827400.225428:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827400.225432:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043153024:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827400.225439:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043153152:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827400.225491:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827400.225505:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827400/real 1787827400] req@ffff9181e007fb80 x1874663043153152/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827425 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827400.225519:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007fb80 x1874663043153152/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827425 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827400.225529:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827400.225533:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827400.225537:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043153152:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827400.288954:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827401.312889:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827402.336878:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827403.361015:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827404.385060:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827405.344892:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827405.344904:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827405.344910:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827405.344922:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9471 00000100:00080000:0.0:1787827405.344935:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827405.344943:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787827405.344991:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4380 x1874663043153280/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827405.345015:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827405.345018:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827405.345020:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827405.345023:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9471 00000100:00080000:0.0:1787827405.345025:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827405.345028:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787827405.345037:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827405.345046:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4700 x1874663043153408/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827405.345060:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827405.345064:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:1.0:1787827405.345064:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043153280:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787827405.345068:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827405.345074:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9471 00000100:00080000:0.0:1787827405.345077:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827405.345079:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787827405.345083:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827405.345091:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5f80 x1874663043153536/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827405.345148:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827405.345191:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827405/real 1787827405] req@ffff9181c55a4380 x1874663043153280/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827430 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827405.345207:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4380 x1874663043153280/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827430 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827405.345217:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827405.345220:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827405.345233:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043153280:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827405.345255:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043153408:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827405.345272:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827405.345281:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827405/real 1787827405] req@ffff9181c55a4700 x1874663043153408/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827430 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827405.345290:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4700 x1874663043153408/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827430 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827405.345297:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827405.345301:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827405.345303:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043153408:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827405.345308:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043153536:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827405.345327:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827405.345338:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827405/real 1787827405] req@ffff9181c55a5f80 x1874663043153536/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827430 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827405.345346:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5f80 x1874663043153536/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827430 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827405.345352:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827405.345354:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827405.345356:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043153536:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787827405.408857:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827406.432974:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827407.456991:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827408.480911:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827409.504978:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827410.464918:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827410.464933:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827410.464939:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 10000000:01000000:2.0:1787827410.464944:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'params' failed (-108). 10000000:00020000:2.0:1787827410.464954:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00080000:0.0:1787827410.464956:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9477 10000000:01000000:2.0:1787827410.464961:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:2.0:1787827410.464966:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff9181c87b19b8 from 1 10000000:01000000:2.0:1787827410.464989:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00000100:00080000:0.0:1787827410.464991:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827410.465006:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787827410.465043:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5c00 x1874663043153792/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:2.0:1787827410.465058:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1842e00/0xbef42c0221d6f617 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827410.465074:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181d1842e00/0xbef42c0221d6f617 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787827410.465079:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827410.465089:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827410.465091:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00010000:00010000:2.0:1787827410.465094:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181d1842e00/0xbef42c0221d6f617 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787827410.465098:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9477 00000100:00080000:0.0:1787827410.465108:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827410.465117:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827410.465127:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5c94a80 going to sleep for 0 seconds 00000008:00100000:0.0:1787827410.465133:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827410.465143:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043153664:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00100000:0.0:1787827410.465154:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5880 x1874663043153920/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:2.0:1787827410.465162:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181d1842e00/0xbef42c0221d6f617 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787827410.465186:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:2.0:1787827410.465209:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181d1842e00/0xbef42c0221d6f617 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787827410.465210:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827410.465218:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827410.465233:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9477 00010000:00010000:2.0:1787827410.465243:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1842e00/0xbef42c0221d6f617 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787827410.465246:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00010000:00010000:2.0:1787827410.465265:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181d1842e00/0xbef42c0221d6f617 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787827410.465266:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043153792:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787827410.465268:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787827410.465296:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:2.0:1787827410.465330:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181d1842e00/0xbef42c0221d6f617 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827410.465339:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181d1842e00 already unused, calling callback (ffffffffc1273c20) 00000100:00100000:0.0:1787827410.465341:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6d80 x1874663043154048/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 10000000:00010000:2.0:1787827410.465348:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181d1842e00/0xbef42c0221d6f617 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827410.465403:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181d1842e00/0xbef42c0221d6f617 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1787827410.465431:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181d1842e00/0xbef42c0221d6f617 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787827410.465438:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:2.0:1787827410.465443:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787827410.465457:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181d1842e00/0xbef42c0221d6f617 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827410.465466:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181d1842e00/0xbef42c0221d6f617 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787827410.465494:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:2.0:1787827410.465502:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:CONNECTING 00000800:00000100:1.0:1787827410.465505:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827410.465578:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827410/real 1787827410] req@ffff9181c55a5c00 x1874663043153792/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827435 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827410.465613:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5c00 x1874663043153792/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827435 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827410.465645:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827410.465687:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827410.465706:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043153792:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827410.465724:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043153920:192.168.203.147@tcp:38:kworker.0 00000100:00080000:0.0:1787827410.465794:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827410.465803:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827410.465807:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827410.465820:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9482 00000100:00080000:0.0:1787827410.465828:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827410.465850:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:1.0:1787827410.465864:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827410.465892:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7480 x1874663043154176/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827410.465896:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827410/real 1787827410] req@ffff9181c55a5880 x1874663043153920/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827435 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827410.465963:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5880 x1874663043153920/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827435 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827410.465990:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827410.465997:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827410.466002:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043153920:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827410.466038:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043154048:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827410.466072:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827410.466091:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827410/real 1787827410] req@ffff9181c55a6d80 x1874663043154048/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827435 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827410.466109:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6d80 x1874663043154048/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827435 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827410.466121:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827410.466135:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827410.466140:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043154048:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827410.466149:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043154176:192.168.203.147@tcp:250:kworker.0 00000800:00000100:1.0:1787827410.466172:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827410.466222:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827410/real 1787827410] req@ffff9181c55a7480 x1874663043154176/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827435 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827410.466241:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7480 x1874663043154176/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827435 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827410.466276:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827410.466281:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827410.466321:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043154176:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787827410.529076:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827411.552998:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827412.576993:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827413.600973:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827414.624992:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827415.584945:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827415.584955:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827415.584960:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827415.584971:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9482 00000100:00080000:2.0:1787827415.584984:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827415.584992:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827415.585033:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d880 x1874663043154304/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827415.585066:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827415.585071:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827415.585073:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827415.585077:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9482 00000100:00080000:2.0:1787827415.585080:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827415.585084:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827415.585094:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827415.585106:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ce00 x1874663043154432/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827415.585116:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827415.585119:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827415.585121:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827415.585124:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9482 00000100:00080000:2.0:1787827415.585127:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827415.585130:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827415.585136:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827415.585147:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f480 x1874663043154560/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827415.585195:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043154304:192.168.203.147@tcp:250:kworker.0 00000800:00000100:1.0:1787827415.585285:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827415.585366:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827415/real 1787827415] req@ffff9181e007d880 x1874663043154304/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827440 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827415.585389:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d880 x1874663043154304/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827440 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827415.585401:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827415.585406:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827415.585425:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043154304:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827415.585434:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043154432:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827415.585467:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827415.585490:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827415/real 1787827415] req@ffff9181e007ce00 x1874663043154432/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827440 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827415.585513:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ce00 x1874663043154432/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827440 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827415.585527:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827415.585534:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827415.585540:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043154432:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827415.585551:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043154560:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827415.585586:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827415.585605:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827415/real 1787827415] req@ffff9181e007f480 x1874663043154560/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827440 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827415.585619:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f480 x1874663043154560/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827440 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827415.585628:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827415.585631:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827415.585634:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043154560:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787827415.649005:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827416.672932:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827417.697059:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827418.720920:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827419.744984:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827420.704862:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827420.704875:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827420.704880:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827420.704892:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9487 00000100:00080000:2.0:1787827420.704916:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827420.704926:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827420.704969:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f800 x1874663043154688/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827420.705026:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827420.705032:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827420.705034:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827420.705040:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9487 00000100:00080000:2.0:1787827420.705045:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827420.705049:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827420.705073:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827420.705092:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c700 x1874663043154816/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827420.705108:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827420.705114:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827420.705121:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1787827420.705122:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043154688:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787827420.705141:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9487 00000100:00080000:2.0:1787827420.705151:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827420.705156:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827420.705162:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827420.705175:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f100 x1874663043154944/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827420.705241:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827420.705284:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827420/real 1787827420] req@ffff9181e007f800 x1874663043154688/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827445 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827420.705339:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f800 x1874663043154688/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827445 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827420.705361:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827420.705370:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827420.705397:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043154688:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827420.705443:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043154816:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827420.705482:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827420.705500:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827420/real 1787827420] req@ffff9181e007c700 x1874663043154816/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827445 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827420.705518:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c700 x1874663043154816/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827445 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827420.705530:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827420.705534:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827420.705539:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043154816:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827420.705546:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043154944:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827420.705566:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827420.705578:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827420/real 1787827420] req@ffff9181e007f100 x1874663043154944/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827445 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827420.705593:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f100 x1874663043154944/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827445 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827420.705603:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827420.705606:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827420.705610:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043154944:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827420.768937:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827421.792916:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827422.816882:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827423.840932:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827424.864965:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827425.824921:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827425.824937:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827425.824944:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827425.824959:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9492 00000100:00080000:1.0:1787827425.824973:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827425.824985:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787827425.825047:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2680 x1874663043155072/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827425.825091:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827425.825097:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827425.825100:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827425.825108:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9492 00000100:00080000:1.0:1787827425.825116:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827425.825122:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827425.825131:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043155072:192.168.203.147@tcp:250:kworker.0 00000008:00100000:1.0:1787827425.825135:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827425.825174:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0e00 x1874663043155200/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827425.825205:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827425.825212:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827425.825215:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827425.825221:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9492 00000100:00080000:1.0:1787827425.825230:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827425.825239:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827425.825248:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827425.825286:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3480 x1874663043155328/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827425.825332:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043155200:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827425.825405:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043155328:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787827425.888943:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827426.912898:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827427.936898:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1787827428.897082:0:2376:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:1.0:1787827428.897153:0:2376:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787827428.897187:0:2376:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1787827428.897228:0:2376:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1787827428.897235:0:2376:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:3.0:1787827428.897323:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827425/real 1787827428] req@ffff9181c35b0e00 x1874663043155200/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827450 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827428.897366:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0e00 x1874663043155200/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827450 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827428.897381:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827428.897387:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827428.897393:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043155200:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827428.897402:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827425/real 1787827428] req@ffff9181c35b3480 x1874663043155328/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827450 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827428.897416:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3480 x1874663043155328/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827450 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827428.897423:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827428.897427:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827428.897430:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043155328:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827428.897438:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827425/real 1787827428] req@ffff9181c35b2680 x1874663043155072/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827450 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827428.897450:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2680 x1874663043155072/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827450 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827428.897458:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827428.897462:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827428.897481:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043155072:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787827428.960942:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827429.984885:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:2.0:1787827430.496878:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-sptlrpc' failed (-108). 10000000:00020000:2.0:1787827430.496890:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00080000:2.0:1787827430.944917:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827430.944935:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827430.944941:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827430.944957:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9497 00000100:00080000:2.0:1787827430.944973:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827430.944982:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827430.945031:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e300 x1874663043155456/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827430.945066:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827430.945069:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827430.945072:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827430.945076:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9497 00000100:00080000:2.0:1787827430.945078:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827430.945082:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827430.945093:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827430.945103:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ea00 x1874663043155584/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827430.945112:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827430.945115:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827430.945117:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827430.945121:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9497 00000100:00080000:2.0:1787827430.945125:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827430.945129:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827430.945135:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827430.945145:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d180 x1874663043155712/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827430.945195:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043155456:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1787827430.945373:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827430.945416:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827430/real 1787827430] req@ffff9181e007e300 x1874663043155456/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827455 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827430.945439:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e300 x1874663043155456/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827455 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827430.945452:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827430.945457:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827430.945463:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043155456:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827430.945473:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043155584:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827430.945503:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827430.945520:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827430/real 1787827430] req@ffff9181e007ea00 x1874663043155584/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827455 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827430.945537:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ea00 x1874663043155584/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827455 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827430.945549:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827430.945554:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827430.945558:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043155584:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827430.945588:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043155712:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827430.945616:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827430.945632:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827430/real 1787827430] req@ffff9181e007d180 x1874663043155712/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827455 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827430.945654:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d180 x1874663043155712/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827455 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827430.945665:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827430.945669:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827430.945673:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043155712:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827431.008975:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827432.032960:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:2.0:1787827432.288803:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:2.0:1787827432.288812:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff9181c5620000 from 341 10000000:01000000:2.0:1787827432.288818:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:2.0:1787827432.288923:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1842e00/0xbef42c0221d6f61e lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827432.288936:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181d1842e00/0xbef42c0221d6f61e lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827432.288955:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181d1842e00/0xbef42c0221d6f61e lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827432.289010:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5c94a80 going to sleep for 0 seconds 00000100:00100000:2.0:1787827432.289026:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043155840:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:2.0:1787827432.289077:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181d1842e00/0xbef42c0221d6f61e lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827432.289094:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181d1842e00/0xbef42c0221d6f61e lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827432.289120:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1842e00/0xbef42c0221d6f61e lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827432.289132:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181d1842e00/0xbef42c0221d6f61e lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827432.289145:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181d1842e00/0xbef42c0221d6f61e lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827432.289155:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181d1842e00 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:2.0:1787827432.289178:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181d1842e00/0xbef42c0221d6f61e lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827432.289193:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181d1842e00/0xbef42c0221d6f61e lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1787827432.289203:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181d1842e00/0xbef42c0221d6f61e lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787827432.289213:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:2.0:1787827432.289219:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787827432.289278:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181d1842e00/0xbef42c0221d6f61e lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827432.289286:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181d1842e00/0xbef42c0221d6f61e lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787827432.289312:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:2.0:1787827432.289315:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:0.0:1787827432.289402:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827432.289412:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827432.289415:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827432.289425:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9502 00000100:00080000:0.0:1787827432.289435:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827432.289442:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787827432.289476:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7b80 x1874663043155968/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827432.289503:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827432.289506:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827432.289508:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827432.289511:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9502 00000100:00080000:0.0:1787827432.289513:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827432.289516:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787827432.289524:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827432.289535:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7100 x1874663043156096/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827432.289544:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827432.289547:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827432.289549:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827432.289563:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9502 00000100:00080000:0.0:1787827432.289566:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827432.289572:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827432.289574:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043155968:192.168.203.147@tcp:250:kworker.0 00000008:00100000:0.0:1787827432.289577:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827432.289590:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6680 x1874663043156224/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787827432.289651:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827432.289686:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827432/real 1787827432] req@ffff9181c55a7b80 x1874663043155968/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827457 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827432.289707:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7b80 x1874663043155968/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827457 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827432.289739:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827432.289746:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827432.289820:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043155968:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827432.289862:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043156096:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827432.289922:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827432.289971:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827432/real 1787827432] req@ffff9181c55a7100 x1874663043156096/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827457 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827432.289993:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7100 x1874663043156096/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827457 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827432.290005:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827432.290008:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827432.290011:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043156096:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827432.290032:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043156224:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827432.290069:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827432.290081:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827432/real 1787827432] req@ffff9181c55a6680 x1874663043156224/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827457 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827432.290093:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6680 x1874663043156224/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827457 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827432.290102:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827432.290105:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827432.290108:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043156224:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827433.056889:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827434.080908:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827435.104845:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827436.128869:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827437.152926:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827437.600895:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827437.600903:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827437.600906:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827437.600914:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9504 00000100:00080000:2.0:1787827437.600924:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827437.600930:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827437.600973:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d500 x1874663043156352/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827437.601006:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827437.601009:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827437.601010:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827437.601014:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9504 00000100:00080000:2.0:1787827437.601016:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827437.601018:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827437.601027:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827437.601035:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007df80 x1874663043156480/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827437.601043:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827437.601045:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827437.601046:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827437.601049:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9504 00000100:00080000:2.0:1787827437.601051:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827437.601054:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827437.601057:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827437.601063:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ed80 x1874663043156608/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827437.601068:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043156352:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1787827437.601149:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827437.601201:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827437/real 1787827437] req@ffff9181e007d500 x1874663043156352/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827462 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827437.601222:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d500 x1874663043156352/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827462 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827437.601233:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827437.601237:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827437.601253:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043156352:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827437.601262:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043156480:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827437.601292:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827437.601307:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827437/real 1787827437] req@ffff9181e007df80 x1874663043156480/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827462 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827437.601321:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007df80 x1874663043156480/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827462 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827437.601330:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827437.601334:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827437.601337:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043156480:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827437.601361:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043156608:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827437.601383:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827437.601395:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827437/real 1787827437] req@ffff9181e007ed80 x1874663043156608/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827462 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827437.601418:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ed80 x1874663043156608/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827462 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827437.601427:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827437.601430:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827437.601434:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043156608:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787827438.176847:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827439.200858:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827440.224901:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827441.248922:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827442.272949:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827442.720897:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827442.720905:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827442.720909:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827442.720917:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9509 00000100:00080000:1.0:1787827442.720927:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827442.720933:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787827442.720970:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2a00 x1874663043156736/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827442.720997:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827442.721000:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827442.721002:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827442.721005:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9509 00000100:00080000:1.0:1787827442.721009:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827442.721012:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827442.721020:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827442.721031:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0a80 x1874663043156864/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827442.721039:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787827442.721041:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043156736:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787827442.721043:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827442.721048:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827442.721052:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9509 00000100:00080000:1.0:1787827442.721054:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827442.721056:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827442.721059:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827442.721086:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2d80 x1874663043156992/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787827442.721126:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827442.721179:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827442/real 1787827442] req@ffff9181c35b2a00 x1874663043156736/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827467 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827442.721212:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2a00 x1874663043156736/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827467 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827442.721221:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827442.721224:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827442.721236:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043156736:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827442.721262:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043156864:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827442.721287:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827442.721302:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827442/real 1787827442] req@ffff9181c35b0a80 x1874663043156864/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827467 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827442.721313:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0a80 x1874663043156864/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827467 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827442.721319:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827442.721321:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827442.721324:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043156864:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827442.721328:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043156992:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827442.721343:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827442.721350:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827442/real 1787827442] req@ffff9181c35b2d80 x1874663043156992/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827467 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827442.721358:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2d80 x1874663043156992/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827467 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827442.721369:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827442.721371:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827442.721373:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043156992:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827443.296926:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827444.320892:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827445.344901:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827446.368907:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827447.392922:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827447.840845:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827447.840853:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827447.840856:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827447.840865:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9514 00000100:00080000:0.0:1787827447.840874:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827447.840880:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787827447.840913:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6680 x1874663043157120/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827447.840935:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827447.840937:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827447.840939:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827447.840942:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9514 00000100:00080000:0.0:1787827447.840944:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827447.840947:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787827447.840954:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827447.840968:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7100 x1874663043157248/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827447.840979:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043157120:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787827447.840983:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827447.840988:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827447.840991:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827447.840996:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9514 00000100:00080000:0.0:1787827447.841001:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827447.841006:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787827447.841011:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827447.841023:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7b80 x1874663043157376/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787827447.841045:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827447.841080:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827447/real 1787827447] req@ffff9181c55a6680 x1874663043157120/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827472 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827447.841093:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6680 x1874663043157120/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827472 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827447.841101:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827447.841104:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827447.841128:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043157120:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827447.841148:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043157248:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827447.841171:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827447.841181:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827447/real 1787827447] req@ffff9181c55a7100 x1874663043157248/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827472 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827447.841197:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7100 x1874663043157248/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827472 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827447.841203:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827447.841206:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827447.841209:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043157248:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827447.841216:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043157376:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827447.841229:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827447.841235:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827447/real 1787827447] req@ffff9181c55a7b80 x1874663043157376/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827472 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827447.841242:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7b80 x1874663043157376/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827472 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827447.841247:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827447.841249:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827447.841251:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043157376:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787827448.416936:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827449.440887:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827450.464924:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827451.488897:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:2.0:1787827452.448862:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-cliir' failed (-108). 10000000:00000400:2.0:1787827452.448869:0:159735:0:(mgc_request.c:1900:mgc_process_log()) MGC192.168.203.147@tcp: IR log lustre-cliir failed, not fatal: rc = -108 10000000:01000000:2.0:1787827452.448876:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:2.0:1787827452.448878:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff9181c5620000 from 52 10000000:01000000:2.0:1787827452.448881:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:2.0:1787827452.448919:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1842e00/0xbef42c0221d6f625 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827452.448933:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181d1842e00/0xbef42c0221d6f625 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827452.448941:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181d1842e00/0xbef42c0221d6f625 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827452.448957:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5c94a80 going to sleep for 0 seconds 00000100:00100000:2.0:1787827452.448966:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ed80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043157504:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:2.0:1787827452.448976:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181d1842e00/0xbef42c0221d6f625 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827452.448981:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181d1842e00/0xbef42c0221d6f625 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827452.448986:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1842e00/0xbef42c0221d6f625 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827452.448991:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181d1842e00/0xbef42c0221d6f625 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827452.448996:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181d1842e00/0xbef42c0221d6f625 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827452.449000:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181d1842e00 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:2.0:1787827452.449003:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181d1842e00/0xbef42c0221d6f625 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827452.449008:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181d1842e00/0xbef42c0221d6f625 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1787827452.449013:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181d1842e00/0xbef42c0221d6f625 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787827452.449017:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:2.0:1787827452.449020:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787827452.449024:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181d1842e00/0xbef42c0221d6f625 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827452.449029:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181d1842e00/0xbef42c0221d6f625 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787827452.449043:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:2.0:1787827452.449045:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:0.0:1787827452.449131:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827452.449138:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827452.449142:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827452.449149:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9519 00000100:00080000:0.0:1787827452.449160:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827452.449166:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787827452.449191:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7480 x1874663043157632/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827452.449216:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827452.449220:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827452.449222:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827452.449227:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9519 00000100:00080000:0.0:1787827452.449231:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827452.449236:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787827452.449246:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827452.449250:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043157632:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787827452.449263:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6d80 x1874663043157760/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827452.449281:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827452.449285:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827452.449287:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827452.449292:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9519 00000100:00080000:0.0:1787827452.449296:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827452.449301:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787827452.449307:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827452.449322:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5880 x1874663043157888/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787827452.449326:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827452.449359:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827452/real 1787827452] req@ffff9181c55a7480 x1874663043157632/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827477 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827452.449397:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7480 x1874663043157632/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827477 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827452.449410:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827452.449414:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827452.449426:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043157632:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827452.449443:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043157760:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827452.449470:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827452.449483:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827452/real 1787827452] req@ffff9181c55a6d80 x1874663043157760/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827477 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827452.449503:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6d80 x1874663043157760/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827477 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827452.449511:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827452.449514:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827452.449518:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043157760:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827452.449524:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043157888:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827452.449565:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827452.449578:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827452/real 1787827452] req@ffff9181c55a5880 x1874663043157888/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827477 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827452.449590:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5880 x1874663043157888/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827477 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827452.449598:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827452.449601:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827452.449605:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043157888:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787827452.512908:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827453.536903:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827454.560934:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827455.584971:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827456.608906:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827457.568838:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827457.568848:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827457.568852:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827457.568862:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9524 00000100:00080000:3.0:1787827457.568873:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827457.568880:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827457.568915:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043158016/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827457.568945:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827457.568949:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827457.568951:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827457.568955:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9524 00000100:00080000:3.0:1787827457.568959:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827457.568964:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827457.568974:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827457.568986:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043158016:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827457.568989:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634e00 x1874663043158144/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827457.569015:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827457.569019:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827457.569021:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827457.569026:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9524 00000100:00080000:3.0:1787827457.569030:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827457.569034:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827457.569039:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000800:00000100:1.0:1787827457.569046:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827457.569071:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043158272/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827457.569090:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827457/real 1787827457] req@ffff9181c5634380 x1874663043158016/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827482 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827457.569117:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043158016/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827482 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827457.569127:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827457.569130:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827457.569142:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043158016:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827457.569162:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043158144:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827457.569181:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827457.569196:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827457/real 1787827457] req@ffff9181c5634e00 x1874663043158144/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827482 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827457.569210:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634e00 x1874663043158144/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827482 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827457.569219:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827457.569222:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827457.569226:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043158144:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827457.569232:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043158272:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827457.569251:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827457.569259:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827457/real 1787827457] req@ffff9181c5635500 x1874663043158272/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827482 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827457.569267:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043158272/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827482 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827457.569277:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827457.569279:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827457.569281:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043158272:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827457.632871:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827458.656917:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827459.680875:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827460.704894:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827461.728869:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827462.688821:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827462.688830:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827462.688834:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827462.688842:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9529 00000100:00080000:3.0:1787827462.688859:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827462.688866:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827462.688899:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635f80 x1874663043158400/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827462.688921:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827462.688925:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827462.688927:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827462.688931:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9529 00000100:00080000:3.0:1787827462.688935:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827462.688940:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827462.688949:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827462.688966:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043158528/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827462.688976:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043158400:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787827462.688980:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827462.688986:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827462.688988:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827462.688993:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9529 00000100:00080000:3.0:1787827462.688997:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827462.689022:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827462.689029:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827462.689041:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634e00 x1874663043158656/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827462.689057:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827462.689095:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827462/real 1787827462] req@ffff9181c5635f80 x1874663043158400/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827487 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827462.689112:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635f80 x1874663043158400/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827487 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827462.689122:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827462.689126:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827462.689142:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043158400:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827462.689171:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043158528:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827462.689197:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827462.689210:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827462/real 1787827462] req@ffff9181c5635500 x1874663043158528/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827487 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827462.689221:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043158528/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827487 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827462.689231:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827462.689234:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827462.689237:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043158528:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827462.689244:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043158656:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827462.689261:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827462.689270:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827462/real 1787827462] req@ffff9181c5634e00 x1874663043158656/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827487 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827462.689281:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634e00 x1874663043158656/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827487 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827462.689289:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827462.689293:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827462.689296:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043158656:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827462.752920:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827463.776877:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827464.800864:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827465.824921:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827466.848958:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827467.808857:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827467.808866:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827467.808869:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827467.808878:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9534 00000100:00080000:3.0:1787827467.808887:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827467.808893:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827467.808926:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043158784/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827467.808947:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827467.808950:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827467.808951:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827467.808954:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9534 00000100:00080000:3.0:1787827467.808956:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827467.808964:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827467.808971:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827467.808996:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043158912/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827467.809004:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827467.809006:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827467.809008:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827467.809011:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9534 00000100:00080000:3.0:1787827467.809015:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827467.809023:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827467.809027:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827467.809038:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043159040/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827467.809039:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043158784:192.168.203.147@tcp:250:kworker.0 00000800:00000100:1.0:1787827467.809100:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827467.809129:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827467/real 1787827467] req@ffff9181c5634380 x1874663043158784/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827492 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827467.809142:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043158784/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827492 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827467.809151:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827467.809153:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827467.809168:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043158784:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827467.809173:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043158912:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827467.809187:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827467.809195:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827467/real 1787827467] req@ffff9181c5636a00 x1874663043158912/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827492 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827467.809204:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043158912/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827492 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827467.809210:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827467.809213:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827467.809216:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043158912:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827467.809240:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043159040:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827467.809255:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827467.809264:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827467/real 1787827467] req@ffff9181c5637480 x1874663043159040/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827492 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827467.809272:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043159040/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827492 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827467.809278:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827467.809280:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827467.809282:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043159040:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827467.872897:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827468.896881:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827469.920922:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827470.944875:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827471.968915:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827472.928806:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827472.928816:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827472.928819:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827472.928830:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9539 00000100:00080000:3.0:1787827472.928840:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827472.928847:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 10000000:01000000:2.0:1787827472.928863:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-client' failed (-108). 10000000:00020000:2.0:1787827472.928869:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:2.0:1787827472.928873:0:159735:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:2.0:1787827472.928875:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff9181c5620000 from 7 10000000:01000000:2.0:1787827472.928878:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00000100:00100000:3.0:1787827472.928880:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043159168/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:2.0:1787827472.928908:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1843200/0xbef42c0221d6f62c lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787827472.928910:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827472.928914:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827472.928916:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00010000:00010000:2.0:1787827472.928920:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181d1843200/0xbef42c0221d6f62c lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787827472.928925:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9539 00000100:00080000:3.0:1787827472.928928:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827472.928932:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:2.0:1787827472.928933:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181d1843200/0xbef42c0221d6f62c lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000008:00100000:3.0:1787827472.928949:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827472.928961:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5c94a80 going to sleep for 0 seconds 00000100:00100000:3.0:1787827472.928963:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043159424/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827472.928970:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ed80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043159296:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00080000:3.0:1787827472.928974:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827472.928979:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:1.0:1787827472.928982:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043159168:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787827472.928982:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00010000:00010000:2.0:1787827472.928984:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181d1843200/0xbef42c0221d6f62c lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787827472.928997:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9539 00010000:00010000:2.0:1787827472.928999:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181d1843200/0xbef42c0221d6f62c lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787827472.929001:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827472.929005:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:2.0:1787827472.929007:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1843200/0xbef42c0221d6f62c lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000008:00100000:3.0:1787827472.929011:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:2.0:1787827472.929016:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181d1843200/0xbef42c0221d6f62c lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787827472.929024:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043159552/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:2.0:1787827472.929025:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181d1843200/0xbef42c0221d6f62c lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827472.929033:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181d1843200 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:2.0:1787827472.929037:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181d1843200/0xbef42c0221d6f62c lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827472.929045:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181d1843200/0xbef42c0221d6f62c lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000800:00000100:1.0:1787827472.929055:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 10000000:00010000:2.0:1787827472.929055:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181d1843200/0xbef42c0221d6f62c lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787827472.929065:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:2.0:1787827472.929069:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787827472.929077:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181d1843200/0xbef42c0221d6f62c lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827472.929085:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181d1843200/0xbef42c0221d6f62c lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787827472.929092:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827472/real 1787827472] req@ffff9181c5637100 x1874663043159168/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827497 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 10000000:01000000:2.0:1787827472.929104:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:2.0:1787827472.929107:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:CONNECTING 00000100:00100000:1.0:1787827472.929113:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043159168/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827497 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827472.929122:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827472.929135:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827472.929141:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043159168:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827472.929158:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043159424:192.168.203.147@tcp:38:kworker.0 00000100:00080000:3.0:1787827472.929161:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827472.929167:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827472.929170:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827472.929175:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9544 00000800:00000100:1.0:1787827472.929180:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1787827472.929182:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827472.929189:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787827472.929193:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827472/real 1787827472] req@ffff9181c5637800 x1874663043159424/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827497 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827472.929207:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043159680/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827472.929214:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043159424/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827497 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827472.929233:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827472.929237:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827472.929240:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043159424:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827472.929246:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043159552:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827472.929259:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827472.929267:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827472/real 1787827472] req@ffff9181c5636d80 x1874663043159552/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827497 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827472.929274:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043159552/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827497 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827472.929280:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827472.929282:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827472.929284:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043159552:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827472.929292:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043159680:192.168.203.147@tcp:250:kworker.0 00000800:00000100:1.0:1787827472.929301:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827472.929307:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827472/real 1787827472] req@ffff9181c5635c00 x1874663043159680/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827497 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827472.929314:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043159680/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827497 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827472.929319:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827472.929320:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827472.929331:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043159680:192.168.203.147@tcp:250:kworker.0 00010000:00010000:0.0:1787827472.993046:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827474.016901:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827475.040862:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827476.064890:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827477.088909:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827478.048840:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827478.048849:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827478.048853:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827478.048861:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9544 00000100:00080000:2.0:1787827478.048871:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827478.048877:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827478.048923:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ed80 x1874663043159808/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827478.048947:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827478.048950:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827478.048951:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827478.048954:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9544 00000100:00080000:2.0:1787827478.048957:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827478.048960:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827478.048968:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827478.048979:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007df80 x1874663043159936/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827478.048986:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827478.048989:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827478.048991:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827478.048994:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9544 00000100:00080000:2.0:1787827478.048997:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1787827478.049001:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043159808:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787827478.049002:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827478.049009:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827478.049016:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d500 x1874663043160064/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827478.049062:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827478.049092:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827478/real 1787827478] req@ffff9181e007ed80 x1874663043159808/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827503 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827478.049105:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ed80 x1874663043159808/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827503 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827478.049113:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827478.049116:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827478.049129:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043159808:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827478.049134:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043159936:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827478.049151:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827478.049161:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827478/real 1787827478] req@ffff9181e007df80 x1874663043159936/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827503 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827478.049169:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007df80 x1874663043159936/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827503 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827478.049175:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827478.049177:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827478.049179:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043159936:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827478.049197:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043160064:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827478.049207:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827478.049213:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827478/real 1787827478] req@ffff9181e007d500 x1874663043160064/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827503 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827478.049225:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d500 x1874663043160064/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827503 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827478.049230:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827478.049232:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827478.049234:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043160064:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827478.112933:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827479.136891:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827480.160878:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827481.184858:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827482.208957:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827483.168884:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827483.168893:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827483.168896:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827483.168905:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9549 00000100:00080000:2.0:1787827483.168915:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827483.168921:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827483.168954:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d180 x1874663043160192/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827483.168978:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827483.168981:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827483.168983:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827483.168986:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9549 00000100:00080000:2.0:1787827483.168988:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827483.168991:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827483.168997:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827483.169007:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ea00 x1874663043160320/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827483.169016:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827483.169018:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827483.169020:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827483.169023:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9549 00000100:00080000:2.0:1787827483.169025:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827483.169030:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787827483.169031:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043160192:192.168.203.147@tcp:250:kworker.0 00000008:00100000:2.0:1787827483.169033:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827483.169045:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e300 x1874663043160448/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827483.169097:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827483.169129:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827483/real 1787827483] req@ffff9181e007d180 x1874663043160192/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827508 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827483.169146:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d180 x1874663043160192/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827508 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827483.169155:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827483.169159:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827483.169186:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043160192:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827483.169193:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043160320:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827483.169218:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827483.169235:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827483/real 1787827483] req@ffff9181e007ea00 x1874663043160320/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827508 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827483.169247:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ea00 x1874663043160320/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827508 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827483.169255:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827483.169259:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827483.169264:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043160320:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827483.169293:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043160448:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827483.169321:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827483.169341:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827483/real 1787827483] req@ffff9181e007e300 x1874663043160448/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827508 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827483.169356:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e300 x1874663043160448/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827508 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827483.169365:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827483.169369:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827483.169373:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043160448:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827483.232956:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827484.256884:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827485.280932:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827486.304878:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827487.328953:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827488.288858:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827488.288889:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827488.288893:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827488.288905:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9554 00000100:00080000:2.0:1787827488.288918:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827488.288927:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827488.288971:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f100 x1874663043160576/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827488.289010:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827488.289031:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827488.289034:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827488.289041:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9554 00000100:00080000:2.0:1787827488.289045:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827488.289052:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827488.289065:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827488.289089:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c700 x1874663043160704/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827488.289092:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043160576:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787827488.289117:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827488.289121:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827488.289123:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827488.289127:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9554 00000100:00080000:2.0:1787827488.289132:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827488.289137:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827488.289143:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827488.289156:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f800 x1874663043160832/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827488.289218:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043160704:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827488.289264:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043160832:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827488.352990:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827489.376856:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827490.401041:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:3.0:1787827491.361083:0:2377:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:3.0:1787827491.361124:0:2377:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1787827491.361139:0:2377:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:3.0:1787827491.361189:0:2377:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:3.0:1787827491.361195:0:2377:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:1.0:1787827491.361233:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827488/real 1787827491] req@ffff9181e007c700 x1874663043160704/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827513 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827491.361257:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c700 x1874663043160704/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827513 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827491.361270:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827491.361275:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827491.361279:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043160704:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827491.361288:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827488/real 1787827491] req@ffff9181e007f800 x1874663043160832/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827513 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827491.361300:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f800 x1874663043160832/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827513 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827491.361307:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827491.361312:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827491.361315:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043160832:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827491.361322:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827488/real 1787827491] req@ffff9181e007f100 x1874663043160576/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827513 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827491.361336:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f100 x1874663043160576/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827513 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827491.361344:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827491.361347:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827491.361362:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043160576:192.168.203.147@tcp:250:kworker.0 00010000:00010000:2.0:1787827491.424933:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827492.448884:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:2.0:1787827492.960914:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'params' failed (-108). 10000000:00020000:2.0:1787827492.960925:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:2.0:1787827492.960942:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:2.0:1787827492.960945:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff9181c87b19b8 from 1 10000000:01000000:2.0:1787827492.960949:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00010000:00010000:2.0:1787827492.961017:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1842a00/0xbef42c0221d6f633 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827492.961041:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181d1842a00/0xbef42c0221d6f633 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827492.961053:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181d1842a00/0xbef42c0221d6f633 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827492.961080:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5c94a80 going to sleep for 0 seconds 00000100:00100000:2.0:1787827492.961111:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043160960:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:2.0:1787827492.961131:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181d1842a00/0xbef42c0221d6f633 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827492.961172:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181d1842a00/0xbef42c0221d6f633 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827492.961180:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1842a00/0xbef42c0221d6f633 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827492.961190:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181d1842a00/0xbef42c0221d6f633 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827492.961199:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181d1842a00/0xbef42c0221d6f633 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827492.961204:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181d1842a00 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:2.0:1787827492.961218:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181d1842a00/0xbef42c0221d6f633 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827492.961224:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181d1842a00/0xbef42c0221d6f633 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1787827492.961229:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181d1842a00/0xbef42c0221d6f633 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787827492.961234:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:2.0:1787827492.961237:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787827492.961242:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181d1842a00/0xbef42c0221d6f633 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827492.961248:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181d1842a00/0xbef42c0221d6f633 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787827492.961270:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:2.0:1787827492.961284:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:3.0:1787827492.961375:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827492.961381:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827492.961384:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827492.961390:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9560 00000100:00080000:3.0:1787827492.961399:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827492.961404:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827492.961423:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043161088/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827492.961457:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827492.961461:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827492.961463:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827492.961468:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9560 00000100:00080000:3.0:1787827492.961472:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827492.961477:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827492.961487:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827492.961502:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634a80 x1874663043161216/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827492.961509:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043161088:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787827492.961515:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827492.961519:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827492.961521:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827492.961525:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9560 00000100:00080000:3.0:1787827492.961527:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827492.961530:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827492.961535:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827492.961546:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637b80 x1874663043161344/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827492.961571:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827492.961599:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827492/real 1787827492] req@ffff9181c5634000 x1874663043161088/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827517 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827492.961611:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043161088/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827517 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827492.961619:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827492.961622:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827492.961634:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043161088:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827492.961649:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043161216:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827492.961672:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827492.961682:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827492/real 1787827492] req@ffff9181c5634a80 x1874663043161216/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827517 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827492.961690:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634a80 x1874663043161216/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827517 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827492.961696:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827492.961698:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827492.961700:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043161216:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827492.961704:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043161344:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827492.961717:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827492.961734:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827492/real 1787827492] req@ffff9181c5637b80 x1874663043161344/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827517 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827492.961782:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637b80 x1874663043161344/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827517 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827492.961792:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827492.961797:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827492.961835:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043161344:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787827493.473005:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827494.496987:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827495.520875:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827496.545012:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827497.568907:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827498.016924:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827498.016934:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827498.016939:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827498.016950:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9564 00000100:00080000:2.0:1787827498.016962:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827498.016970:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827498.017026:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f480 x1874663043161472/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827498.017066:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827498.017069:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827498.017071:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827498.017074:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9564 00000100:00080000:2.0:1787827498.017077:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827498.017079:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827498.017089:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827498.017100:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ce00 x1874663043161600/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827498.017108:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827498.017111:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827498.017113:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827498.017117:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9564 00000100:00080000:2.0:1787827498.017119:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827498.017123:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827498.017126:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827498.017134:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043161472:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787827498.017139:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d880 x1874663043161728/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827498.017214:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827498.017250:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827498/real 1787827498] req@ffff9181e007f480 x1874663043161472/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827523 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827498.017276:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f480 x1874663043161472/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827523 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827498.017286:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827498.017290:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827498.017301:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043161472:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827498.017313:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043161600:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827498.017338:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827498.017351:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827498/real 1787827498] req@ffff9181e007ce00 x1874663043161600/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827523 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827498.017360:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ce00 x1874663043161600/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827523 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827498.017368:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827498.017370:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827498.017373:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043161600:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827498.017394:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043161728:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827498.017419:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827498.017426:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827498/real 1787827498] req@ffff9181e007d880 x1874663043161728/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827523 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827498.017435:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d880 x1874663043161728/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827523 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827498.017441:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827498.017444:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827498.017446:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043161728:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827498.592993:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827499.616996:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827500.641023:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827501.664877:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827502.688910:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827503.136838:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827503.136848:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827503.136853:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827503.136865:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9569 00000100:00080000:1.0:1787827503.136877:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827503.136884:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787827503.136927:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0700 x1874663043161856/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827503.136953:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827503.136967:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827503.136969:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827503.136972:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9569 00000100:00080000:1.0:1787827503.136974:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827503.136977:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827503.136987:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827503.136995:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1f80 x1874663043161984/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827503.137003:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827503.137006:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827503.137007:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827503.137012:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9569 00000100:00080000:1.0:1787827503.137015:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827503.137018:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827503.137031:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827503.137037:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3800 x1874663043162112/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827503.137073:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043161856:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1787827503.137174:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827503.137229:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827503/real 1787827503] req@ffff9181c35b0700 x1874663043161856/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827528 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827503.137262:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0700 x1874663043161856/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827528 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827503.137276:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827503.137281:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827503.137305:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043161856:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827503.137317:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043161984:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827503.137352:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827503.137387:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827503/real 1787827503] req@ffff9181c35b1f80 x1874663043161984/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827528 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827503.137407:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1f80 x1874663043161984/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827528 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827503.137421:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827503.137426:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827503.137432:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043161984:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827503.137470:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043162112:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827503.137503:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827503.137523:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827503/real 1787827503] req@ffff9181c35b3800 x1874663043162112/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827528 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827503.137544:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3800 x1874663043162112/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827528 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827503.137566:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827503.137572:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827503.137594:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043162112:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827503.712905:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827504.736858:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827505.760868:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827506.784890:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827507.808881:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827508.256800:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827508.256807:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827508.256810:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827508.256818:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9574 00000100:00080000:1.0:1787827508.256827:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827508.256833:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787827508.256864:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3b80 x1874663043162240/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827508.256886:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827508.256889:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827508.256894:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827508.256897:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9574 00000100:00080000:1.0:1787827508.256899:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827508.256902:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827508.256914:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827508.256928:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043162240:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827508.256929:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3800 x1874663043162368/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827508.256942:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827508.256944:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827508.256946:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827508.256950:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9574 00000100:00080000:1.0:1787827508.256952:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827508.256955:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827508.256959:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827508.256968:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1f80 x1874663043162496/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787827508.256989:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827508.257024:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827508/real 1787827508] req@ffff9181c35b3b80 x1874663043162240/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827533 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827508.257037:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3b80 x1874663043162240/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827533 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827508.257045:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827508.257048:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827508.257059:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043162240:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827508.257077:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043162368:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827508.257093:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827508.257103:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827508/real 1787827508] req@ffff9181c35b3800 x1874663043162368/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827533 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827508.257116:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3800 x1874663043162368/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827533 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827508.257126:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827508.257131:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827508.257136:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043162368:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827508.257147:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043162496:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827508.257168:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827508.257176:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827508/real 1787827508] req@ffff9181c35b1f80 x1874663043162496/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827533 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827508.257184:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1f80 x1874663043162496/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827533 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827508.257189:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827508.257192:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827508.257194:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043162496:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827508.832944:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827509.856966:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827510.880953:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827511.904912:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827512.928927:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827513.376882:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827513.376897:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827513.376903:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 10000000:01000000:3.0:1787827513.376911:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-sptlrpc' failed (-108). 00000100:00080000:2.0:1787827513.376921:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9579 10000000:00020000:3.0:1787827513.376922:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00080000:2.0:1787827513.376940:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827513.376958:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827513.377010:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007fb80 x1874663043162624/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827513.377042:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827513.377046:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827513.377049:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827513.377055:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9579 00000100:00080000:2.0:1787827513.377061:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827513.377068:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827513.377077:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043162624:192.168.203.147@tcp:250:kworker.0 00000008:00100000:2.0:1787827513.377082:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827513.377104:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ca80 x1874663043162752/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827513.377120:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827513.377124:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827513.377128:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827513.377134:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9579 00000100:00080000:2.0:1787827513.377139:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000800:00000100:3.0:1787827513.377143:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1787827513.377150:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827513.377159:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827513.377177:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e680 x1874663043162880/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827513.377178:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827513/real 1787827513] req@ffff9181e007fb80 x1874663043162624/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827538 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827513.377236:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007fb80 x1874663043162624/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827538 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827513.377247:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827513.377251:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827513.377254:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043162624:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827513.377273:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043162752:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827513.377290:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827513.377299:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827513/real 1787827513] req@ffff9181e007ca80 x1874663043162752/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827538 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827513.377307:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ca80 x1874663043162752/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827538 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827513.377312:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827513.377314:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827513.377316:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043162752:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827513.377320:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043162880:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827513.377331:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827513.377337:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827513/real 1787827513] req@ffff9181e007e680 x1874663043162880/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827538 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827513.377344:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e680 x1874663043162880/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827538 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827513.377349:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827513.377351:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827513.377353:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043162880:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827513.952881:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:3.0:1787827514.592838:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:3.0:1787827514.592845:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff9181c5620000 from 341 10000000:01000000:3.0:1787827514.592849:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:3.0:1787827514.592896:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f63a lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827514.592913:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f63a lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827514.592926:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f63a lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787827514.592950:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d225f0c0 going to sleep for 0 seconds 00000100:00100000:3.0:1787827514.592961:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043163008:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:3.0:1787827514.592977:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f63a lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827514.592983:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f63a lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827514.592988:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f63a lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827514.592993:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f63a lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827514.592998:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f63a lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827514.593003:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3480000 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:3.0:1787827514.593005:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f63a lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827514.593011:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f63a lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1787827514.593017:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f63a lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787827514.593021:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:3.0:1787827514.593023:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787827514.593028:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f63a lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787827514.593032:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f63a lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787827514.593050:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:3.0:1787827514.593052:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:1.0:1787827514.593126:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827514.593132:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827514.593136:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827514.593142:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9585 00000100:00080000:1.0:1787827514.593153:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827514.593158:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787827514.593186:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0700 x1874663043163136/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827514.593204:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827514.593207:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827514.593208:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827514.593212:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9585 00000100:00080000:1.0:1787827514.593215:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827514.593219:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827514.593227:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827514.593234:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043163136:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827514.593239:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2d80 x1874663043163264/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827514.593247:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827514.593250:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827514.593251:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827514.593254:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9585 00000100:00080000:1.0:1787827514.593261:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827514.593265:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827514.593268:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827514.593277:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0a80 x1874663043163392/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787827514.593302:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827514.593328:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827514/real 1787827514] req@ffff9181c35b0700 x1874663043163136/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827539 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827514.593340:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0700 x1874663043163136/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827539 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827514.593347:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827514.593351:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827514.593363:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043163136:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827514.593379:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043163264:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827514.593402:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827514.593411:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827514/real 1787827514] req@ffff9181c35b2d80 x1874663043163264/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827539 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827514.593420:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2d80 x1874663043163264/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827539 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827514.593426:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827514.593428:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827514.593430:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043163264:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827514.593434:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043163392:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827514.593447:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827514.593455:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827514/real 1787827514] req@ffff9181c35b0a80 x1874663043163392/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827539 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827514.593464:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0a80 x1874663043163392/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827539 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827514.593470:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827514.593473:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827514.593475:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043163392:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827514.976934:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827516.000970:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827517.024936:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827518.048889:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827519.072903:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827520.032866:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827520.032874:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827520.032877:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827520.032886:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9586 00000100:00080000:2.0:1787827520.032896:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827520.032902:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827520.032935:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007dc00 x1874663043163520/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827520.032958:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827520.032961:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827520.032964:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827520.032967:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9586 00000100:00080000:2.0:1787827520.032969:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827520.032972:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827520.032979:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827520.032988:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c000 x1874663043163648/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827520.032996:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827520.032999:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827520.033000:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827520.033003:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9586 00000100:00080000:2.0:1787827520.033004:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827520.033007:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827520.033010:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827520.033020:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043163520:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787827520.033029:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c380 x1874663043163776/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787827520.033096:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827520.033126:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827520/real 1787827520] req@ffff9181e007dc00 x1874663043163520/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827545 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827520.033139:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007dc00 x1874663043163520/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827545 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827520.033151:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827520.033154:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827520.033166:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043163520:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827520.033171:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043163648:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827520.033188:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827520.033220:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827520/real 1787827520] req@ffff9181e007c000 x1874663043163648/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827545 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827520.033232:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c000 x1874663043163648/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827545 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827520.033238:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827520.033240:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827520.033243:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043163648:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827520.033262:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043163776:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827520.033291:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827520.033298:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827520/real 1787827520] req@ffff9181e007c380 x1874663043163776/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827545 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827520.033305:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c380 x1874663043163776/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827545 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827520.033310:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827520.033312:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827520.033313:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043163776:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827520.096881:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827521.120947:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827522.144931:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827523.168879:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827524.192875:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827525.152876:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827525.152892:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827525.152897:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827525.152908:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9591 00000100:00080000:2.0:1787827525.152920:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827525.152927:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827525.152986:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c380 x1874663043163904/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827525.153015:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827525.153019:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827525.153020:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827525.153024:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9591 00000100:00080000:2.0:1787827525.153026:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827525.153029:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827525.153039:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827525.153053:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c000 x1874663043164032/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827525.153065:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827525.153069:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827525.153071:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827525.153076:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9591 00000100:00080000:2.0:1787827525.153081:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827525.153088:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827525.153094:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827525.153103:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043163904:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787827525.153108:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007dc00 x1874663043164160/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787827525.153199:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827525.153238:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827525/real 1787827525] req@ffff9181e007c380 x1874663043163904/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827550 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827525.153254:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c380 x1874663043163904/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827550 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827525.153264:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827525.153267:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827525.153284:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043163904:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827525.153292:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043164032:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827525.153325:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827525.153354:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827525/real 1787827525] req@ffff9181e007c000 x1874663043164032/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827550 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827525.153364:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c000 x1874663043164032/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827550 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827525.153371:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827525.153373:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827525.153376:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043164032:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827525.153397:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043164160:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827525.153428:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827525.153436:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827525/real 1787827525] req@ffff9181e007dc00 x1874663043164160/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827550 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827525.153445:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007dc00 x1874663043164160/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827550 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827525.153451:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827525.153454:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827525.153456:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043164160:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827525.216959:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827526.240966:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827527.264901:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827528.288924:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827529.312873:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827530.272865:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827530.272883:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827530.272889:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827530.272918:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9596 00000100:00080000:2.0:1787827530.272933:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827530.272944:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827530.273003:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e680 x1874663043164288/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827530.273053:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827530.273058:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827530.273061:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827530.273067:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9596 00000100:00080000:2.0:1787827530.273073:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827530.273080:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827530.273094:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827530.273116:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ca80 x1874663043164416/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827530.273122:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043164288:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787827530.273151:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827530.273157:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827530.273161:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827530.273171:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9596 00000100:00080000:2.0:1787827530.273179:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827530.273186:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827530.273208:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1787827530.273230:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787827530.273231:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007fb80 x1874663043164544/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827530.273287:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827530/real 1787827530] req@ffff9181e007e680 x1874663043164288/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827555 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827530.273309:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e680 x1874663043164288/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827555 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827530.273322:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827530.273327:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827530.273346:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043164288:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827530.273378:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043164416:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827530.273415:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827530.273428:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827530/real 1787827530] req@ffff9181e007ca80 x1874663043164416/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827555 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827530.273442:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ca80 x1874663043164416/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827555 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827530.273452:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827530.273455:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827530.273459:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043164416:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827530.273465:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043164544:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827530.273483:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827530.273493:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827530/real 1787827530] req@ffff9181e007fb80 x1874663043164544/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827555 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827530.273505:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007fb80 x1874663043164544/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827555 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827530.273513:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827530.273516:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827530.273519:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043164544:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827530.336928:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827531.361001:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827532.385013:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827533.408963:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827534.432922:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:2.0:1787827534.880963:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-cliir' failed (-108). 10000000:00000400:2.0:1787827534.880978:0:159735:0:(mgc_request.c:1900:mgc_process_log()) MGC192.168.203.147@tcp: IR log lustre-cliir failed, not fatal: rc = -108 10000000:01000000:2.0:1787827534.886220:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:2.0:1787827534.886224:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff9181c5620000 from 52 10000000:01000000:2.0:1787827534.886227:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:2.0:1787827534.886269:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1842400/0xbef42c0221d6f641 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827534.886278:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181d1842400/0xbef42c0221d6f641 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827534.886287:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181d1842400/0xbef42c0221d6f641 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827534.886304:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5c94240 going to sleep for 0 seconds 00000100:00100000:2.0:1787827534.886314:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043164672:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:2.0:1787827534.886327:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181d1842400/0xbef42c0221d6f641 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827534.886332:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181d1842400/0xbef42c0221d6f641 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827534.886337:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1842400/0xbef42c0221d6f641 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827534.886342:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181d1842400/0xbef42c0221d6f641 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827534.886347:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181d1842400/0xbef42c0221d6f641 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827534.886352:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181d1842400 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:2.0:1787827534.886355:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181d1842400/0xbef42c0221d6f641 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827534.886360:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181d1842400/0xbef42c0221d6f641 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1787827534.886364:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181d1842400/0xbef42c0221d6f641 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787827534.886368:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:2.0:1787827534.886370:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787827534.886374:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181d1842400/0xbef42c0221d6f641 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827534.886379:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181d1842400/0xbef42c0221d6f641 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787827534.886397:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:2.0:1787827534.886401:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:1.0:1787827534.886470:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827534.886476:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827534.886478:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827534.886484:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9602 00000100:00080000:1.0:1787827534.886492:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827534.886499:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787827534.886536:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2a00 x1874663043164800/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827534.886584:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827534.886587:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827534.886589:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827534.886599:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9602 00000100:00080000:1.0:1787827534.886628:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827534.886633:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827534.886641:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827534.886643:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043164800:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827534.886654:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2680 x1874663043164928/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827534.886668:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827534.886671:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827534.886672:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827534.886702:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9602 00000100:00080000:1.0:1787827534.886705:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827534.886708:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827534.886712:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827534.886726:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3480 x1874663043165056/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787827534.886741:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827534.886838:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827534/real 1787827534] req@ffff9181c35b2a00 x1874663043164800/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827559 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827534.886856:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2a00 x1874663043164800/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827559 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827534.886871:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827534.886876:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827534.886888:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043164800:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827534.886910:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043164928:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827534.886934:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827534.886957:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827534/real 1787827534] req@ffff9181c35b2680 x1874663043164928/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827559 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827534.886969:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2680 x1874663043164928/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827559 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827534.886977:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827534.886980:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827534.886983:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043164928:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827534.886990:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043165056:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827534.887017:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827534.887027:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827534/real 1787827534] req@ffff9181c35b3480 x1874663043165056/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827559 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827534.887038:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3480 x1874663043165056/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827559 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827534.887046:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827534.887049:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827534.887052:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043165056:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787827535.456855:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827536.480976:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827537.505003:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827538.529074:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827539.553011:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827540.000864:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827540.000874:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827540.000878:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827540.000887:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9606 00000100:00080000:2.0:1787827540.000898:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827540.000906:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827540.000967:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d880 x1874663043165184/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827540.001019:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827540.001023:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827540.001025:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827540.001031:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9606 00000100:00080000:2.0:1787827540.001035:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827540.001039:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827540.001051:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827540.001067:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ce00 x1874663043165312/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827540.001078:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827540.001082:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827540.001083:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827540.001087:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9606 00000100:00080000:2.0:1787827540.001090:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827540.001094:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827540.001099:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827540.001109:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f480 x1874663043165440/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827540.001124:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043165184:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1787827540.001269:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827540.001328:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827540/real 1787827540] req@ffff9181e007d880 x1874663043165184/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827565 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827540.001364:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d880 x1874663043165184/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827565 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827540.001379:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827540.001385:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827540.001409:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043165184:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827540.001421:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043165312:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827540.001459:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827540.001500:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827540/real 1787827540] req@ffff9181e007ce00 x1874663043165312/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827565 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827540.001541:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ce00 x1874663043165312/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827565 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827540.001551:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827540.001556:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827540.001560:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043165312:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827540.001606:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043165440:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827540.001625:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827540.001634:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827540/real 1787827540] req@ffff9181e007f480 x1874663043165440/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827565 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827540.001643:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f480 x1874663043165440/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827565 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827540.001650:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827540.001652:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827540.001655:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043165440:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787827540.576877:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827541.600935:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827542.624891:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827543.648891:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827544.672882:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827545.120847:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827545.120856:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827545.120860:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827545.120870:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9611 00000100:00080000:2.0:1787827545.120882:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827545.120890:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827545.120926:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f100 x1874663043165568/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827545.120948:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827545.120952:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827545.120954:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827545.120959:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9611 00000100:00080000:2.0:1787827545.120963:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827545.120968:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827545.120977:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827545.120987:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043165568:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787827545.120990:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f800 x1874663043165696/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827545.121000:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827545.121003:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827545.121005:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827545.121009:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9611 00000100:00080000:2.0:1787827545.121011:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827545.121015:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827545.121019:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827545.121028:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c700 x1874663043165824/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787827545.121048:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827545.121080:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827545/real 1787827545] req@ffff9181e007f100 x1874663043165568/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827570 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827545.121093:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f100 x1874663043165568/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827570 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827545.121101:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827545.121103:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827545.121115:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043165568:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827545.121141:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043165696:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827545.121166:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827545.121179:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827545/real 1787827545] req@ffff9181e007f800 x1874663043165696/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827570 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827545.121189:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f800 x1874663043165696/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827570 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827545.121194:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827545.121197:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827545.121199:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043165696:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827545.121203:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043165824:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827545.121217:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827545.121225:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827545/real 1787827545] req@ffff9181e007c700 x1874663043165824/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827570 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827545.121237:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c700 x1874663043165824/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827570 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827545.121245:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827545.121248:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827545.121252:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043165824:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787827545.696882:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827546.720951:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827547.744942:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827548.768924:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827549.793036:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827550.240908:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827550.240921:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827550.240927:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827550.240940:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9616 00000100:00080000:1.0:1787827550.240955:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827550.240964:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787827550.241019:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0e00 x1874663043165952/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827550.241056:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827550.241060:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827550.241062:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827550.241066:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9616 00000100:00080000:1.0:1787827550.241071:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827550.241076:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827550.241089:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827550.241104:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1180 x1874663043166080/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827550.241133:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043165952:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787827550.241134:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827550.241152:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827550.241158:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827550.241164:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9616 00000100:00080000:1.0:1787827550.241168:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827550.241172:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827550.241178:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827550.241191:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0000 x1874663043166208/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787827550.241231:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827550.241297:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827550/real 1787827550] req@ffff9181c35b0e00 x1874663043165952/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827575 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827550.241346:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0e00 x1874663043165952/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827575 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827550.241363:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827550.241369:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827550.241388:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043165952:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827550.241465:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043166080:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827550.241509:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827550.241520:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827550/real 1787827550] req@ffff9181c35b1180 x1874663043166080/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827575 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827550.241530:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1180 x1874663043166080/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827575 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827550.241537:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827550.241540:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827550.241542:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043166080:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827550.241548:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043166208:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827550.241561:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827550.241567:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827550/real 1787827550] req@ffff9181c35b0000 x1874663043166208/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827575 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827550.241576:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0000 x1874663043166208/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827575 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827550.241582:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827550.241584:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827550.241586:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043166208:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787827550.816882:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827551.840928:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827552.864876:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827553.888881:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827554.912931:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827555.360815:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827555.360823:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827555.360826:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827555.360835:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9621 00000100:00080000:1.0:1787827555.360844:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827555.360879:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 10000000:01000000:2.0:1787827555.360909:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-client' failed (-108). 10000000:00020000:2.0:1787827555.360914:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:2.0:1787827555.360918:0:159735:0:(mgc_request.c:611:do_requeue()) updating log params 00000100:00100000:1.0:1787827555.360919:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1c00 x1874663043166336/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 10000000:01000000:2.0:1787827555.360921:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff9181c5620000 from 7 10000000:01000000:2.0:1787827555.360926:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00000100:00080000:1.0:1787827555.360953:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827555.360957:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827555.360959:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00010000:00010000:2.0:1787827555.360960:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1842000/0xbef42c0221d6f648 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787827555.360964:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9621 00000100:00080000:1.0:1787827555.360970:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00010000:00010000:2.0:1787827555.360975:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181d1842000/0xbef42c0221d6f648 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787827555.360976:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827555.360987:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:2.0:1787827555.360996:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181d1842000/0xbef42c0221d6f648 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787827555.360999:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043166336:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827555.361002:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0380 x1874663043166592/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827555.361019:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827555.361023:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827555.361025:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1787827555.361026:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5c94240 going to sleep for 0 seconds 00000100:00080000:1.0:1787827555.361029:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9621 00000100:00080000:1.0:1787827555.361033:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1787827555.361035:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043166464:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00080000:1.0:1787827555.361040:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827555.361045:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:2.0:1787827555.361048:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181d1842000/0xbef42c0221d6f648 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787827555.361055:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3100 x1874663043166720/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:2.0:1787827555.361057:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181d1842000/0xbef42c0221d6f648 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827555.361074:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1842000/0xbef42c0221d6f648 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827555.361082:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181d1842000/0xbef42c0221d6f648 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827555.361087:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181d1842000/0xbef42c0221d6f648 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827555.361092:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181d1842000 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:2.0:1787827555.361096:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181d1842000/0xbef42c0221d6f648 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827555.361103:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181d1842000/0xbef42c0221d6f648 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787827555.361107:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043166592:192.168.203.147@tcp:38:kworker.0 10000000:00010000:2.0:1787827555.361110:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181d1842000/0xbef42c0221d6f648 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787827555.361116:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:2.0:1787827555.361119:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787827555.361124:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181d1842000/0xbef42c0221d6f648 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827555.361129:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181d1842000/0xbef42c0221d6f648 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787827555.361140:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043166720:192.168.203.147@tcp:38:kworker.0 10000000:01000000:2.0:1787827555.361148:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:2.0:1787827555.361151:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:CONNECTING 00010000:00010000:3.0:1787827555.936881:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827556.960885:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827557.984913:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1787827558.432952:0:2378:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:0.0:1787827558.432987:0:2378:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1787827558.433002:0:2378:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1787827558.433032:0:2378:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1787827558.433038:0:2378:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:3.0:1787827558.433087:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827555/real 1787827558] req@ffff9181c35b0380 x1874663043166592/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827580 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827558.433117:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0380 x1874663043166592/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827580 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827558.433130:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827558.433136:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827558.433141:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043166592:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827558.433148:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827555/real 1787827558] req@ffff9181c35b3100 x1874663043166720/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827580 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827558.433158:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3100 x1874663043166720/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827580 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827558.433163:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827558.433166:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827558.433168:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043166720:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827558.433173:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827555/real 1787827558] req@ffff9181c35b1c00 x1874663043166336/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827580 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827558.433180:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1c00 x1874663043166336/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827580 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827558.433185:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827558.433210:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827558.433226:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043166336:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787827558.433255:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827558.433261:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827558.433265:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827558.433271:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9627 00000100:00080000:2.0:1787827558.433278:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827558.433287:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827558.433321:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e300 x1874663043166848/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827558.433359:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827558.433363:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827558.433366:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827558.433371:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9627 00000100:00080000:2.0:1787827558.433375:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827558.433380:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827558.433387:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043166848:192.168.203.147@tcp:250:kworker.0 00000008:00100000:2.0:1787827558.433389:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827558.433402:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ea00 x1874663043166976/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827558.433411:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827558.433414:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827558.433416:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827558.433419:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9627 00000100:00080000:2.0:1787827558.433433:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827558.433437:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827558.433442:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1787827558.433445:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787827558.433452:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d180 x1874663043167104/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827558.433469:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827558/real 1787827558] req@ffff9181e007e300 x1874663043166848/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827583 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827558.433482:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e300 x1874663043166848/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827583 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827558.433497:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827558.433501:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827558.433514:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043166848:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827558.433528:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043166976:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827558.433555:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827558.433568:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827558/real 1787827558] req@ffff9181e007ea00 x1874663043166976/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827583 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827558.433579:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ea00 x1874663043166976/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827583 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827558.433586:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827558.433608:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827558.433612:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043166976:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827558.433618:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043167104:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827558.433638:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827558.433650:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827558/real 1787827558] req@ffff9181e007d180 x1874663043167104/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827583 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827558.434343:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d180 x1874663043167104/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827583 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827558.434365:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827558.434370:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827558.434378:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043167104:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787827559.008891:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827560.032877:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827561.056891:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827562.080899:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827563.104933:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827563.552840:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827563.552858:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827563.552861:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827563.552870:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9630 00000100:00080000:1.0:1787827563.552880:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827563.552885:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787827563.552917:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1500 x1874663043167232/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827563.552946:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827563.552949:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827563.552951:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827563.552954:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9630 00000100:00080000:1.0:1787827563.552956:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827563.552959:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827563.552967:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827563.552978:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1c00 x1874663043167360/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827563.552986:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787827563.552990:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043167232:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787827563.552991:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827563.552993:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827563.553012:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9630 00000100:00080000:1.0:1787827563.553015:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827563.553017:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827563.553020:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827563.553029:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3100 x1874663043167488/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787827563.553064:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827563.553104:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827563/real 1787827563] req@ffff9181c35b1500 x1874663043167232/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827588 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827563.553117:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1500 x1874663043167232/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827588 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827563.553124:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827563.553127:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827563.553140:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043167232:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827563.553158:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043167360:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827563.553176:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827563.553186:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827563/real 1787827563] req@ffff9181c35b1c00 x1874663043167360/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827588 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827563.553194:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1c00 x1874663043167360/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827588 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827563.553199:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827563.553202:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827563.553204:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043167360:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827563.553208:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043167488:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827563.553219:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827563.553225:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827563/real 1787827563] req@ffff9181c35b3100 x1874663043167488/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827588 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827563.553232:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3100 x1874663043167488/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827588 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827563.553237:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827563.553239:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827563.553240:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043167488:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787827564.128881:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827565.152940:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827566.176884:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827567.200936:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827568.224847:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827568.672947:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827568.672960:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827568.672964:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827568.672973:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9635 00000100:00080000:2.0:1787827568.672982:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827568.673000:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827568.673035:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d500 x1874663043167616/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827568.673095:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827568.673099:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827568.673101:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827568.673104:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9635 00000100:00080000:2.0:1787827568.673107:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827568.673110:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827568.673119:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827568.673132:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007df80 x1874663043167744/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827568.673142:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827568.673148:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827568.673150:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787827568.673150:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043167616:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787827568.673157:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9635 00000100:00080000:2.0:1787827568.673175:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827568.673178:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827568.673182:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827568.673191:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ed80 x1874663043167872/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787827568.673247:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827568.673322:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827568/real 1787827568] req@ffff9181e007d500 x1874663043167616/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827593 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827568.673345:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d500 x1874663043167616/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827593 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827568.673374:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827568.673380:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827568.673397:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043167616:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827568.673427:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043167744:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827568.673463:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827568.673507:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827568/real 1787827568] req@ffff9181e007df80 x1874663043167744/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827593 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827568.673526:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007df80 x1874663043167744/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827593 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827568.673539:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827568.673544:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827568.673549:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043167744:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827568.673558:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043167872:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827568.673602:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827568.673616:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827568/real 1787827568] req@ffff9181e007ed80 x1874663043167872/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827593 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827568.673631:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ed80 x1874663043167872/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827593 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827568.673642:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827568.673646:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827568.673650:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043167872:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787827569.248882:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827570.272874:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827571.296866:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827572.320866:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827573.344851:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827573.792880:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827573.792891:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827573.792895:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827573.792905:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9640 00000100:00080000:2.0:1787827573.792917:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827573.792925:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827573.792966:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ed80 x1874663043168000/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827573.792993:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827573.792997:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827573.792999:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827573.793003:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9640 00000100:00080000:2.0:1787827573.793006:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827573.793010:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827573.793019:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827573.793031:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007df80 x1874663043168128/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827573.793041:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827573.793045:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827573.793046:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827573.793050:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9640 00000100:00080000:2.0:1787827573.793053:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827573.793059:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827573.793061:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043168000:192.168.203.147@tcp:250:kworker.0 00000008:00100000:2.0:1787827573.793064:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827573.793078:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d500 x1874663043168256/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787827573.793173:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827573.793213:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827573/real 1787827573] req@ffff9181e007ed80 x1874663043168000/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827598 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827573.793231:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ed80 x1874663043168000/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827598 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827573.793262:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827573.793267:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827573.793296:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043168000:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827573.793304:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043168128:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827573.793334:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827573.793351:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827573/real 1787827573] req@ffff9181e007df80 x1874663043168128/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827598 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827573.793365:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007df80 x1874663043168128/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827598 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827573.793371:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827573.793374:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827573.793376:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043168128:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827573.793395:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043168256:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827573.793411:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827573.793419:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827573/real 1787827573] req@ffff9181e007d500 x1874663043168256/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827598 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827573.793431:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d500 x1874663043168256/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827598 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827573.793436:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827573.793438:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827573.793440:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043168256:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1787827574.368830:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827575.392911:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787827575.392948:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'params' failed (-108). 10000000:00020000:1.0:1787827575.392955:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:1.0:1787827575.392961:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:1.0:1787827575.392962:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff9181c87b19b8 from 1 10000000:01000000:1.0:1787827575.392966:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00010000:00010000:1.0:1787827575.393021:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f64f lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827575.393033:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f64f lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827575.393044:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f64f lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787827575.393074:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215d3c0 going to sleep for 0 seconds 00000100:00100000:1.0:1787827575.393086:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043168384:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787827575.393119:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f64f lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827575.393127:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f64f lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827575.393135:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f64f lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827575.393143:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f64f lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827575.393150:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f64f lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827575.393156:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4400 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787827575.393160:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f64f lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827575.393168:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f64f lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787827575.393175:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f64f lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787827575.393180:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1787827575.393183:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787827575.393191:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f64f lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827575.393198:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f64f lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787827575.393223:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787827575.393226:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:3.0:1787827575.393323:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827575.393329:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827575.393333:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827575.393339:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9645 00000100:00080000:3.0:1787827575.393348:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827575.393373:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827575.393404:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637b80 x1874663043168512/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827575.393441:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827575.393449:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827575.393453:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827575.393461:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9645 00000100:00080000:3.0:1787827575.393484:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827575.393490:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787827575.393496:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043168512:192.168.203.147@tcp:250:kworker.0 00000008:00100000:3.0:1787827575.393504:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827575.393522:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634a80 x1874663043168640/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827575.393541:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827575.393545:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827575.393547:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827575.393552:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9645 00000100:00080000:3.0:1787827575.393556:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827575.393560:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827575.393567:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000800:00000100:1.0:1787827575.393581:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827575.393583:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043168768/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827575.393622:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827575/real 1787827575] req@ffff9181c5637b80 x1874663043168512/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827600 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827575.393650:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637b80 x1874663043168512/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827600 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827575.393663:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827575.393668:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827575.393684:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043168512:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827575.393706:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043168640:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827575.393734:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827575.393811:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827575/real 1787827575] req@ffff9181c5634a80 x1874663043168640/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827600 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827575.393826:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634a80 x1874663043168640/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827600 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827575.393834:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827575.393838:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827575.393842:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043168640:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827575.393849:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043168768:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827575.393874:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827575.393887:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827575/real 1787827575] req@ffff9181c5634000 x1874663043168768/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827600 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827575.393905:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043168768/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827600 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827575.393913:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827575.393917:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827575.393921:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043168768:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1787827576.416906:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827577.440867:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827578.464838:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827579.488870:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827580.448794:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827580.448801:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827580.448804:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827580.448813:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9647 00000100:00080000:3.0:1787827580.448822:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827580.448828:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827580.448862:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043168896/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827580.448891:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827580.448899:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827580.448901:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827580.448905:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9647 00000100:00080000:3.0:1787827580.448909:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827580.448913:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827580.448921:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827580.448932:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043168896:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827580.448934:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043169024/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827580.448950:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827580.448953:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827580.448956:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827580.448961:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9647 00000100:00080000:3.0:1787827580.448965:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827580.448970:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827580.448976:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000800:00000100:1.0:1787827580.448991:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827580.448991:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043169152/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827580.449024:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827580/real 1787827580] req@ffff9181c5635c00 x1874663043168896/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827605 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827580.449038:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043168896/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827605 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827580.449046:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827580.449049:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827580.449062:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043168896:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827580.449081:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043169024:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827580.449099:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827580.449109:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827580/real 1787827580] req@ffff9181c5636d80 x1874663043169024/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827605 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827580.449117:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043169024/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827605 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827580.449127:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827580.449129:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827580.449132:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043169024:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827580.449135:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043169152:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827580.449145:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827580.449151:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827580/real 1787827580] req@ffff9181c5637800 x1874663043169152/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827605 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827580.449158:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043169152/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827605 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827580.449164:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827580.449166:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827580.449168:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043169152:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1787827580.512854:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827581.536872:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827582.560875:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827583.584873:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827584.608826:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827585.568839:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827585.568848:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827585.568851:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827585.568859:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9652 00000100:00080000:1.0:1787827585.568869:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827585.568875:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787827585.568908:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0380 x1874663043169280/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827585.568931:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827585.568935:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827585.568937:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827585.568940:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9652 00000100:00080000:1.0:1787827585.568943:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827585.568946:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827585.568953:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827585.568980:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0000 x1874663043169408/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827585.568994:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043169280:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787827585.569012:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827585.569017:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827585.569019:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827585.569025:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9652 00000100:00080000:1.0:1787827585.569045:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827585.569049:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827585.569054:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827585.569065:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1180 x1874663043169536/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787827585.569082:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827585.569127:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827585/real 1787827585] req@ffff9181c35b0380 x1874663043169280/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827610 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827585.569143:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0380 x1874663043169280/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827610 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827585.569190:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827585.569194:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827585.569206:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043169280:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827585.569226:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043169408:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827585.569249:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827585.569267:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827585/real 1787827585] req@ffff9181c35b0000 x1874663043169408/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827610 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827585.569282:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0000 x1874663043169408/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827610 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827585.569292:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827585.569296:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827585.569299:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043169408:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827585.569304:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043169536:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827585.569321:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827585.569329:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827585/real 1787827585] req@ffff9181c35b1180 x1874663043169536/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827610 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827585.569338:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1180 x1874663043169536/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827610 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827585.569344:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827585.569346:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827585.569348:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043169536:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787827585.632892:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827586.656853:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827587.680854:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827588.704861:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827589.728874:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827590.688881:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827590.688892:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827590.688897:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827590.688907:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9657 00000100:00080000:2.0:1787827590.688918:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827590.688926:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827590.688964:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d180 x1874663043169664/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827590.688992:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827590.688996:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827590.688998:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827590.689002:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9657 00000100:00080000:2.0:1787827590.689006:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827590.689010:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827590.689019:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827590.689031:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ea00 x1874663043169792/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827590.689052:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827590.689055:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827590.689057:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827590.689061:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9657 00000100:00080000:2.0:1787827590.689064:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827590.689069:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827590.689077:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827590.689089:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e300 x1874663043169920/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827590.689133:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043169664:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1787827590.689220:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827590.689270:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827590/real 1787827590] req@ffff9181e007d180 x1874663043169664/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827615 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827590.689292:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d180 x1874663043169664/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827615 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827590.689303:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827590.689307:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827590.689326:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043169664:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827590.689333:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043169792:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827590.689367:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827590.689392:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827590/real 1787827590] req@ffff9181e007ea00 x1874663043169792/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827615 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827590.689403:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ea00 x1874663043169792/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827615 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827590.689410:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827590.689414:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827590.689419:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043169792:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827590.689509:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043169920:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827590.689531:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827590.689545:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827590/real 1787827590] req@ffff9181e007e300 x1874663043169920/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827615 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827590.689557:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e300 x1874663043169920/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827615 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827590.689691:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827590.689694:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827590.689697:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043169920:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1787827590.752951:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827591.776830:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827592.800917:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827593.824849:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827594.848917:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827595.808837:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827595.808847:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827595.808851:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827595.808861:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9662 00000100:00080000:2.0:1787827595.808872:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827595.808880:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 10000000:01000000:1.0:1787827595.808897:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-sptlrpc' failed (-108). 10000000:00020000:1.0:1787827595.808904:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00100000:2.0:1787827595.808923:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c700 x1874663043170048/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827595.808954:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827595.808957:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827595.808959:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827595.808963:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9662 00000100:00080000:2.0:1787827595.808967:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827595.808970:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827595.808979:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827595.808992:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f800 x1874663043170176/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827595.809001:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827595.809007:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:3.0:1787827595.809007:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043170048:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787827595.809010:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827595.809020:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9662 00000100:00080000:2.0:1787827595.809022:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827595.809025:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827595.809029:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827595.809041:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f100 x1874663043170304/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787827595.809078:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827595.809129:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827595/real 1787827595] req@ffff9181e007c700 x1874663043170048/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827620 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827595.809147:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c700 x1874663043170048/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827620 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827595.809161:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827595.809166:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827595.809171:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043170048:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827595.809196:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043170176:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827595.809221:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827595.809232:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827595/real 1787827595] req@ffff9181e007f800 x1874663043170176/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827620 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827595.809243:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f800 x1874663043170176/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827620 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827595.809250:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827595.809252:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827595.809255:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043170176:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827595.809261:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043170304:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827595.809283:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827595.809290:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827595/real 1787827595] req@ffff9181e007f100 x1874663043170304/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827620 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827595.809299:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f100 x1874663043170304/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827620 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827595.809305:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827595.809307:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827595.809310:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043170304:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787827595.872973:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827596.896901:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787827597.024888:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:1.0:1787827597.024899:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff9181c5620000 from 341 10000000:01000000:1.0:1787827597.024906:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:1.0:1787827597.024999:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f656 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827597.025032:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f656 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827597.025049:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f656 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787827597.025099:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215d3c0 going to sleep for 0 seconds 00000100:00100000:1.0:1787827597.025128:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043170432:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787827597.025170:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f656 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827597.025180:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f656 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827597.025187:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f656 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827597.025195:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f656 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827597.025203:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f656 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827597.025210:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4400 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787827597.025215:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f656 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827597.025223:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f656 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787827597.025232:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f656 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787827597.025239:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:1.0:1787827597.025243:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787827597.025251:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f656 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827597.025258:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f656 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787827597.025298:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787827597.025302:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:2.0:1787827597.025393:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827597.025404:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827597.025408:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827597.025419:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9667 00000100:00080000:2.0:1787827597.025433:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827597.025445:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827597.025505:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f480 x1874663043170560/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827597.025543:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827597.025549:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827597.025552:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827597.025559:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9667 00000100:00080000:2.0:1787827597.025567:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827597.025576:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827597.025591:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827597.025611:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043170560:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787827597.025624:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ce00 x1874663043170688/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827597.025657:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827597.025663:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827597.025666:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827597.025674:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9667 00000100:00080000:2.0:1787827597.025692:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827597.025698:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827597.025706:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1787827597.025712:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787827597.025839:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d880 x1874663043170816/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827597.025856:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827597/real 1787827597] req@ffff9181e007f480 x1874663043170560/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827622 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827597.025897:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f480 x1874663043170560/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827622 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827597.025912:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827597.025918:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827597.025940:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043170560:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827597.025984:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043170688:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827597.026016:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827597.026032:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827597/real 1787827597] req@ffff9181e007ce00 x1874663043170688/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827622 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827597.026046:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ce00 x1874663043170688/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827622 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827597.026056:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827597.026059:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827597.026063:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043170688:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827597.026070:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043170816:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827597.026129:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827597.026143:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827597/real 1787827597] req@ffff9181e007d880 x1874663043170816/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827622 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827597.026156:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d880 x1874663043170816/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827622 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827597.026165:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827597.026169:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827597.026172:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043170816:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787827597.920934:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827598.944895:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827599.968880:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827600.992923:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827602.016967:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827602.464836:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827602.464846:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827602.464850:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827602.464860:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9668 00000100:00080000:3.0:1787827602.464873:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827602.464880:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827602.464914:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043170944/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827602.464951:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827602.464954:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827602.464955:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827602.464958:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9668 00000100:00080000:3.0:1787827602.464961:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827602.464964:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827602.464971:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827602.464979:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043171072/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827602.464986:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827602.464988:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827602.464989:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827602.464991:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9668 00000100:00080000:3.0:1787827602.464993:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827602.464995:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827602.464998:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827602.465004:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043171200/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827602.465028:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043170944:192.168.203.147@tcp:250:kworker.0 00000800:00000100:1.0:1787827602.465108:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827602.465137:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827602/real 1787827602] req@ffff9181c5637100 x1874663043170944/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827627 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827602.465149:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043170944/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827627 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827602.465157:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827602.465160:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827602.465177:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043170944:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827602.465182:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043171072:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827602.465199:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827602.465208:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827602/real 1787827602] req@ffff9181c5637480 x1874663043171072/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827627 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827602.465216:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043171072/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827627 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827602.465222:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827602.465224:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827602.465227:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043171072:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827602.465231:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043171200:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827602.465243:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827602.465254:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827602/real 1787827602] req@ffff9181c5636a00 x1874663043171200/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827627 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827602.465263:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043171200/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827627 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827602.465268:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827602.465270:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827602.465272:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043171200:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1787827603.040907:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827604.064876:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827605.088852:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827606.112928:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827607.136905:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827607.584890:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827607.584903:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827607.584908:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827607.584932:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9674 00000100:00080000:2.0:1787827607.584950:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827607.584960:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827607.585048:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007fb80 x1874663043171328/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827607.585075:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827607.585079:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827607.585080:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827607.585083:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9674 00000100:00080000:2.0:1787827607.585086:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827607.585090:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827607.585100:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827607.585111:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ca80 x1874663043171456/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827607.585118:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827607.585121:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827607.585123:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1787827607.585128:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043171328:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787827607.585129:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9674 00000100:00080000:2.0:1787827607.585135:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827607.585139:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827607.585142:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827607.585150:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e680 x1874663043171584/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827607.585205:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827607.585242:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827607/real 1787827607] req@ffff9181e007fb80 x1874663043171328/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827632 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827607.585257:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007fb80 x1874663043171328/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827632 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827607.585291:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827607.585295:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827607.585318:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043171328:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827607.585359:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043171456:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827607.585402:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827607.585420:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827607/real 1787827607] req@ffff9181e007ca80 x1874663043171456/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827632 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827607.585434:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ca80 x1874663043171456/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827632 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827607.585444:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827607.585450:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827607.585455:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043171456:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827607.585465:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043171584:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827607.585499:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827607.585518:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827607/real 1787827607] req@ffff9181e007e680 x1874663043171584/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827632 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827607.585537:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e680 x1874663043171584/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827632 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827607.585546:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827607.585550:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827607.585553:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043171584:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787827608.160964:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827609.184977:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827610.208921:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827611.232897:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827612.256895:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827612.704927:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827612.704939:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827612.704944:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827612.704956:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9679 00000100:00080000:1.0:1787827612.704988:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827612.704997:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787827612.705058:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0e00 x1874663043171712/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827612.705092:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827612.705096:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827612.705098:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827612.705103:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9679 00000100:00080000:1.0:1787827612.705106:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827612.705110:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827612.705121:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827612.705132:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3480 x1874663043171840/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827612.705142:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827612.705145:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827612.705147:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827612.705151:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9679 00000100:00080000:1.0:1787827612.705154:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827612.705157:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827612.705164:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827612.705173:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2680 x1874663043171968/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827612.705190:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043171712:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1787827612.705278:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827612.705320:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827612/real 1787827612] req@ffff9181c35b0e00 x1874663043171712/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827637 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827612.705340:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0e00 x1874663043171712/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827637 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827612.705352:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827612.705356:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827612.705376:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043171712:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827612.705384:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043171840:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827612.705421:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827612.705445:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827612/real 1787827612] req@ffff9181c35b3480 x1874663043171840/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827637 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827612.705458:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3480 x1874663043171840/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827637 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827612.705466:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827612.705469:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827612.705474:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043171840:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827612.705497:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043171968:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827612.705511:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827612.705519:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827612/real 1787827612] req@ffff9181c35b2680 x1874663043171968/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827637 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827612.705529:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2680 x1874663043171968/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827637 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827612.705535:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827612.705537:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827612.705539:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043171968:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787827613.280892:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827614.304922:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827615.329011:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827616.352916:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:0.0:1787827617.312929:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-cliir' failed (-108). 10000000:00000400:0.0:1787827617.312965:0:159735:0:(mgc_request.c:1900:mgc_process_log()) MGC192.168.203.147@tcp: IR log lustre-cliir failed, not fatal: rc = -108 10000000:01000000:0.0:1787827617.312977:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:0.0:1787827617.312982:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff9181c5620000 from 52 10000000:01000000:0.0:1787827617.312988:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:0.0:1787827617.313048:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f65d lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827617.313068:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f65d lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827617.313086:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f65d lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827617.313116:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d01a5000 going to sleep for 0 seconds 00000100:00100000:0.0:1787827617.313133:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043172096:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:0.0:1787827617.313152:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f65d lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827617.313166:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f65d lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827617.313187:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f65d lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827617.313201:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f65d lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827617.313215:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f65d lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827617.313226:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0512800 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:0.0:1787827617.313232:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f65d lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827617.313246:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f65d lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1787827617.313258:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f65d lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787827617.313268:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:0.0:1787827617.313274:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787827617.313283:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f65d lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827617.313311:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f65d lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787827617.313341:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:0.0:1787827617.313346:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:1.0:1787827617.313421:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827617.313432:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827617.313438:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827617.313448:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9684 00000100:00080000:1.0:1787827617.313462:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827617.313470:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787827617.313496:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2a00 x1874663043172224/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827617.313527:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827617.313533:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827617.313536:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827617.313543:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9684 00000100:00080000:1.0:1787827617.313549:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827617.313559:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827617.313572:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827617.313592:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043172224:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827617.313597:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0a80 x1874663043172352/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827617.313617:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827617.313622:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827617.313626:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827617.313633:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9684 00000100:00080000:1.0:1787827617.313639:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827617.313646:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827617.313654:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827617.313673:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2d80 x1874663043172480/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787827617.313687:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827617.313737:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827617/real 1787827617] req@ffff9181c35b2a00 x1874663043172224/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827642 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827617.313793:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2a00 x1874663043172224/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827642 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827617.313808:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827617.313813:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827617.313829:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043172224:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827617.313854:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043172352:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827617.313888:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827617.313909:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827617/real 1787827617] req@ffff9181c35b0a80 x1874663043172352/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827642 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827617.313923:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0a80 x1874663043172352/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827642 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827617.313960:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827617.313964:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827617.313968:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043172352:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827617.313975:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043172480:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827617.313998:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827617.314010:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827617/real 1787827617] req@ffff9181c35b2d80 x1874663043172480/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827642 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827617.314022:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2d80 x1874663043172480/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827642 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827617.314030:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827617.314033:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827617.314036:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043172480:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787827617.377029:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827618.400965:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827619.424961:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827620.448894:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827621.472930:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827622.432892:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827622.432923:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827622.432930:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827622.432961:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9689 00000100:00080000:2.0:1787827622.432972:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827622.432979:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827622.433024:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007dc00 x1874663043172608/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827622.433056:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827622.433059:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827622.433064:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827622.433068:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9689 00000100:00080000:2.0:1787827622.433070:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827622.433074:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827622.433084:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827622.433094:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c000 x1874663043172736/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827622.433101:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043172608:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787827622.433104:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827622.433110:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827622.433112:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827622.433115:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9689 00000100:00080000:2.0:1787827622.433118:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827622.433121:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827622.433124:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827622.433133:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c380 x1874663043172864/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827622.433212:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043172736:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827622.433251:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043172864:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1787827622.496954:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827623.521030:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827624.544914:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1787827625.505099:0:2379:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:1.0:1787827625.505159:0:2379:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787827625.505180:0:2379:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1787827625.505245:0:2379:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1787827625.505263:0:2379:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:3.0:1787827625.505298:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827622/real 1787827625] req@ffff9181e007c000 x1874663043172736/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827647 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827625.505344:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c000 x1874663043172736/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827647 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827625.505362:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827625.505368:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827625.505375:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043172736:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827625.505385:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827622/real 1787827625] req@ffff9181e007c380 x1874663043172864/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827647 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827625.505402:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c380 x1874663043172864/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827647 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827625.505413:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827625.505419:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827625.505424:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043172864:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827625.505434:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827622/real 1787827625] req@ffff9181e007dc00 x1874663043172608/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827647 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827625.505450:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007dc00 x1874663043172608/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827647 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827625.505460:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827625.505466:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827625.505488:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043172608:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1787827625.569026:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827626.592897:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827627.552863:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827627.552892:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827627.552897:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827627.552909:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9694 00000100:00080000:1.0:1787827627.552936:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827627.552944:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787827627.552982:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0700 x1874663043172992/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827627.553010:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827627.553014:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827627.553016:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827627.553019:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9694 00000100:00080000:1.0:1787827627.553022:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827627.553026:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827627.553035:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827627.553046:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1f80 x1874663043173120/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827627.553056:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827627.553060:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827627.553061:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827627.553065:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9694 00000100:00080000:1.0:1787827627.553068:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787827627.553071:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043172992:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787827627.553074:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827627.553084:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827627.553091:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3800 x1874663043173248/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787827627.553159:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827627.553206:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827627/real 1787827627] req@ffff9181c35b0700 x1874663043172992/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827652 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827627.553230:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0700 x1874663043172992/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827652 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827627.553249:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827627.553255:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827627.553276:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043172992:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827627.553287:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043173120:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827627.553319:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827627.553341:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827627/real 1787827627] req@ffff9181c35b1f80 x1874663043173120/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827652 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827627.553359:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1f80 x1874663043173120/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827652 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827627.553386:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827627.553391:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827627.553395:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043173120:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827627.553425:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043173248:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827627.553452:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827627.553468:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827627/real 1787827627] req@ffff9181c35b3800 x1874663043173248/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827652 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827627.553492:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3800 x1874663043173248/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827652 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827627.553501:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827627.553505:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827627.553509:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043173248:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787827627.617039:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827628.640852:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827629.664999:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827630.688978:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827631.713032:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827632.672947:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827632.672961:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827632.672966:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827632.672980:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9699 00000100:00080000:2.0:1787827632.672996:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827632.673016:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827632.673071:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007dc00 x1874663043173376/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827632.673111:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827632.673131:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827632.673133:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827632.673141:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9699 00000100:00080000:2.0:1787827632.673146:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827632.673151:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827632.673163:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827632.673172:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043173376:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787827632.673180:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c380 x1874663043173504/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827632.673192:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827632.673196:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827632.673198:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827632.673204:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9699 00000100:00080000:2.0:1787827632.673207:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827632.673210:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827632.673216:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827632.673231:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c000 x1874663043173632/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787827632.673280:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827632.673370:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827632/real 1787827632] req@ffff9181e007dc00 x1874663043173376/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827657 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827632.673398:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007dc00 x1874663043173376/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827657 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827632.673415:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827632.673422:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827632.673445:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043173376:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827632.673478:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043173504:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827632.673530:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827632.673546:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827632/real 1787827632] req@ffff9181e007c380 x1874663043173504/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827657 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827632.673563:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c380 x1874663043173504/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827657 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827632.673570:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827632.673573:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827632.673576:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043173504:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827632.673581:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043173632:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827632.673594:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827632.673601:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827632/real 1787827632] req@ffff9181e007c000 x1874663043173632/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827657 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827632.673610:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c000 x1874663043173632/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827657 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827632.673616:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827632.673618:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827632.673620:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043173632:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1787827632.737040:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827633.761063:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827634.784903:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827635.808839:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827636.832888:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827637.792871:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827637.792887:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827637.792893:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827637.792909:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9704 00000100:00080000:2.0:1787827637.792924:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827637.792935:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 10000000:01000000:0.0:1787827637.792960:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-client' failed (-108). 10000000:00020000:0.0:1787827637.792969:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:0.0:1787827637.792976:0:159735:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:0.0:1787827637.792979:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff9181c5620000 from 7 10000000:01000000:0.0:1787827637.792984:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00000100:00100000:2.0:1787827637.792992:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e680 x1874663043173760/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827637.793032:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827637.793037:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00010000:00010000:0.0:1787827637.793048:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f664 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787827637.793049:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827637.793062:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9704 00010000:00010000:0.0:1787827637.793075:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f664 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787827637.793076:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827637.793089:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:0.0:1787827637.793100:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f664 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000008:00100000:2.0:1787827637.793103:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827637.793115:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043173760:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787827637.793134:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ca80 x1874663043174016/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827637.793137:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d01a5000 going to sleep for 0 seconds 00000100:00100000:0.0:1787827637.793156:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043173888:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00080000:2.0:1787827637.793157:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827637.793171:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827637.793174:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827637.793181:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9704 00010000:00010000:0.0:1787827637.793185:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f664 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827637.793198:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f664 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787827637.793203:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827637.793215:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:0.0:1787827637.793226:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f664 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000800:00000100:3.0:1787827637.793231:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:2.0:1787827637.793234:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:0.0:1787827637.793252:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f664 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827637.793260:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007fb80 x1874663043174144/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:0.0:1787827637.793266:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f664 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787827637.793296:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827637/real 1787827637] req@ffff9181e007e680 x1874663043173760/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827662 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:0.0:1787827637.793299:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0512800 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:0.0:1787827637.793311:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f664 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827637.793321:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f664 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1787827637.793364:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f664 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787827637.793372:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e680 x1874663043173760/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827662 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 10000000:01000000:0.0:1787827637.793382:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:0.0:1787827637.793404:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00000100:00080000:3.0:1787827637.793417:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00010000:00010000:0.0:1787827637.793426:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f664 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787827637.793427:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827637.793440:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043173760:192.168.203.147@tcp:250:kworker.0 00010000:00010000:0.0:1787827637.793464:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f664 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787827637.793480:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043174016:192.168.203.147@tcp:38:kworker.0 10000000:01000000:0.0:1787827637.793509:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000800:00000100:3.0:1787827637.793524:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:0.0:1787827637.793527:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00100000:3.0:1787827637.793572:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827637/real 1787827637] req@ffff9181e007ca80 x1874663043174016/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827662 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827637.793604:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787827637.793612:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ca80 x1874663043174016/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827662 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827637.793615:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827637.793625:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827637.793652:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9709 00000100:00080000:2.0:1787827637.793665:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827637.793666:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827637.793677:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00080000:3.0:1787827637.793687:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827637.793698:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043174016:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787827637.793703:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d880 x1874663043174272/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827637.793737:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043174144:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827637.793854:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827637.793867:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827637/real 1787827637] req@ffff9181e007fb80 x1874663043174144/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827662 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827637.793878:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007fb80 x1874663043174144/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827662 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827637.793885:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827637.793888:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827637.793891:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043174144:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827637.793906:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043174272:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1787827637.793920:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827637.793933:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827637/real 1787827637] req@ffff9181e007d880 x1874663043174272/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827662 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827637.793943:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d880 x1874663043174272/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827662 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827637.793949:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827637.793951:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827637.793968:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043174272:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1787827637.856904:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827638.880861:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827639.904858:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827640.928878:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827641.952903:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827642.912912:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827642.912921:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827642.912924:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827642.912933:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9709 00000100:00080000:0.0:1787827642.912959:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827642.912965:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787827642.912998:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5c00 x1874663043174400/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827642.913041:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827642.913045:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827642.913046:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827642.913049:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9709 00000100:00080000:0.0:1787827642.913052:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827642.913055:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787827642.913062:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827642.913073:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5f80 x1874663043174528/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827642.913097:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827642.913101:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827642.913103:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827642.913107:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9709 00000100:00080000:0.0:1787827642.913109:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787827642.913110:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043174400:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787827642.913113:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787827642.913121:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827642.913129:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4700 x1874663043174656/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787827642.913172:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827642.913202:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827642/real 1787827642] req@ffff9181c55a5c00 x1874663043174400/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827667 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827642.913215:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5c00 x1874663043174400/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827667 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827642.913223:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827642.913226:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827642.913238:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043174400:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827642.913258:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043174528:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827642.913275:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827642.913286:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827642/real 1787827642] req@ffff9181c55a5f80 x1874663043174528/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827667 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827642.913300:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5f80 x1874663043174528/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827667 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827642.913309:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827642.913313:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827642.913316:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043174528:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827642.913323:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043174656:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827642.913343:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827642.913352:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827642/real 1787827642] req@ffff9181c55a4700 x1874663043174656/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827667 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827642.913360:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4700 x1874663043174656/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827667 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827642.913366:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827642.913368:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827642.913371:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043174656:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1787827642.976981:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827644.000942:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827645.025412:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827646.049497:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827647.073431:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827648.032840:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827648.032852:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827648.032856:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827648.032868:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9714 00000100:00080000:0.0:1787827648.032879:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827648.032887:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787827648.032927:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4380 x1874663043174784/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827648.032955:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827648.032958:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827648.032960:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827648.032963:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9714 00000100:00080000:0.0:1787827648.032967:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827648.032970:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787827648.032980:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827648.032989:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043174784:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787827648.032994:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5180 x1874663043174912/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827648.033005:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827648.033008:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827648.033010:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827648.033013:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9714 00000100:00080000:0.0:1787827648.033015:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827648.033018:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787827648.033022:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827648.033033:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5500 x1874663043175040/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827648.033078:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787827648.033122:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827648/real 1787827648] req@ffff9181c55a4380 x1874663043174784/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827673 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827648.033139:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4380 x1874663043174784/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827673 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827648.033148:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827648.033151:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787827648.033165:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043174784:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787827648.033208:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043174912:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787827648.033230:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787827648.033240:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827648/real 1787827648] req@ffff9181c55a5180 x1874663043174912/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827673 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827648.033249:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5180 x1874663043174912/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827673 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827648.033256:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827648.033258:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787827648.033261:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043174912:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787827648.033265:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043175040:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787827648.033277:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787827648.033283:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827648/real 1787827648] req@ffff9181c55a5500 x1874663043175040/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827673 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827648.033291:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5500 x1874663043175040/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827673 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827648.033297:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827648.033299:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787827648.033300:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043175040:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787827648.096958:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827649.120911:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827650.144971:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827651.168876:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827652.192890:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827653.152857:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827653.152867:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827653.152872:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827653.152883:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9719 00000100:00080000:0.0:1787827653.152893:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827653.152901:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787827653.152941:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7800 x1874663043175168/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827653.152967:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827653.152970:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827653.152972:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827653.152975:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9719 00000100:00080000:0.0:1787827653.152977:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827653.152980:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787827653.152990:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827653.153000:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4e00 x1874663043175296/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827653.153009:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827653.153014:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827653.153016:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827653.153022:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9719 00000100:00080000:0.0:1787827653.153027:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827653.153036:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827653.153039:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043175168:192.168.203.147@tcp:250:kworker.0 00000008:00100000:0.0:1787827653.153044:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827653.153069:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5500 x1874663043175424/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827653.153128:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787827653.153170:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827653/real 1787827653] req@ffff9181c55a7800 x1874663043175168/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827678 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827653.153191:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7800 x1874663043175168/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827678 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827653.153204:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827653.153209:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787827653.153228:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043175168:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787827653.153237:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043175296:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787827653.153270:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787827653.153285:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827653/real 1787827653] req@ffff9181c55a4e00 x1874663043175296/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827678 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827653.153300:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4e00 x1874663043175296/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827678 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827653.153310:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827653.153314:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787827653.153318:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043175296:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787827653.153345:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043175424:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787827653.153369:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787827653.153383:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827653/real 1787827653] req@ffff9181c55a5500 x1874663043175424/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827678 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827653.153396:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5500 x1874663043175424/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827678 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827653.153405:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827653.153409:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787827653.153413:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043175424:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1787827653.216899:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827654.240918:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827655.264928:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827656.288913:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827657.312867:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827658.272931:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827658.272939:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827658.272943:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827658.272952:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9724 00000100:00080000:2.0:1787827658.272968:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827658.272974:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 10000000:01000000:0.0:1787827658.272992:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'params' failed (-108). 10000000:00020000:0.0:1787827658.272999:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:0.0:1787827658.273005:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:0.0:1787827658.273009:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff9181c87b19b8 from 1 00000100:00100000:2.0:1787827658.273010:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ce00 x1874663043175552/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 10000000:01000000:0.0:1787827658.273015:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00000100:00080000:2.0:1787827658.273053:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827658.273056:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827658.273058:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827658.273061:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9724 00000100:00080000:2.0:1787827658.273064:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00010000:00010000:0.0:1787827658.273066:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f66b lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787827658.273070:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827658.273083:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:0.0:1787827658.273084:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f66b lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827658.273110:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f480 x1874663043175808/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:0.0:1787827658.273112:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f66b lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787827658.273128:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787827658.273137:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043175552:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787827658.273138:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827658.273142:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827658.273164:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9724 00000100:00100000:0.0:1787827658.273168:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d01a5000 going to sleep for 0 seconds 00000100:00080000:2.0:1787827658.273169:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827658.273177:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827658.273182:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827658.273183:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043175680:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00100000:2.0:1787827658.273196:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f100 x1874663043175936/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:0.0:1787827658.273203:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f66b lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827658.273217:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f66b lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827658.273226:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f66b lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827658.273235:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f66b lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827658.273247:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f66b lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000800:00000100:3.0:1787827658.273249:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00010000:00010000:0.0:1787827658.273262:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0512800 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:0.0:1787827658.273267:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f66b lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827658.273288:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f66b lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1787827658.273297:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f66b lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787827658.273304:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:0.0:1787827658.273311:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00000100:00100000:3.0:1787827658.273319:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827658/real 1787827658] req@ffff9181e007ce00 x1874663043175552/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827683 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:0.0:1787827658.273321:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f66b lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827658.273347:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f66b lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787827658.273350:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ce00 x1874663043175552/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827683 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827658.273400:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827658.273408:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 10000000:01000000:0.0:1787827658.273415:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00100000:3.0:1787827658.273415:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043175552:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787827658.273433:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00100000:3.0:1787827658.273468:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043175808:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827658.273494:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1787827658.273510:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827658.273517:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00100000:3.0:1787827658.273517:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827658/real 1787827658] req@ffff9181e007f480 x1874663043175808/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827683 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827658.273523:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827658.273536:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9730 00000100:00080000:2.0:1787827658.273546:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827658.273556:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827658.273562:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f480 x1874663043175808/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827683 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827658.273571:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f800 x1874663043176064/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827658.273578:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827658.273590:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827658.273596:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043175808:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827658.273606:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043175936:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827658.273629:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827658.273642:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827658/real 1787827658] req@ffff9181e007f100 x1874663043175936/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827683 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827658.273670:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f100 x1874663043175936/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827683 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827658.273679:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827658.273683:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827658.273686:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043175936:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827658.273701:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043176064:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1787827658.273720:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827658.273731:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827658/real 1787827658] req@ffff9181e007f800 x1874663043176064/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827683 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827658.273778:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f800 x1874663043176064/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827683 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827658.273790:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827658.273794:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827658.273816:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043176064:192.168.203.147@tcp:250:kworker.0 00010000:00010000:2.0:1787827658.337056:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827659.360899:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827660.384860:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827661.408927:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827662.432897:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827663.392884:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827663.392895:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827663.392899:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827663.392916:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9730 00000100:00080000:1.0:1787827663.392927:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827663.392935:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787827663.392976:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3b80 x1874663043176192/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827663.393000:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827663.393003:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827663.393005:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827663.393008:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9730 00000100:00080000:1.0:1787827663.393015:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827663.393019:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827663.393027:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827663.393038:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3800 x1874663043176320/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827663.393050:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827663.393052:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827663.393054:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827663.393057:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9730 00000100:00080000:1.0:1787827663.393060:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827663.393063:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827663.393067:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827663.393074:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1f80 x1874663043176448/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827663.393132:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043176192:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1787827663.393233:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827663.393284:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827663/real 1787827663] req@ffff9181c35b3b80 x1874663043176192/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827688 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827663.393299:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3b80 x1874663043176192/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827688 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827663.393320:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827663.393324:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827663.393351:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043176192:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827663.393372:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043176320:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827663.393417:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827663.393432:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827663/real 1787827663] req@ffff9181c35b3800 x1874663043176320/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827688 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827663.393463:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3800 x1874663043176320/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827688 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827663.393470:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827663.393473:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827663.393475:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043176320:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827663.393493:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043176448:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827663.393511:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827663.393530:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827663/real 1787827663] req@ffff9181c35b1f80 x1874663043176448/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827688 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827663.393539:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1f80 x1874663043176448/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827688 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827663.393545:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827663.393547:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827663.393561:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043176448:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787827663.456895:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827664.480938:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827665.504922:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827666.528914:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827667.552854:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827668.512867:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827668.512878:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827668.512882:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827668.512904:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9735 00000100:00080000:1.0:1787827668.512915:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827668.512923:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787827668.512962:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0700 x1874663043176576/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827668.512994:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827668.512999:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827668.513001:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827668.513006:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9735 00000100:00080000:1.0:1787827668.513011:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827668.513016:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827668.513035:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827668.513049:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043176576:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827668.513053:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2d80 x1874663043176704/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827668.513069:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827668.513073:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827668.513076:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827668.513080:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9735 00000100:00080000:1.0:1787827668.513085:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827668.513090:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827668.513095:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827668.513109:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0a80 x1874663043176832/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787827668.513120:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827668.513168:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827668/real 1787827668] req@ffff9181c35b0700 x1874663043176576/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827693 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827668.513188:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0700 x1874663043176576/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827693 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827668.513200:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827668.513205:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827668.513219:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043176576:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827668.513245:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043176704:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827668.513271:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827668.513285:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827668/real 1787827668] req@ffff9181c35b2d80 x1874663043176704/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827693 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827668.513298:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2d80 x1874663043176704/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827693 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827668.513308:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827668.513311:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827668.513315:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043176704:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827668.513322:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043176832:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827668.513343:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827668.513353:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827668/real 1787827668] req@ffff9181c35b0a80 x1874663043176832/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827693 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827668.513366:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0a80 x1874663043176832/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827693 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827668.513374:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827668.513378:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827668.513381:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043176832:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787827668.576929:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827669.600863:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827670.625417:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827671.649248:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827672.672935:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827673.632867:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827673.632878:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827673.632882:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827673.632892:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9740 00000100:00080000:0.0:1787827673.632912:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827673.632920:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787827673.632956:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5180 x1874663043176960/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827673.632987:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827673.632992:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827673.632994:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827673.632999:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9740 00000100:00080000:0.0:1787827673.633002:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827673.633007:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787827673.633016:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827673.633031:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4380 x1874663043177088/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827673.633049:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043176960:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787827673.633051:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827673.633059:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827673.633061:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827673.633066:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9740 00000100:00080000:0.0:1787827673.633069:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827673.633073:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787827673.633078:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827673.633091:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4700 x1874663043177216/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787827673.633110:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827673.633149:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827673/real 1787827673] req@ffff9181c55a5180 x1874663043176960/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827698 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827673.633169:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5180 x1874663043176960/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827698 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827673.633181:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827673.633184:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827673.633195:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043176960:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827673.633216:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043177088:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827673.633237:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827673.633248:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827673/real 1787827673] req@ffff9181c55a4380 x1874663043177088/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827698 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827673.633256:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4380 x1874663043177088/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827698 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827673.633272:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827673.633276:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827673.633280:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043177088:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827673.633286:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043177216:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827673.633307:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827673.633321:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827673/real 1787827673] req@ffff9181c55a4700 x1874663043177216/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827698 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827673.633333:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4700 x1874663043177216/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827698 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827673.633341:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827673.633345:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827673.633348:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043177216:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827673.696916:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827674.720938:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827675.744853:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827676.768872:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827677.792871:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827678.752835:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827678.752843:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827678.752846:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827678.752855:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9745 00000100:00080000:0.0:1787827678.752864:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 10000000:01000000:1.0:1787827678.752869:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-sptlrpc' failed (-108). 00000100:00080000:0.0:1787827678.752872:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 10000000:00020000:1.0:1787827678.752877:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00100000:0.0:1787827678.752905:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5f80 x1874663043177344/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827678.752927:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827678.752930:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827678.752931:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827678.752934:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9745 00000100:00080000:0.0:1787827678.752936:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827678.752939:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787827678.752947:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827678.752962:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5c00 x1874663043177472/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827678.752970:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043177344:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787827678.752973:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827678.752979:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827678.752981:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827678.752984:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9745 00000100:00080000:0.0:1787827678.752987:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827678.752989:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787827678.752993:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827678.753001:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5880 x1874663043177600/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787827678.753032:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827678.753060:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827678/real 1787827678] req@ffff9181c55a5f80 x1874663043177344/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827703 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827678.753079:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5f80 x1874663043177344/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827703 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827678.753086:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827678.753089:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827678.753092:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043177344:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827678.753108:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043177472:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827678.753123:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827678.753130:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827678/real 1787827678] req@ffff9181c55a5c00 x1874663043177472/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827703 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827678.753138:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5c00 x1874663043177472/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827703 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827678.753144:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827678.753146:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827678.753148:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043177472:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827678.753151:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043177600:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827678.753161:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827678.753167:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827678/real 1787827678] req@ffff9181c55a5880 x1874663043177600/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827703 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827678.753174:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5880 x1874663043177600/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827703 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827678.753183:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827678.753186:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827678.753188:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043177600:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787827678.816968:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827679.840869:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787827680.544844:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:1.0:1787827680.544854:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff9181c5620000 from 341 10000000:01000000:1.0:1787827680.544863:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:1.0:1787827680.544928:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f672 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827680.544945:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f672 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827680.544960:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f672 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787827680.544985:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215d3c0 going to sleep for 0 seconds 00000100:00100000:1.0:1787827680.545000:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043177728:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787827680.545021:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f672 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827680.545031:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f672 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827680.545041:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f672 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827680.545050:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f672 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827680.545059:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f672 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827680.545067:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4c00 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787827680.545072:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f672 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827680.545083:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f672 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787827680.545092:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f672 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787827680.545100:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:1.0:1787827680.545104:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787827680.545112:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f672 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827680.545120:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f672 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787827680.545152:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787827680.545157:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:0.0:1787827680.545256:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827680.545269:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827680.545276:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827680.545288:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9750 00000100:00080000:0.0:1787827680.545303:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827680.545314:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787827680.545352:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6d80 x1874663043177856/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827680.545389:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827680.545395:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827680.545397:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827680.545402:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9750 00000100:00080000:0.0:1787827680.545408:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827680.545414:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787827680.545427:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827680.545446:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7480 x1874663043177984/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827680.545461:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827680.545466:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827680.545468:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827680.545473:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9750 00000100:00080000:0.0:1787827680.545478:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827680.545489:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827680.545491:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043177856:192.168.203.147@tcp:250:kworker.0 00000008:00100000:0.0:1787827680.545521:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827680.545550:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7b80 x1874663043178112/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787827680.545609:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827680.545655:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827680/real 1787827680] req@ffff9181c55a6d80 x1874663043177856/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827705 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827680.545672:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6d80 x1874663043177856/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827705 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827680.545682:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827680.545686:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827680.545703:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043177856:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827680.545710:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043177984:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827680.545729:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827680.545794:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827680/real 1787827680] req@ffff9181c55a7480 x1874663043177984/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827705 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827680.545806:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7480 x1874663043177984/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827705 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827680.545813:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827680.545816:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827680.545819:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043177984:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827680.545845:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043178112:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827680.545862:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827680.545873:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827680/real 1787827680] req@ffff9181c55a7b80 x1874663043178112/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827705 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827680.545882:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7b80 x1874663043178112/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827705 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827680.545888:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827680.545891:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827680.545893:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043178112:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787827680.865064:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827681.889229:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827682.912935:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827683.937076:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827684.961381:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827685.920839:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827685.920847:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827685.920850:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827685.920861:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9752 00000100:00080000:2.0:1787827685.920872:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827685.920880:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827685.920917:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c700 x1874663043178240/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827685.920944:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827685.920947:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827685.920948:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827685.920951:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9752 00000100:00080000:2.0:1787827685.920954:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827685.920957:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827685.920965:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827685.920976:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e300 x1874663043178368/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827685.920989:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827685.920992:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827685.920994:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827685.920999:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9752 00000100:00080000:2.0:1787827685.921002:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827685.921006:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827685.921009:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827685.921017:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ea00 x1874663043178496/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827685.921023:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043178240:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827685.921135:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043178368:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827685.921168:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043178496:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827685.984871:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827687.009040:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827688.033009:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1787827688.992994:0:2376:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:1.0:1787827688.993035:0:2376:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787827688.993054:0:2376:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1787827688.993094:0:2376:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1787827688.993106:0:2376:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:3.0:1787827688.993158:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827685/real 1787827688] req@ffff9181e007ea00 x1874663043178496/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827710 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827688.993185:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ea00 x1874663043178496/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827710 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827688.993206:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827688.993212:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827688.993216:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043178496:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827688.993234:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827685/real 1787827688] req@ffff9181e007c700 x1874663043178240/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827710 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827688.993250:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c700 x1874663043178240/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827710 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827688.993270:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827688.993274:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827688.993295:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043178240:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827688.993303:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827685/real 1787827688] req@ffff9181e007e300 x1874663043178368/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827710 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827688.993315:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e300 x1874663043178368/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827710 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827688.993325:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827688.993329:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827688.993343:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043178368:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787827689.056927:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827690.081056:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827691.040812:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827691.040821:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827691.040831:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827691.040842:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9757 00000100:00080000:2.0:1787827691.040854:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827691.040864:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827691.040899:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d180 x1874663043178624/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827691.040925:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827691.040928:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827691.040930:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827691.040933:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9757 00000100:00080000:2.0:1787827691.040937:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827691.040942:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827691.040951:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827691.040968:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d500 x1874663043178752/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827691.040977:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043178624:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787827691.040994:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827691.041000:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827691.041002:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827691.041008:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9757 00000100:00080000:2.0:1787827691.041014:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827691.041018:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827691.041024:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827691.041042:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007df80 x1874663043178880/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827691.041073:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827691.041122:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827691/real 1787827691] req@ffff9181e007d180 x1874663043178624/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827716 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827691.041143:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d180 x1874663043178624/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827716 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827691.041154:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827691.041158:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827691.041179:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043178624:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827691.041205:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043178752:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827691.041222:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827691.041231:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827691/real 1787827691] req@ffff9181e007d500 x1874663043178752/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827716 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827691.041243:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d500 x1874663043178752/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827716 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827691.041250:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827691.041253:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827691.041256:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043178752:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827691.041262:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043178880:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827691.041283:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827691.041295:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827691/real 1787827691] req@ffff9181e007df80 x1874663043178880/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827716 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827691.041307:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007df80 x1874663043178880/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827716 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827691.041323:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827691.041327:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827691.041331:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043178880:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787827691.104952:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827692.129019:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827693.152989:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827694.177025:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827695.200981:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827696.160849:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827696.160860:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827696.160864:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827696.160874:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9762 00000100:00080000:2.0:1787827696.160886:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827696.160894:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827696.160941:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ed80 x1874663043179008/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827696.160983:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827696.160990:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827696.160994:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827696.161001:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9762 00000100:00080000:2.0:1787827696.161006:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827696.161013:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827696.161041:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827696.161061:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043179008:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787827696.161074:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007df80 x1874663043179136/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827696.161098:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827696.161104:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827696.161108:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827696.161115:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9762 00000100:00080000:2.0:1787827696.161120:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827696.161125:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827696.161133:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827696.161147:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d500 x1874663043179264/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827696.161187:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827696.161253:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827696/real 1787827696] req@ffff9181e007ed80 x1874663043179008/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827721 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827696.161271:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ed80 x1874663043179008/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827721 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827696.161309:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827696.161313:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827696.161331:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043179008:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827696.161372:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043179136:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827696.161418:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827696.161451:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827696/real 1787827696] req@ffff9181e007df80 x1874663043179136/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827721 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827696.161462:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007df80 x1874663043179136/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827721 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827696.161470:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827696.161473:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827696.161476:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043179136:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827696.161482:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043179264:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827696.161497:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827696.161504:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827696/real 1787827696] req@ffff9181e007d500 x1874663043179264/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827721 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827696.161514:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d500 x1874663043179264/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827721 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827696.161520:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827696.161522:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827696.161524:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043179264:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827696.224920:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827697.249008:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827698.273591:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827699.297399:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827700.320933:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:0.0:1787827700.768915:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-cliir' failed (-108). 10000000:00000400:0.0:1787827700.768928:0:159735:0:(mgc_request.c:1900:mgc_process_log()) MGC192.168.203.147@tcp: IR log lustre-cliir failed, not fatal: rc = -108 10000000:01000000:0.0:1787827700.768939:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:0.0:1787827700.768941:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff9181c5620000 from 52 10000000:01000000:0.0:1787827700.768945:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:0.0:1787827700.769007:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f679 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827700.769018:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f679 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827700.769029:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f679 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827700.769048:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d01a5000 going to sleep for 0 seconds 00000100:00100000:0.0:1787827700.769059:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043179392:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:0.0:1787827700.769071:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f679 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827700.769078:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f679 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827700.769084:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f679 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827700.769090:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f679 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827700.769096:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f679 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827700.769101:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0512800 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:0.0:1787827700.769105:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f679 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827700.769111:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f679 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1787827700.769117:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f679 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787827700.769121:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:0.0:1787827700.769124:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787827700.769130:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f679 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827700.769136:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f679 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787827700.769175:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:0.0:1787827700.769179:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:3.0:1787827700.769285:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827700.769291:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827700.769294:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827700.769300:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9767 00000100:00080000:3.0:1787827700.769309:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827700.769315:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827700.769336:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043179520/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827700.769372:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827700.769376:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827700.769378:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827700.769382:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9767 00000100:00080000:3.0:1787827700.769385:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827700.769389:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827700.769398:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827700.769412:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634e00 x1874663043179648/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827700.769421:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043179520:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787827700.769436:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827700.769443:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827700.769446:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827700.769453:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9767 00000100:00080000:3.0:1787827700.769458:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827700.769464:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827700.769471:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827700.769488:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043179776/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827700.769493:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827700.769551:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827700/real 1787827700] req@ffff9181c5634380 x1874663043179520/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827725 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827700.769569:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043179520/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827725 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827700.769578:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827700.769582:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827700.769619:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043179520:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827700.769643:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043179648:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827700.769688:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827700.769709:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827700/real 1787827700] req@ffff9181c5634e00 x1874663043179648/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827725 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827700.769720:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634e00 x1874663043179648/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827725 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827700.769727:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827700.769748:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827700.769752:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043179648:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827700.769793:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043179776:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827700.769845:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827700.769867:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827700/real 1787827700] req@ffff9181c5635500 x1874663043179776/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827725 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827700.769884:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043179776/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827725 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827700.769896:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827700.769900:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827700.769905:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043179776:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787827701.344921:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827702.368914:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827703.392851:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827704.416955:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827705.441275:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827705.888933:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827705.888947:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827705.888953:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827705.888966:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9772 00000100:00080000:1.0:1787827705.888980:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827705.888990:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787827705.889051:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2a00 x1874663043179904/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827705.889084:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827705.889088:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827705.889091:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827705.889095:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9772 00000100:00080000:1.0:1787827705.889099:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827705.889103:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827705.889115:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827705.889128:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2680 x1874663043180032/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827705.889140:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827705.889148:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:3.0:1787827705.889148:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043179904:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787827705.889154:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827705.889165:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9772 00000100:00080000:1.0:1787827705.889168:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827705.889172:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827705.889177:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827705.889191:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3480 x1874663043180160/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787827705.889251:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827705.889310:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827705/real 1787827705] req@ffff9181c35b2a00 x1874663043179904/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827730 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827705.889364:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2a00 x1874663043179904/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827730 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827705.889377:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827705.889383:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827705.889400:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043179904:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827705.889431:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043180032:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827705.889469:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827705.889487:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827705/real 1787827705] req@ffff9181c35b2680 x1874663043180032/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827730 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827705.889500:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2680 x1874663043180032/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827730 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827705.889509:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827705.889512:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827705.889515:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043180032:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827705.889520:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043180160:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827705.889542:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827705.889550:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827705/real 1787827705] req@ffff9181c35b3480 x1874663043180160/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827730 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827705.889562:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3480 x1874663043180160/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827730 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827705.889569:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827705.889572:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827705.889574:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043180160:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827706.464882:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827707.488958:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827708.512899:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827709.536879:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827710.560932:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827711.008909:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827711.008920:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827711.008924:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827711.008935:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9777 00000100:00080000:3.0:1787827711.008947:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827711.008955:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827711.008996:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635f80 x1874663043180288/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827711.009031:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827711.009035:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827711.009037:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827711.009042:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9777 00000100:00080000:3.0:1787827711.009058:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827711.009063:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827711.009076:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827711.009127:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043180416/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827711.009142:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043180288:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787827711.009153:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827711.009161:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827711.009162:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827711.009186:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9777 00000100:00080000:3.0:1787827711.009191:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827711.009197:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827711.009203:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827711.009219:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634e00 x1874663043180544/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827711.009260:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827711.009363:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827711/real 1787827711] req@ffff9181c5635f80 x1874663043180288/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827736 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827711.009383:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635f80 x1874663043180288/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827736 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827711.009393:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827711.009397:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827711.009423:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043180288:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827711.009453:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043180416:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827711.009479:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827711.009500:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827711/real 1787827711] req@ffff9181c5635500 x1874663043180416/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827736 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827711.009510:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043180416/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827736 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827711.009525:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827711.009528:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827711.009531:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043180416:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827711.009542:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043180544:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827711.009556:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827711.009574:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827711/real 1787827711] req@ffff9181c5634e00 x1874663043180544/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827736 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827711.009582:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634e00 x1874663043180544/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827736 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827711.009587:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827711.009590:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827711.009591:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043180544:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787827711.585241:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827712.609263:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827713.632963:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827714.656952:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827715.680904:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827716.128888:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827716.128900:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827716.128906:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827716.128919:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9782 00000100:00080000:0.0:1787827716.128933:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827716.128942:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787827716.129000:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7100 x1874663043180672/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827716.129030:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827716.129034:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827716.129036:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827716.129039:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9782 00000100:00080000:0.0:1787827716.129043:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827716.129047:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787827716.129057:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827716.129068:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6680 x1874663043180800/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827716.129083:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827716.129087:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827716.129089:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827716.129093:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9782 00000100:00080000:0.0:1787827716.129098:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827716.129102:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787827716.129105:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827716.129116:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7b80 x1874663043180928/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827716.129136:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043180672:192.168.203.147@tcp:250:kworker.0 00000800:00000100:1.0:1787827716.129221:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827716.129278:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827716/real 1787827716] req@ffff9181c55a7100 x1874663043180672/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827741 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827716.129294:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7100 x1874663043180672/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827741 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827716.129305:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827716.129309:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827716.129323:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043180672:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827716.129330:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043180800:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827716.129352:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827716.129373:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827716/real 1787827716] req@ffff9181c55a6680 x1874663043180800/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827741 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827716.129383:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6680 x1874663043180800/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827741 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827716.129389:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827716.129392:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827716.129394:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043180800:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827716.129417:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043180928:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827716.129429:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827716.129436:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827716/real 1787827716] req@ffff9181c55a7b80 x1874663043180928/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827741 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827716.129443:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7b80 x1874663043180928/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827741 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827716.129449:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827716.129451:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827716.129453:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043180928:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827716.704881:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827717.729312:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827718.753035:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827719.776918:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827720.800912:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827721.248850:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827721.248857:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827721.248860:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827721.248869:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9787 10000000:01000000:0.0:1787827721.248876:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-client' failed (-108). 00000100:00080000:3.0:1787827721.248879:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 10000000:00020000:0.0:1787827721.248884:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00080000:3.0:1787827721.248888:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827721.248932:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043181056/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827721.248952:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827721.248955:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827721.248956:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827721.248960:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9787 00000100:00080000:3.0:1787827721.248965:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827721.248969:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827721.248977:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827721.248986:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043181184/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827721.248994:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827721.248996:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827721.248999:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1787827721.249001:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043181056:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787827721.249003:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9787 00000100:00080000:3.0:1787827721.249015:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827721.249021:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827721.249026:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827721.249032:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043181312/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827721.249084:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827721.249125:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827721/real 1787827721] req@ffff9181c5634380 x1874663043181056/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827746 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827721.249139:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043181056/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827746 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827721.249147:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827721.249150:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827721.249153:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043181056:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827721.249180:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043181184:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827721.249196:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827721.249204:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827721/real 1787827721] req@ffff9181c5636a00 x1874663043181184/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827746 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827721.249212:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043181184/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827746 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827721.249244:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827721.249247:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827721.249267:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043181184:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827721.249273:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043181312:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827721.249295:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827721.249306:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827721/real 1787827721] req@ffff9181c5637480 x1874663043181312/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827746 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827721.249318:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043181312/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827746 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827721.249332:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827721.249335:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827721.249338:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043181312:192.168.203.147@tcp:38:kworker.0 10000000:01000000:0.0:1787827721.252971:0:159735:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:0.0:1787827721.252974:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff9181c5620000 from 7 10000000:01000000:0.0:1787827721.252977:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00010000:00010000:0.0:1787827721.253009:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f680 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827721.253020:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f680 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827721.253032:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f680 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827721.253068:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d01a5000 going to sleep for 0 seconds 00000100:00100000:0.0:1787827721.253078:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043181440:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:0.0:1787827721.253091:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f680 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827721.253100:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f680 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827721.253107:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f680 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827721.253115:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f680 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827721.253122:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f680 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827721.253128:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0512800 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:0.0:1787827721.253131:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f680 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827721.253139:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f680 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1787827721.253146:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f680 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787827721.253152:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:0.0:1787827721.253155:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787827721.253161:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f680 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827721.253182:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f680 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787827721.253198:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:0.0:1787827721.253199:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:3.0:1787827721.253294:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827721.253300:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827721.253303:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827721.253308:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9792 00000100:00080000:3.0:1787827721.253313:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827721.253333:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827721.253346:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043181568/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827721.253408:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043181568:192.168.203.147@tcp:250:kworker.0 00000800:00000100:1.0:1787827721.253428:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827721.253440:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827721/real 1787827721] req@ffff9181c5637100 x1874663043181568/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827746 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827721.253448:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043181568/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827746 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827721.253454:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827721.253456:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827721.253467:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043181568:192.168.203.147@tcp:250:kworker.0 00010000:00010000:2.0:1787827721.824893:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827722.848989:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827723.873337:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827724.896914:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827725.921666:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827726.368860:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827726.368873:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827726.368878:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827726.368892:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9792 00000100:00080000:2.0:1787827726.368906:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827726.368915:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827726.368964:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d180 x1874663043181696/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827726.368998:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827726.369003:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827726.369005:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827726.369010:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9792 00000100:00080000:2.0:1787827726.369014:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827726.369020:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827726.369038:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827726.369055:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e300 x1874663043181824/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827726.369066:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043181696:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787827726.369077:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827726.369084:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827726.369086:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827726.369094:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9792 00000100:00080000:2.0:1787827726.369099:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827726.369105:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827726.369111:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827726.369125:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c700 x1874663043181952/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827726.369152:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827726.369210:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827726/real 1787827726] req@ffff9181e007d180 x1874663043181696/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827751 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827726.369231:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d180 x1874663043181696/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827751 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827726.369240:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827726.369244:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827726.369257:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043181696:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827726.369284:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043181824:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827726.369320:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827726.369335:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827726/real 1787827726] req@ffff9181e007e300 x1874663043181824/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827751 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827726.369345:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e300 x1874663043181824/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827751 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827726.369351:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827726.369354:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827726.369356:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043181824:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827726.369361:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043181952:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827726.369377:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827726.369384:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827726/real 1787827726] req@ffff9181e007c700 x1874663043181952/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827751 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827726.369392:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c700 x1874663043181952/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827751 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827726.369397:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827726.369399:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827726.369401:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043181952:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827726.945409:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827727.968877:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827728.992989:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827730.016934:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827731.040901:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827731.488899:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827731.488910:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827731.488914:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827731.488937:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9798 00000100:00080000:3.0:1787827731.488949:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827731.488957:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827731.489018:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043182080/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827731.489047:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827731.489051:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827731.489052:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827731.489056:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9798 00000100:00080000:3.0:1787827731.489058:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827731.489061:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827731.489071:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827731.489082:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043182208/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827731.489100:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827731.489104:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827731.489105:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827731.489108:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9798 00000100:00080000:3.0:1787827731.489110:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827731.489112:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827731.489115:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827731.489123:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043182336/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827731.489178:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043182080:192.168.203.147@tcp:250:kworker.0 00000800:00000100:1.0:1787827731.489258:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827731.489315:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827731/real 1787827731] req@ffff9181c5637800 x1874663043182080/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827756 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827731.489331:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043182080/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827756 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827731.489341:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827731.489344:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827731.489359:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043182080:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827731.489365:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043182208:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827731.489384:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827731.489397:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827731/real 1787827731] req@ffff9181c5636d80 x1874663043182208/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827756 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827731.489414:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043182208/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827756 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827731.489423:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827731.489427:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827731.489446:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043182208:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827731.489474:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043182336:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827731.489494:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827731.489506:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827731/real 1787827731] req@ffff9181c5635c00 x1874663043182336/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827756 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827731.489527:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043182336/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827756 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827731.489536:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827731.489539:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827731.489542:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043182336:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787827732.064956:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827733.088913:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827734.112937:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827735.136907:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827736.161256:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827736.609262:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827736.609288:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827736.609294:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827736.609309:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9803 00000100:00080000:2.0:1787827736.609327:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827736.609339:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827736.609396:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ea00 x1874663043182464/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827736.609433:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827736.609438:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827736.609440:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827736.609445:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9803 00000100:00080000:2.0:1787827736.609449:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827736.609454:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827736.609466:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827736.609480:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f800 x1874663043182592/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827736.609491:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827736.609495:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827736.609497:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827736.609501:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9803 00000100:00080000:2.0:1787827736.609505:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827736.609508:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827736.609514:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827736.609525:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f100 x1874663043182720/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827736.609608:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043182464:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1787827736.609680:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827736.609712:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827736/real 1787827736] req@ffff9181e007ea00 x1874663043182464/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827761 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827736.609727:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ea00 x1874663043182464/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827761 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827736.609736:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827736.609739:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827736.609789:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043182464:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827736.609797:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043182592:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827736.609820:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827736.609841:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827736/real 1787827736] req@ffff9181e007f800 x1874663043182592/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827761 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827736.609853:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f800 x1874663043182592/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827761 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827736.609860:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827736.609863:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827736.609868:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043182592:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827736.609877:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043182720:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827736.609898:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827736.609911:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827736/real 1787827736] req@ffff9181e007f100 x1874663043182720/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827761 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827736.609923:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f100 x1874663043182720/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827761 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827736.609931:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827736.609935:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827736.609939:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043182720:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787827737.185374:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827738.208882:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827739.234045:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827740.257327:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827741.280995:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:0.0:1787827741.728957:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'params' failed (-108). 10000000:00020000:0.0:1787827741.728972:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:0.0:1787827741.728983:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:0.0:1787827741.728986:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff9181c87b19b8 from 1 10000000:01000000:0.0:1787827741.728998:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00000100:00080000:2.0:1787827741.729057:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827741.729086:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827741.729092:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00010000:00010000:0.0:1787827741.729097:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f687 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787827741.729107:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9808 00000100:00080000:2.0:1787827741.729131:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00010000:00010000:0.0:1787827741.729137:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f687 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787827741.729153:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:0.0:1787827741.729162:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f687 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827741.729198:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d01a5000 going to sleep for 0 seconds 00000100:00100000:2.0:1787827741.729209:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f480 x1874663043182976/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827741.729217:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043182848:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:0.0:1787827741.729255:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f687 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827741.729270:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f687 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787827741.729271:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827741.729284:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827741.729290:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00010000:00010000:0.0:1787827741.729295:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f687 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827741.729317:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f687 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787827741.729326:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9808 00000100:00080000:2.0:1787827741.729343:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787827741.729345:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043182976:192.168.203.147@tcp:250:kworker.0 00010000:00010000:0.0:1787827741.729346:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f687 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787827741.729359:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:0.0:1787827741.729382:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0512800 already unused, calling callback (ffffffffc1273c20) 00000008:00100000:2.0:1787827741.729386:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 10000000:00010000:0.0:1787827741.729390:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f687 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827741.729406:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f687 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1787827741.729416:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f687 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827741.729423:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ce00 x1874663043183104/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 10000000:01000000:0.0:1787827741.729442:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:0.0:1787827741.729452:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787827741.729462:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f687 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000800:00000100:3.0:1787827741.729471:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00010000:00010000:0.0:1787827741.729476:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f687 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787827741.729476:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827741.729496:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827741.729502:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827741.729512:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9808 00000100:00080000:2.0:1787827741.729520:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827741.729527:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 10000000:01000000:0.0:1787827741.729530:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000008:00100000:2.0:1787827741.729536:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00080000:0.0:1787827741.729539:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:CONNECTING 00000100:00100000:3.0:1787827741.729539:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827741/real 1787827741] req@ffff9181e007f480 x1874663043182976/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827766 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827741.729568:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d880 x1874663043183232/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827741.729577:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f480 x1874663043182976/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827766 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827741.729612:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827741.729648:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827741.729672:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043182976:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787827741.729674:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827741.729680:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827741.729682:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827741.729691:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9813 00000100:00080000:2.0:1787827741.729697:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827741.729702:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827741.729724:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007fb80 x1874663043183360/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827741.729729:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043183104:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827741.729816:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827741.729828:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827741/real 1787827741] req@ffff9181e007ce00 x1874663043183104/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827766 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827741.729843:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ce00 x1874663043183104/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827766 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827741.729850:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827741.729853:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827741.729855:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043183104:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827741.729860:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043183232:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827741.729874:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827741.729881:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827741/real 1787827741] req@ffff9181e007d880 x1874663043183232/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827766 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827741.729889:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d880 x1874663043183232/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827766 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827741.729894:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827741.729896:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827741.729898:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043183232:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827741.729909:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043183360:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1787827741.729919:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827741.729929:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827741/real 1787827741] req@ffff9181e007fb80 x1874663043183360/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827766 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827741.729937:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007fb80 x1874663043183360/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827766 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827741.729946:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827741.729949:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827741.729958:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043183360:192.168.203.147@tcp:250:kworker.0 00010000:00010000:0.0:1787827742.304897:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827743.328874:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827744.352959:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827745.376913:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827746.400900:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827746.848904:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827746.848918:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827746.848923:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827746.848936:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9813 00000100:00080000:3.0:1787827746.848950:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827746.848960:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827746.849011:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043183488/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827746.849064:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827746.849069:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827746.849071:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827746.849077:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9813 00000100:00080000:3.0:1787827746.849081:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827746.849087:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827746.849109:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827746.849129:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634a80 x1874663043183616/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827746.849131:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043183488:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787827746.849152:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827746.849159:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827746.849161:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827746.849165:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9813 00000100:00080000:3.0:1787827746.849170:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827746.849175:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827746.849191:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827746.849204:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637b80 x1874663043183744/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827746.849228:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827746.849287:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827746/real 1787827746] req@ffff9181c5634000 x1874663043183488/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827771 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827746.849308:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043183488/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827771 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827746.849321:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827746.849326:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827746.849342:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043183488:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827746.849372:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043183616:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827746.849401:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827746.849415:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827746/real 1787827746] req@ffff9181c5634a80 x1874663043183616/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827771 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827746.849429:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634a80 x1874663043183616/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827771 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827746.849446:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827746.849450:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827746.849454:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043183616:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827746.849461:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043183744:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827746.849480:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827746.849491:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827746/real 1787827746] req@ffff9181c5637b80 x1874663043183744/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827771 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827746.849504:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637b80 x1874663043183744/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827771 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827746.849514:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827746.849518:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827746.849521:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043183744:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827747.424963:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827748.448900:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827749.472924:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827750.496890:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827751.520860:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827751.968886:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827751.968895:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827751.968898:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827751.968907:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9818 00000100:00080000:1.0:1787827751.968916:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827751.968922:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787827751.968955:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0e00 x1874663043183872/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827751.968979:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827751.968982:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827751.968983:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827751.968987:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9818 00000100:00080000:1.0:1787827751.968989:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827751.968992:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827751.969000:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827751.969025:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1180 x1874663043184000/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827751.969025:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043183872:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787827751.969040:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827751.969043:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827751.969044:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827751.969047:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9818 00000100:00080000:1.0:1787827751.969050:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827751.969053:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827751.969056:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827751.969065:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0000 x1874663043184128/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827751.969118:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043184000:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827751.969141:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043184128:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1787827752.544835:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827753.568886:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827754.592892:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:3.0:1787827755.041131:0:2377:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:3.0:1787827755.041178:0:2377:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1787827755.041196:0:2377:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:3.0:1787827755.041243:0:2377:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:3.0:1787827755.041253:0:2377:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:1.0:1787827755.041308:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827751/real 1787827755] req@ffff9181c35b1180 x1874663043184000/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827776 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827755.041337:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1180 x1874663043184000/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827776 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827755.041353:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827755.041359:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827755.041365:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043184000:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827755.041374:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827751/real 1787827755] req@ffff9181c35b0000 x1874663043184128/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827776 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827755.041388:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0000 x1874663043184128/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827776 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827755.041397:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827755.041402:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827755.041407:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043184128:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827755.041414:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827751/real 1787827755] req@ffff9181c35b0e00 x1874663043183872/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827776 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827755.041428:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0e00 x1874663043183872/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827776 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827755.041436:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827755.041440:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827755.041458:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043183872:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787827755.616880:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827756.640901:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827757.088885:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827757.088909:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827757.088913:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827757.088924:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9823 00000100:00080000:2.0:1787827757.088936:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827757.088944:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827757.088995:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ca80 x1874663043184256/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827757.089051:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827757.089056:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827757.089058:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827757.089063:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9823 00000100:00080000:2.0:1787827757.089069:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827757.089074:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787827757.089085:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043184256:192.168.203.147@tcp:250:kworker.0 00000008:00100000:2.0:1787827757.089086:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827757.089106:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e680 x1874663043184384/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827757.089120:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827757.089140:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827757.089142:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827757.089147:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9823 00000100:00080000:2.0:1787827757.089152:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827757.089156:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827757.089162:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000800:00000100:1.0:1787827757.089173:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787827757.089178:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c000 x1874663043184512/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827757.089235:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827757/real 1787827757] req@ffff9181e007ca80 x1874663043184256/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827782 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827757.089257:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ca80 x1874663043184256/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827782 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827757.089266:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827757.089270:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827757.089289:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043184256:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827757.089316:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043184384:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827757.089347:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827757.089379:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827757/real 1787827757] req@ffff9181e007e680 x1874663043184384/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827782 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827757.089392:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e680 x1874663043184384/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827782 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827757.089401:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827757.089405:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827757.089408:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043184384:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827757.089414:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043184512:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827757.089435:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827757.089447:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827757/real 1787827757] req@ffff9181e007c000 x1874663043184512/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827782 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827757.089460:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c000 x1874663043184512/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827782 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827757.089475:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827757.089484:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827757.089488:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043184512:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827757.664865:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827758.688895:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827759.713487:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827760.736992:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827761.760897:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:0.0:1787827762.208821:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-sptlrpc' failed (-108). 10000000:00020000:0.0:1787827762.208830:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00080000:3.0:1787827762.208851:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827762.208857:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827762.208860:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827762.208868:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9828 00000100:00080000:3.0:1787827762.208878:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827762.208883:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827762.208913:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637b80 x1874663043184640/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827762.208949:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827762.208952:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827762.208953:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827762.208957:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9828 00000100:00080000:3.0:1787827762.208960:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827762.208963:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827762.208981:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827762.208993:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634a80 x1874663043184768/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827762.208998:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043184640:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787827762.209007:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827762.209010:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827762.209011:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827762.209015:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9828 00000100:00080000:3.0:1787827762.209016:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827762.209019:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827762.209022:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827762.209030:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043184896/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827762.209085:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827762.209115:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827762/real 1787827762] req@ffff9181c5637b80 x1874663043184640/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827787 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827762.209129:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637b80 x1874663043184640/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827787 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827762.209137:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827762.209139:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827762.209142:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043184640:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827762.209159:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043184768:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827762.209175:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827762.209183:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827762/real 1787827762] req@ffff9181c5634a80 x1874663043184768/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827787 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827762.209191:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634a80 x1874663043184768/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827787 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827762.209197:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827762.209199:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827762.209202:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043184768:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827762.209205:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043184896:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827762.209215:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827762.209220:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827762/real 1787827762] req@ffff9181c5634000 x1874663043184896/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827787 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827762.209227:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043184896/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827787 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827762.209233:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827762.209234:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827762.209236:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043184896:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827762.784917:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:0.0:1787827763.296843:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:0.0:1787827763.296850:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff9181c5620000 from 341 10000000:01000000:0.0:1787827763.296855:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:0.0:1787827763.296905:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f68e lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827763.296918:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f68e lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827763.296945:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f68e lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827763.296964:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d01a5000 going to sleep for 0 seconds 00000100:00100000:0.0:1787827763.296973:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043185024:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:0.0:1787827763.296985:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f68e lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827763.296992:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f68e lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827763.296996:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f68e lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827763.297001:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f68e lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827763.297008:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f68e lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827763.297017:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0512800 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:0.0:1787827763.297021:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f68e lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827763.297030:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f68e lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1787827763.297038:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f68e lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787827763.297046:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:0.0:1787827763.297051:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787827763.297057:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f68e lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827763.297063:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f68e lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787827763.297086:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:0.0:1787827763.297089:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:3.0:1787827763.297131:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827763.297136:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827763.297138:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827763.297144:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9833 00000100:00080000:3.0:1787827763.297152:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827763.297158:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827763.297176:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043185152/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827763.297195:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827763.297198:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827763.297199:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827763.297202:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9833 00000100:00080000:3.0:1787827763.297204:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827763.297207:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827763.297214:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827763.297224:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043185280/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827763.297232:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827763.297234:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827763.297236:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1787827763.297240:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043185152:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787827763.297241:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9833 00000100:00080000:3.0:1787827763.297246:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827763.297261:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827763.297264:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827763.297273:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043185408/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827763.297314:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827763.297351:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827763/real 1787827763] req@ffff9181c5635c00 x1874663043185152/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827788 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827763.297363:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043185152/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827788 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827763.297371:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827763.297374:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827763.297384:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043185152:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827763.297401:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043185280:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827763.297425:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827763.297434:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827763/real 1787827763] req@ffff9181c5636d80 x1874663043185280/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827788 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827763.297443:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043185280/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827788 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827763.297448:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827763.297451:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827763.297453:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043185280:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827763.297457:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043185408:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827763.297467:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827763.297473:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827763/real 1787827763] req@ffff9181c5637800 x1874663043185408/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827788 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827763.297481:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043185408/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827788 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827763.297486:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827763.297488:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827763.297489:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043185408:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827763.808865:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827764.833019:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827765.856972:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827766.880953:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827767.904847:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827768.352907:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827768.352915:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827768.352936:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827768.352945:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9835 00000100:00080000:2.0:1787827768.352956:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827768.352963:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827768.353010:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c380 x1874663043185536/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827768.353039:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827768.353042:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827768.353044:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827768.353048:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9835 00000100:00080000:2.0:1787827768.353051:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827768.353055:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827768.353063:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827768.353075:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007dc00 x1874663043185664/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827768.353084:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827768.353087:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827768.353089:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827768.353091:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9835 00000100:00080000:2.0:1787827768.353093:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827768.353097:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827768.353100:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827768.353130:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c000 x1874663043185792/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827768.353131:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043185536:192.168.203.147@tcp:250:kworker.0 00000800:00000100:1.0:1787827768.353234:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827768.353283:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827768/real 1787827768] req@ffff9181e007c380 x1874663043185536/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827793 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827768.353302:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c380 x1874663043185536/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827793 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827768.353312:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827768.353315:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827768.353330:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043185536:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827768.353339:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043185664:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827768.353371:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827768.353391:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827768/real 1787827768] req@ffff9181e007dc00 x1874663043185664/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827793 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827768.353400:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007dc00 x1874663043185664/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827793 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827768.353407:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827768.353409:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827768.353412:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043185664:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827768.353436:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043185792:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827768.353454:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827768.353461:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827768/real 1787827768] req@ffff9181e007c000 x1874663043185792/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827793 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827768.353473:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c000 x1874663043185792/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827793 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827768.353481:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827768.353485:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827768.353489:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043185792:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827768.928911:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827769.952892:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827770.976853:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827772.000881:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827773.024876:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827773.472843:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827773.472870:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827773.472889:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827773.472899:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9840 00000100:00080000:2.0:1787827773.472909:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827773.472916:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827773.472970:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e680 x1874663043185920/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827773.473003:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827773.473009:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827773.473011:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827773.473015:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9840 00000100:00080000:2.0:1787827773.473018:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827773.473022:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827773.473029:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827773.473042:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ca80 x1874663043186048/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827773.473051:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827773.473054:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827773.473056:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827773.473059:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9840 00000100:00080000:2.0:1787827773.473061:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827773.473064:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827773.473068:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827773.473074:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043185920:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787827773.473080:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007fb80 x1874663043186176/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827773.473158:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827773.473219:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827773/real 1787827773] req@ffff9181e007e680 x1874663043185920/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827798 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827773.473237:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e680 x1874663043185920/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827798 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827773.473249:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827773.473255:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827773.473278:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043185920:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827773.473312:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043186048:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827773.473344:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827773.473363:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827773/real 1787827773] req@ffff9181e007ca80 x1874663043186048/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827798 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827773.473381:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ca80 x1874663043186048/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827798 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827773.473390:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827773.473394:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827773.473398:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043186048:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827773.473442:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043186176:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827773.473471:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827773.473487:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827773/real 1787827773] req@ffff9181e007fb80 x1874663043186176/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827798 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827773.473502:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007fb80 x1874663043186176/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827798 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827773.473529:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827773.473532:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827773.473536:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043186176:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827774.048897:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827775.072901:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827776.096942:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827777.120855:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827778.144918:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827778.592888:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827778.592901:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827778.592906:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827778.592921:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9845 00000100:00080000:3.0:1787827778.592934:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827778.592943:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827778.592993:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043186304/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827778.593048:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827778.593053:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827778.593055:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827778.593060:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9845 00000100:00080000:3.0:1787827778.593064:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827778.593068:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827778.593080:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827778.593096:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043186432/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827778.593108:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043186304:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787827778.593123:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827778.593130:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827778.593133:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827778.593138:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9845 00000100:00080000:3.0:1787827778.593142:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827778.593147:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827778.593153:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827778.593174:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043186560/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827778.593198:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827778.593237:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827778/real 1787827778] req@ffff9181c5637100 x1874663043186304/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827803 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827778.593252:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043186304/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827803 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827778.593273:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827778.593276:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827778.593291:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043186304:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827778.593313:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043186432:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827778.593339:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827778.593350:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827778/real 1787827778] req@ffff9181c5637480 x1874663043186432/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827803 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827778.593359:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043186432/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827803 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827778.593366:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827778.593369:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827778.593371:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043186432:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827778.593374:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043186560:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827778.593385:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827778.593391:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827778/real 1787827778] req@ffff9181c5636a00 x1874663043186560/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827803 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827778.593398:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043186560/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827803 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827778.593409:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827778.593411:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827778.593413:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043186560:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827779.169035:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827780.193022:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827781.217024:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827782.240917:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827783.264992:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827783.712821:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827783.712846:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827783.712850:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827783.712862:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9850 00000100:00080000:3.0:1787827783.712873:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827783.712880:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 10000000:01000000:0.0:1787827783.712903:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-cliir' failed (-108). 10000000:00000400:0.0:1787827783.712909:0:159735:0:(mgc_request.c:1900:mgc_process_log()) MGC192.168.203.147@tcp: IR log lustre-cliir failed, not fatal: rc = -108 10000000:01000000:0.0:1787827783.712918:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 00000100:00100000:3.0:1787827783.712919:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043186688/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 10000000:01000000:0.0:1787827783.712924:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff9181c5620000 from 52 10000000:01000000:0.0:1787827783.712932:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00000100:00080000:3.0:1787827783.712958:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827783.712962:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827783.712964:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827783.712979:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9850 00000100:00080000:3.0:1787827783.712983:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00010000:00010000:0.0:1787827783.712984:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f695 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787827783.712990:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:0.0:1787827783.713004:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f695 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000008:00100000:3.0:1787827783.713005:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:0.0:1787827783.713023:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f695 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787827783.713026:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634e00 x1874663043186944/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827783.713048:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043186688:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787827783.713055:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d01a5000 going to sleep for 0 seconds 00000100:00080000:3.0:1787827783.713055:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827783.713063:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827783.713065:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1787827783.713070:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043186816:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00080000:3.0:1787827783.713074:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9850 00000100:00080000:3.0:1787827783.713083:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827783.713087:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:0.0:1787827783.713090:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f695 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000008:00100000:3.0:1787827783.713094:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:0.0:1787827783.713116:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f695 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787827783.713123:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043187072/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:0.0:1787827783.713127:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f695 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000800:00000100:1.0:1787827783.713147:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00010000:00010000:0.0:1787827783.713155:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f695 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827783.713165:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f695 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827783.713177:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0512800 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:0.0:1787827783.713181:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f695 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827783.713188:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f695 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1787827783.713195:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f695 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787827783.713202:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00000100:00100000:1.0:1787827783.713205:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827783/real 1787827783] req@ffff9181c5634380 x1874663043186688/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827808 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:0.0:1787827783.713208:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787827783.713216:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f695 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827783.713225:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f695 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787827783.713229:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043186688/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827808 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827783.713263:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827783.713266:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827783.713270:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043186688:192.168.203.147@tcp:250:kworker.0 10000000:01000000:0.0:1787827783.713271:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:0.0:1787827783.713275:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00100000:1.0:1787827783.713287:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043186944:192.168.203.147@tcp:38:kworker.0 00000100:00080000:3.0:1787827783.713329:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827783.713335:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827783.713337:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000800:00000100:1.0:1787827783.713343:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1787827783.713361:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9855 00000100:00080000:3.0:1787827783.713367:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827783.713374:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787827783.713377:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827783/real 1787827783] req@ffff9181c5634e00 x1874663043186944/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827808 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827783.713393:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635f80 x1874663043187200/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827783.713396:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634e00 x1874663043186944/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827808 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827783.713433:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827783.713438:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827783.713443:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043186944:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827783.713450:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043187072:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827783.713469:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827783.713478:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827783/real 1787827783] req@ffff9181c5635500 x1874663043187072/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827808 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827783.713488:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043187072/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827808 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827783.713495:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827783.713498:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827783.713501:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043187072:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827783.713513:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043187200:192.168.203.147@tcp:250:kworker.0 00000800:00000100:1.0:1787827783.713526:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827783.713533:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827783/real 1787827783] req@ffff9181c5635f80 x1874663043187200/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827808 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827783.713543:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635f80 x1874663043187200/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827808 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827783.713550:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827783.713552:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827783.713577:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043187200:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787827784.288859:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827785.312967:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827786.336885:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827787.361091:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827788.384993:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827788.832873:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827788.832884:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827788.832889:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827788.832901:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9855 00000100:00080000:3.0:1787827788.832913:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827788.832920:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827788.832967:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635f80 x1874663043187328/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827788.832993:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827788.832996:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827788.832997:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827788.833000:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9855 00000100:00080000:3.0:1787827788.833003:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827788.833006:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827788.833016:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827788.833026:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043187456/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827788.833034:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827788.833036:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827788.833038:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1787827788.833043:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043187328:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787827788.833043:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9855 00000100:00080000:3.0:1787827788.833048:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827788.833059:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827788.833063:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827788.833071:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634e00 x1874663043187584/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827788.833129:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827788.833168:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827788/real 1787827788] req@ffff9181c5635f80 x1874663043187328/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827813 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827788.833184:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635f80 x1874663043187328/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827813 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827788.833193:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827788.833196:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827788.833211:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043187328:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827788.833235:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043187456:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827788.833265:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827788.833279:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827788/real 1787827788] req@ffff9181c5635500 x1874663043187456/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827813 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827788.833291:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043187456/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827813 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827788.833300:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827788.833304:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827788.833307:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043187456:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827788.833314:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043187584:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827788.833342:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827788.833353:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827788/real 1787827788] req@ffff9181c5634e00 x1874663043187584/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827813 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827788.833365:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634e00 x1874663043187584/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827813 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827788.833372:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827788.833375:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827788.833378:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043187584:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787827789.408959:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827790.432889:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827791.456941:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827792.480930:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827793.505034:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827793.952865:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827793.952873:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827793.952877:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827793.952885:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9860 00000100:00080000:2.0:1787827793.952895:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827793.952902:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827793.952936:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d880 x1874663043187712/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827793.952961:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827793.952964:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827793.952965:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827793.952969:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9860 00000100:00080000:2.0:1787827793.952972:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827793.952975:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827793.952983:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827793.952994:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ce00 x1874663043187840/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827793.953021:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827793.953025:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827793.953027:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827793.953031:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9860 00000100:00080000:2.0:1787827793.953037:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1787827793.953039:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043187712:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787827793.953045:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827793.953056:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827793.953068:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f480 x1874663043187968/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827793.953119:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827793.953171:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827793/real 1787827793] req@ffff9181e007d880 x1874663043187712/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827818 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827793.953190:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d880 x1874663043187712/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827818 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827793.953201:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827793.953205:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827793.953221:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043187712:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827793.953229:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043187840:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827793.953255:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827793.953272:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827793/real 1787827793] req@ffff9181e007ce00 x1874663043187840/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827818 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827793.953284:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ce00 x1874663043187840/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827818 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827793.953292:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827793.953297:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827793.953301:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043187840:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827793.953339:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043187968:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827793.953354:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827793.953361:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827793/real 1787827793] req@ffff9181e007f480 x1874663043187968/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827818 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827793.953370:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f480 x1874663043187968/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827818 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827793.953376:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827793.953378:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827793.953380:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043187968:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827794.528906:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827795.552969:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827796.577062:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827797.600997:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827798.624965:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827799.072856:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827799.072865:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827799.072868:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827799.072877:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9865 00000100:00080000:3.0:1787827799.072886:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827799.072892:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827799.072923:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043188096/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827799.072949:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827799.072952:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827799.072953:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827799.072961:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9865 00000100:00080000:3.0:1787827799.072963:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827799.072966:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827799.072973:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827799.072982:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043188224/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827799.072990:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827799.072992:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827799.072994:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827799.072996:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9865 00000100:00080000:3.0:1787827799.072998:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827799.073001:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827799.073004:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827799.073011:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043188352/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827799.073016:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043188096:192.168.203.147@tcp:250:kworker.0 00000800:00000100:1.0:1787827799.073072:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827799.073103:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827799/real 1787827799] req@ffff9181c5634380 x1874663043188096/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827824 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827799.073116:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043188096/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827824 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827799.073124:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827799.073127:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827799.073141:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043188096:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827799.073150:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043188224:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827799.073166:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827799.073177:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827799/real 1787827799] req@ffff9181c5636a00 x1874663043188224/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827824 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827799.073186:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043188224/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827824 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827799.073192:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827799.073195:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827799.073197:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043188224:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827799.073213:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043188352:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827799.073224:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827799.073230:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827799/real 1787827799] req@ffff9181c5637480 x1874663043188352/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827824 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827799.073238:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043188352/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827824 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827799.073247:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827799.073249:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827799.073252:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043188352:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787827799.648938:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827800.672993:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827801.696982:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827802.720892:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827803.744962:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827804.192954:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827804.192996:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827804.193003:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827804.193024:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9870 00000100:00080000:3.0:1787827804.193043:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827804.193057:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 10000000:01000000:0.0:1787827804.193095:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-client' failed (-108). 10000000:00020000:0.0:1787827804.193105:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:0.0:1787827804.193115:0:159735:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:0.0:1787827804.193118:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff9181c5620000 from 7 10000000:01000000:0.0:1787827804.193132:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00000100:00100000:3.0:1787827804.193149:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043188480/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:0.0:1787827804.193187:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f69c lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787827804.193194:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827804.193211:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827804.193242:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00010000:00010000:0.0:1787827804.193244:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f69c lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787827804.193250:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9870 00000100:00080000:3.0:1787827804.193260:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827804.193268:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:0.0:1787827804.193270:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f69c lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000008:00100000:3.0:1787827804.193282:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827804.193295:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d01a5000 going to sleep for 0 seconds 00000100:00100000:3.0:1787827804.193306:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043188736/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827804.193310:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043188608:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00100000:1.0:1787827804.193312:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043188480:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787827804.193339:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:0.0:1787827804.193341:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f69c lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787827804.193344:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827804.193350:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00010000:00010000:0.0:1787827804.193361:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f69c lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787827804.193362:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9870 00000100:00080000:3.0:1787827804.193371:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00010000:00010000:0.0:1787827804.193380:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f69c lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787827804.193380:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827804.193392:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:0.0:1787827804.193396:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f69c lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787827804.193410:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043188864/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:0.0:1787827804.193411:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f69c lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827804.193427:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0512800 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:0.0:1787827804.193435:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f69c lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000800:00000100:1.0:1787827804.193451:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00010000:00010000:0.0:1787827804.193454:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f69c lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1787827804.193464:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f69c lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787827804.193471:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:0.0:1787827804.193475:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787827804.193496:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f69c lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827804.193504:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f69c lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787827804.193532:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:0.0:1787827804.193536:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:CONNECTING 00000100:00100000:1.0:1787827804.193560:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827804/real 1787827804] req@ffff9181c5637100 x1874663043188480/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827829 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827804.193601:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043188480/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827829 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827804.193618:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827804.193698:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00080000:3.0:1787827804.193715:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1787827804.193718:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043188480:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787827804.193723:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827804.193729:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827804.193772:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9875 00000100:00080000:3.0:1787827804.193778:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827804.193783:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827804.193801:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043188992/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827804.193832:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043188736:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827804.193885:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827804.193903:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827804/real 1787827804] req@ffff9181c5637800 x1874663043188736/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827829 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827804.193918:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043188736/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827829 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827804.193928:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827804.193932:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827804.193945:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043188736:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827804.193954:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043188864:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827804.193985:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827804.194019:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827804/real 1787827804] req@ffff9181c5636d80 x1874663043188864/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827829 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827804.194040:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043188864/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827829 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827804.194050:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827804.194053:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827804.194057:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043188864:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827804.194081:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043188992:192.168.203.147@tcp:250:kworker.0 00000800:00000100:1.0:1787827804.194111:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827804.194131:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827804/real 1787827804] req@ffff9181c5635c00 x1874663043188992/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827829 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827804.194153:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043188992/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827829 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827804.194162:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827804.194166:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827804.194184:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043188992:192.168.203.147@tcp:250:kworker.0 00010000:00010000:2.0:1787827804.769015:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827805.792938:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827806.816960:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827807.841948:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827808.865217:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827809.312886:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827809.312899:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827809.312904:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827809.312917:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9875 00000100:00080000:2.0:1787827809.312931:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827809.312941:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827809.312991:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f100 x1874663043189120/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827809.313037:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827809.313045:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827809.313049:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827809.313058:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9875 00000100:00080000:2.0:1787827809.313065:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827809.313074:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827809.313088:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827809.313116:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f800 x1874663043189248/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827809.313139:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827809.313146:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827809.313153:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1787827809.313155:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043189120:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787827809.313166:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9875 00000100:00080000:2.0:1787827809.313176:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827809.313183:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827809.313191:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827809.313211:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ea00 x1874663043189376/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827809.313269:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827809.313318:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827809/real 1787827809] req@ffff9181e007f100 x1874663043189120/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827834 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827809.313340:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f100 x1874663043189120/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827834 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827809.313354:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827809.313359:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827809.313398:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043189120:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827809.313437:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043189248:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827809.313481:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827809.313506:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827809/real 1787827809] req@ffff9181e007f800 x1874663043189248/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827834 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827809.313525:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f800 x1874663043189248/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827834 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827809.313536:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827809.313540:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827809.313544:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043189248:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827809.313551:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043189376:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827809.313590:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827809.313634:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827809/real 1787827809] req@ffff9181e007ea00 x1874663043189376/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827834 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827809.313658:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ea00 x1874663043189376/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827834 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827809.313668:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827809.313672:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827809.313675:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043189376:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787827809.888891:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827810.912873:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827811.936859:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827812.960960:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827813.984977:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827814.432915:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827814.432928:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827814.432933:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827814.432945:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9881 00000100:00080000:0.0:1787827814.432960:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827814.432969:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787827814.433017:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7480 x1874663043189504/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827814.433054:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827814.433059:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827814.433061:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827814.433066:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9881 00000100:00080000:0.0:1787827814.433071:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827814.433075:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787827814.433087:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827814.433105:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6d80 x1874663043189632/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827814.433119:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827814.433123:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827814.433126:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827814.433131:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9881 00000100:00080000:0.0:1787827814.433138:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1787827814.433141:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043189504:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787827814.433149:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787827814.433160:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827814.433180:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5880 x1874663043189760/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827814.433271:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043189632:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827814.433328:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043189760:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827815.009036:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827816.033026:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827817.056950:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1787827817.505181:0:2378:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:0.0:1787827817.505247:0:2378:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1787827817.505273:0:2378:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1787827817.505351:0:2378:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1787827817.505370:0:2378:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:1.0:1787827817.505426:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827814/real 1787827817] req@ffff9181c55a5880 x1874663043189760/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827839 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827817.505476:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5880 x1874663043189760/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827839 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827817.505498:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827817.505507:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827817.505516:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043189760:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827817.505531:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827814/real 1787827817] req@ffff9181c55a7480 x1874663043189504/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827839 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827817.505609:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7480 x1874663043189504/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827839 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827817.505625:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827817.505633:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827817.505672:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043189504:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827817.505686:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827814/real 1787827817] req@ffff9181c55a6d80 x1874663043189632/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827839 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827817.505711:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6d80 x1874663043189632/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827839 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827817.505727:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827817.505734:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827817.505790:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043189632:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787827818.080925:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827819.104939:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827819.552885:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827819.552899:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827819.552905:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827819.552930:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9886 00000100:00080000:3.0:1787827819.552947:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827819.552961:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827819.553014:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043189888/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827819.553058:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827819.553063:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827819.553066:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827819.553071:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9886 00000100:00080000:3.0:1787827819.553077:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827819.553082:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827819.553095:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827819.553126:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634a80 x1874663043190016/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827819.553136:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043189888:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787827819.553164:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827819.553172:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827819.553176:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827819.553185:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9886 00000100:00080000:3.0:1787827819.553193:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827819.553201:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827819.553211:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000800:00000100:1.0:1787827819.553252:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827819.553253:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637b80 x1874663043190144/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827819.553319:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827819/real 1787827819] req@ffff9181c5634000 x1874663043189888/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827844 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827819.553350:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043189888/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827844 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827819.553369:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827819.553376:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827819.553422:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043189888:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827819.553462:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043190016:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827819.553508:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827819.553556:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827819/real 1787827819] req@ffff9181c5634a80 x1874663043190016/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827844 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827819.553583:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634a80 x1874663043190016/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827844 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827819.553599:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827819.553605:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827819.553611:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043190016:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827819.553623:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043190144:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827819.553662:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827819.553685:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827819/real 1787827819] req@ffff9181c5637b80 x1874663043190144/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827844 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827819.553705:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637b80 x1874663043190144/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827844 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827819.553719:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827819.553725:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827819.553732:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043190144:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787827820.128904:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827821.152949:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827822.176961:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827823.200875:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827824.224958:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827824.672810:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827824.672819:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827824.672823:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 10000000:01000000:0.0:1787827824.672831:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'params' failed (-108). 00000100:00080000:3.0:1787827824.672832:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9891 10000000:00020000:0.0:1787827824.672838:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:0.0:1787827824.672844:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 00000100:00080000:3.0:1787827824.672845:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 10000000:01000000:0.0:1787827824.672846:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff9181c87b19b8 from 1 10000000:01000000:0.0:1787827824.672852:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00000100:00080000:3.0:1787827824.672856:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827824.672883:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637b80 x1874663043190400/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:0.0:1787827824.672893:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f6a3 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827824.672904:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f6a3 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787827824.672905:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827824.672916:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827824.672917:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00010000:00010000:0.0:1787827824.672920:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f6a3 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787827824.672923:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9891 00000100:00080000:3.0:1787827824.672930:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827824.672935:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787827824.672941:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d01a5000 going to sleep for 0 seconds 00000008:00100000:3.0:1787827824.672945:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827824.672946:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043190400:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787827824.672952:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043190272:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00100000:3.0:1787827824.672963:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634a80 x1874663043190528/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:0.0:1787827824.672967:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f6a3 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787827824.672980:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:0.0:1787827824.672982:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f6a3 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787827824.672991:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827824.672997:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00010000:00010000:0.0:1787827824.673002:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f6a3 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787827824.673007:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9891 00010000:00010000:0.0:1787827824.673011:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f6a3 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787827824.673015:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000800:00000100:1.0:1787827824.673020:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1787827824.673025:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:0.0:1787827824.673035:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f6a3 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000008:00100000:3.0:1787827824.673037:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:0.0:1787827824.673045:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0512800 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:0.0:1787827824.673049:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f6a3 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827824.673055:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f6a3 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787827824.673078:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827824/real 1787827824] req@ffff9181c5637b80 x1874663043190400/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827849 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827824.673078:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043190656/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 10000000:00010000:0.0:1787827824.673080:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f6a3 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787827824.673104:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:0.0:1787827824.673111:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787827824.673120:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f6a3 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827824.673135:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f6a3 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787827824.673136:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637b80 x1874663043190400/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827849 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827824.673161:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827824.673165:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827824.673170:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043190400:192.168.203.147@tcp:250:kworker.0 10000000:01000000:0.0:1787827824.673171:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:0.0:1787827824.673185:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00100000:1.0:1787827824.673202:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043190528:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827824.673242:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1787827824.673252:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1787827824.673253:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827824/real 1787827824] req@ffff9181c5634a80 x1874663043190528/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827849 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827824.673264:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827824.673268:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1787827824.673274:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634a80 x1874663043190528/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827849 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827824.673275:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9896 00000100:00080000:3.0:1787827824.673283:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827824.673290:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827824.673292:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00080000:1.0:1787827824.673295:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827824.673300:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043190528:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827824.673305:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043190656:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827824.673316:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043190784/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827824.673323:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827824.673340:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827824/real 1787827824] req@ffff9181c5634000 x1874663043190656/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827849 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827824.673367:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043190656/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827849 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827824.673390:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827824.673395:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827824.673399:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043190656:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827824.673413:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043190784:192.168.203.147@tcp:250:kworker.0 00000800:00000100:1.0:1787827824.673429:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827824.673438:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827824/real 1787827824] req@ffff9181c5635c00 x1874663043190784/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827849 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827824.673448:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043190784/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827849 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827824.673456:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827824.673458:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827824.673474:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043190784:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787827825.248944:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827826.272972:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827827.296957:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827828.320969:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827829.344940:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827829.792811:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827829.792822:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827829.792827:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827829.792838:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9896 00000100:00080000:3.0:1787827829.792849:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827829.792855:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827829.792900:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043190912/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827829.792924:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827829.792927:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827829.792928:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827829.792932:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9896 00000100:00080000:3.0:1787827829.792934:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827829.792937:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827829.792946:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827829.792956:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043191040/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827829.792964:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827829.792967:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827829.792968:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827829.792971:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9896 00000100:00080000:3.0:1787827829.792975:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827829.792978:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827829.792982:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827829.792989:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043190912:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827829.792994:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043191168/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827829.793062:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827829.793094:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827829/real 1787827829] req@ffff9181c5636d80 x1874663043190912/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827854 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827829.793113:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043190912/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827854 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827829.793121:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827829.793124:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827829.793140:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043190912:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827829.793146:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043191040:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827829.793164:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827829.793174:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827829/real 1787827829] req@ffff9181c5637800 x1874663043191040/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827854 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827829.793190:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043191040/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827854 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827829.793195:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827829.793198:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827829.793201:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043191040:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827829.793220:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043191168:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827829.793230:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827829.793237:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827829/real 1787827829] req@ffff9181c5637100 x1874663043191168/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827854 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827829.793244:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043191168/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827854 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827829.793253:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827829.793256:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827829.793258:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043191168:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787827830.368949:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827831.392920:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827832.416897:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827833.440870:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827834.464911:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827834.912952:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827834.912965:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827834.912969:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827834.912981:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9901 00000100:00080000:1.0:1787827834.912994:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827834.913003:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787827834.913052:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0e00 x1874663043191296/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827834.913085:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827834.913089:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827834.913091:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827834.913095:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9901 00000100:00080000:1.0:1787827834.913098:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827834.913102:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827834.913113:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827834.913124:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0000 x1874663043191424/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827834.913135:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787827834.913138:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043191296:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787827834.913139:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827834.913143:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827834.913152:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9901 00000100:00080000:1.0:1787827834.913156:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827834.913159:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827834.913162:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827834.913171:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1180 x1874663043191552/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787827834.913253:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827834.913313:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827834/real 1787827834] req@ffff9181c35b0e00 x1874663043191296/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827859 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827834.913347:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0e00 x1874663043191296/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827859 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827834.913368:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827834.913373:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827834.913394:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043191296:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827834.913465:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043191424:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827834.913499:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827834.913515:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827834/real 1787827834] req@ffff9181c35b0000 x1874663043191424/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827859 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827834.913528:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0000 x1874663043191424/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827859 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827834.913543:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827834.913546:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827834.913549:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043191424:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827834.913555:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043191552:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827834.913570:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827834.913579:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827834/real 1787827834] req@ffff9181c35b1180 x1874663043191552/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827859 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827834.913588:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1180 x1874663043191552/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827859 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827834.913595:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827834.913597:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827834.913600:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043191552:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827835.488883:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827836.513014:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827837.537298:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827838.560934:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827839.584965:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827840.032932:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827840.032950:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827840.032956:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827840.032970:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9906 00000100:00080000:3.0:1787827840.033003:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827840.033015:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827840.033086:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043191680/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827840.033151:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827840.033159:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827840.033179:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827840.033187:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9906 00000100:00080000:3.0:1787827840.033194:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827840.033203:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827840.033219:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827840.033229:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043191680:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827840.033240:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043191808/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827840.033255:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827840.033259:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827840.033261:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827840.033265:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9906 00000100:00080000:3.0:1787827840.033270:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827840.033274:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827840.033280:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827840.033292:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043191936/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827840.033325:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827840.033387:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827840/real 1787827840] req@ffff9181c5637480 x1874663043191680/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827865 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827840.033427:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043191680/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827865 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827840.033439:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827840.033444:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827840.033460:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043191680:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827840.033489:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043191808:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827840.033518:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827840.033537:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827840/real 1787827840] req@ffff9181c5636a00 x1874663043191808/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827865 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827840.033552:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043191808/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827865 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827840.033562:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827840.033565:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827840.033569:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043191808:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827840.033574:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043191936:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827840.033590:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827840.033599:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827840/real 1787827840] req@ffff9181c5634380 x1874663043191936/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827865 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827840.033611:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043191936/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827865 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827840.033619:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827840.033622:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827840.033625:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043191936:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787827840.608920:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827841.632909:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827842.656917:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827843.680903:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827844.704907:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827845.152840:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827845.152850:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827845.152854:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827845.152864:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9911 00000100:00080000:3.0:1787827845.152875:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827845.152882:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 10000000:01000000:0.0:1787827845.152899:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-sptlrpc' failed (-108). 10000000:00020000:0.0:1787827845.152905:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00100000:3.0:1787827845.152917:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634e00 x1874663043192064/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827845.152942:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827845.152946:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827845.152947:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827845.152951:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9911 00000100:00080000:3.0:1787827845.152953:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827845.152956:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827845.152964:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827845.152973:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043192192/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827845.152981:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827845.152984:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827845.152986:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1787827845.152989:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043192064:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787827845.152991:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9911 00000100:00080000:3.0:1787827845.152998:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827845.153001:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827845.153004:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827845.153011:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635f80 x1874663043192320/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827845.153055:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827845.153083:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827845/real 1787827845] req@ffff9181c5634e00 x1874663043192064/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827870 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827845.153105:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634e00 x1874663043192064/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827870 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827845.153114:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827845.153117:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827845.153120:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043192064:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827845.153138:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043192192:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827845.153160:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827845.153170:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827845/real 1787827845] req@ffff9181c5635500 x1874663043192192/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827870 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827845.153190:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043192192/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827870 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827845.153196:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827845.153199:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827845.153201:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043192192:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827845.153205:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043192320:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827845.153216:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827845.153223:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827845/real 1787827845] req@ffff9181c5635f80 x1874663043192320/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827870 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827845.153231:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635f80 x1874663043192320/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827870 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827845.153237:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827845.153239:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827845.153241:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043192320:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827845.729060:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:0.0:1787827846.560836:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:0.0:1787827846.560843:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff9181c5620000 from 341 10000000:01000000:0.0:1787827846.560847:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:0.0:1787827846.560895:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0513c00/0xbef42c0221d6f6aa lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827846.560905:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e0513c00/0xbef42c0221d6f6aa lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827846.560914:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e0513c00/0xbef42c0221d6f6aa lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827846.560931:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d01a5000 going to sleep for 0 seconds 00000100:00100000:0.0:1787827846.560940:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043192448:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:0.0:1787827846.560953:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e0513c00/0xbef42c0221d6f6aa lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827846.560959:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e0513c00/0xbef42c0221d6f6aa lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827846.560963:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0513c00/0xbef42c0221d6f6aa lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827846.560968:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e0513c00/0xbef42c0221d6f6aa lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827846.560973:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e0513c00/0xbef42c0221d6f6aa lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827846.560982:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0513c00 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:0.0:1787827846.560985:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e0513c00/0xbef42c0221d6f6aa lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827846.560990:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e0513c00/0xbef42c0221d6f6aa lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1787827846.560995:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e0513c00/0xbef42c0221d6f6aa lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787827846.560998:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:0.0:1787827846.561001:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787827846.561005:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e0513c00/0xbef42c0221d6f6aa lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827846.561010:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e0513c00/0xbef42c0221d6f6aa lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787827846.561027:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:0.0:1787827846.561029:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:3.0:1787827846.561126:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827846.561131:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827846.561134:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827846.561140:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9916 00000100:00080000:3.0:1787827846.561148:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827846.561153:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827846.561172:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635f80 x1874663043192576/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827846.561192:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827846.561195:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827846.561197:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827846.561199:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9916 00000100:00080000:3.0:1787827846.561202:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827846.561204:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827846.561211:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827846.561220:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043192704/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827846.561228:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827846.561231:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827846.561234:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1787827846.561235:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043192576:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787827846.561239:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9916 00000100:00080000:3.0:1787827846.561245:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827846.561248:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827846.561251:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827846.561259:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634e00 x1874663043192832/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827846.561295:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827846.561322:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827846/real 1787827846] req@ffff9181c5635f80 x1874663043192576/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827871 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827846.561335:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635f80 x1874663043192576/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827871 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827846.561342:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827846.561360:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827846.561371:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043192576:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827846.561389:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043192704:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827846.561408:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827846.561416:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827846/real 1787827846] req@ffff9181c5635500 x1874663043192704/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827871 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827846.561425:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043192704/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827871 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827846.561434:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827846.561438:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827846.561441:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043192704:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827846.561447:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043192832:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827846.561467:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827846.561479:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827846/real 1787827846] req@ffff9181c5634e00 x1874663043192832/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827871 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827846.561489:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634e00 x1874663043192832/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827871 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827846.561495:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827846.561497:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827846.561499:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043192832:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787827846.752929:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827847.776878:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827848.800908:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827849.824944:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827850.848924:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827851.808844:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827851.808859:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827851.808864:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827851.808880:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9918 00000100:00080000:3.0:1787827851.808895:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827851.808905:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827851.808958:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043192960/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827851.809001:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827851.809006:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827851.809008:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827851.809014:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9918 00000100:00080000:3.0:1787827851.809017:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827851.809022:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827851.809034:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827851.809051:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043193088/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827851.809063:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827851.809069:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827851.809071:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827851.809076:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9918 00000100:00080000:3.0:1787827851.809081:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827851.809084:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827851.809091:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827851.809102:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043192960:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827851.809108:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043193216/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827851.809222:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827851.809280:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827851/real 1787827851] req@ffff9181c5634380 x1874663043192960/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827876 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827851.809339:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043192960/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827876 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827851.809360:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827851.809369:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827851.809393:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043192960:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827851.809404:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043193088:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827851.809441:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827851.809462:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827851/real 1787827851] req@ffff9181c5636a00 x1874663043193088/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827876 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827851.809487:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043193088/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827876 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827851.809497:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827851.809501:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827851.809505:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043193088:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827851.809536:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043193216:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827851.809555:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827851.809566:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827851/real 1787827851] req@ffff9181c5637480 x1874663043193216/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827876 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827851.809580:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043193216/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827876 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827851.809589:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827851.809592:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827851.809595:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043193216:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827851.872987:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827852.896939:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827853.920972:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827854.944925:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827855.968889:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827856.928906:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827856.928925:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827856.928931:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827856.928960:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9923 00000100:00080000:2.0:1787827856.928978:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827856.928990:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827856.929068:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c700 x1874663043193344/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827856.929128:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827856.929135:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827856.929138:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827856.929146:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9923 00000100:00080000:2.0:1787827856.929154:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827856.929160:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827856.929176:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827856.929181:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043193344:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787827856.929198:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e300 x1874663043193472/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827856.929213:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827856.929217:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827856.929219:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827856.929224:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9923 00000100:00080000:2.0:1787827856.929231:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827856.929236:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827856.929242:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000800:00000100:1.0:1787827856.929253:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787827856.929256:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d180 x1874663043193600/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827856.929316:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827856/real 1787827856] req@ffff9181e007c700 x1874663043193344/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827881 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827856.929337:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c700 x1874663043193344/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827881 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827856.929347:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827856.929350:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827856.929363:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043193344:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827856.929391:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043193472:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827856.929435:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827856.929449:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827856/real 1787827856] req@ffff9181e007e300 x1874663043193472/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827881 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827856.929460:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e300 x1874663043193472/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827881 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827856.929466:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827856.929468:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827856.929471:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043193472:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827856.929475:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043193600:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827856.929490:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827856.929497:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827856/real 1787827856] req@ffff9181e007d180 x1874663043193600/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827881 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827856.929504:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d180 x1874663043193600/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827881 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827856.929509:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827856.929511:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827856.929513:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043193600:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787827856.992918:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827858.016983:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827859.040942:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827860.064902:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827861.089029:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827862.048921:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827862.048933:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827862.048937:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827862.048951:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9928 00000100:00080000:2.0:1787827862.048965:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827862.048973:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827862.049047:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d500 x1874663043193728/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827862.049076:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827862.049079:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827862.049100:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827862.049108:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9928 00000100:00080000:2.0:1787827862.049113:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827862.049119:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827862.049132:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827862.049133:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043193728:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787827862.049171:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007df80 x1874663043193856/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827862.049184:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827862.049188:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827862.049190:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827862.049197:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9928 00000100:00080000:2.0:1787827862.049201:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827862.049205:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827862.049211:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000800:00000100:1.0:1787827862.049218:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787827862.049256:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ed80 x1874663043193984/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827862.049284:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827862/real 1787827862] req@ffff9181e007d500 x1874663043193728/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827887 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827862.049303:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d500 x1874663043193728/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827887 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827862.049311:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827862.049315:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827862.049325:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043193728:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827862.049349:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043193856:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827862.049365:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827862.049374:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827862/real 1787827862] req@ffff9181e007df80 x1874663043193856/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827887 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827862.049381:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007df80 x1874663043193856/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827887 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827862.049387:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827862.049389:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827862.049392:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043193856:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827862.049396:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043193984:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827862.049406:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827862.049418:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827862/real 1787827862] req@ffff9181e007ed80 x1874663043193984/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827887 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827862.049426:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ed80 x1874663043193984/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827887 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827862.049431:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827862.049433:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827862.049435:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043193984:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1787827862.112847:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827863.136929:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827864.160897:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827865.185018:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827866.208879:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:0.0:1787827866.656882:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-cliir' failed (-108). 10000000:00000400:0.0:1787827866.656891:0:159735:0:(mgc_request.c:1900:mgc_process_log()) MGC192.168.203.147@tcp: IR log lustre-cliir failed, not fatal: rc = -108 10000000:01000000:0.0:1787827866.656899:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:0.0:1787827866.656901:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff9181c5620000 from 52 10000000:01000000:0.0:1787827866.656904:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:0.0:1787827866.656957:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512a00/0xbef42c0221d6f6b1 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827866.656969:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e0512a00/0xbef42c0221d6f6b1 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827866.656979:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e0512a00/0xbef42c0221d6f6b1 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827866.656998:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d01a5000 going to sleep for 0 seconds 00000100:00100000:0.0:1787827866.657008:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043194112:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:0.0:1787827866.657020:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e0512a00/0xbef42c0221d6f6b1 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827866.657027:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e0512a00/0xbef42c0221d6f6b1 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827866.657049:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512a00/0xbef42c0221d6f6b1 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827866.657056:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e0512a00/0xbef42c0221d6f6b1 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827866.657063:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e0512a00/0xbef42c0221d6f6b1 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827866.657068:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0512a00 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:0.0:1787827866.657071:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512a00/0xbef42c0221d6f6b1 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827866.657078:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e0512a00/0xbef42c0221d6f6b1 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1787827866.657085:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512a00/0xbef42c0221d6f6b1 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787827866.657102:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:0.0:1787827866.657104:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787827866.657110:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e0512a00/0xbef42c0221d6f6b1 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827866.657116:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e0512a00/0xbef42c0221d6f6b1 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787827866.657135:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:0.0:1787827866.657138:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:3.0:1787827866.657246:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827866.657251:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827866.657254:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827866.657261:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9933 00000100:00080000:3.0:1787827866.657270:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827866.657275:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827866.657297:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043194240/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827866.657319:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827866.657322:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827866.657324:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827866.657328:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9933 00000100:00080000:3.0:1787827866.657331:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827866.657335:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827866.657344:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827866.657354:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043194368/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827866.657368:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827866.657372:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827866.657375:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1787827866.657382:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043194240:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787827866.657385:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9933 00000100:00080000:3.0:1787827866.657393:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827866.657400:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827866.657407:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827866.657425:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043194496/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827866.657470:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827866.657502:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827866/real 1787827866] req@ffff9181c5637100 x1874663043194240/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827891 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827866.657536:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043194240/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827891 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827866.657546:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827866.657550:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827866.657565:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043194240:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827866.657572:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043194368:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827866.657597:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827866.657608:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827866/real 1787827866] req@ffff9181c5637800 x1874663043194368/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827891 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827866.657619:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043194368/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827891 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827866.657659:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827866.657662:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827866.657664:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043194368:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827866.657688:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043194496:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827866.657705:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827866.657721:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827866/real 1787827866] req@ffff9181c5636d80 x1874663043194496/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827891 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827866.657730:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043194496/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827891 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827866.657737:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827866.657739:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827866.657770:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043194496:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787827867.232911:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827868.256901:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827869.280913:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827870.304937:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827871.328904:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827871.776893:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827871.776904:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827871.776909:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827871.776918:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9938 00000100:00080000:1.0:1787827871.776928:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827871.776935:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787827871.776970:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0380 x1874663043194624/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827871.777003:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827871.777007:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827871.777009:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827871.777014:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9938 00000100:00080000:1.0:1787827871.777019:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827871.777026:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827871.777036:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827871.777045:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043194624:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827871.777055:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3100 x1874663043194752/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827871.777071:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827871.777075:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827871.777078:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827871.777082:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9938 00000100:00080000:1.0:1787827871.777085:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827871.777090:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827871.777094:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827871.777122:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1c00 x1874663043194880/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787827871.777132:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827871.777226:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827871/real 1787827871] req@ffff9181c35b0380 x1874663043194624/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827896 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827871.777243:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0380 x1874663043194624/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827896 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827871.777253:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827871.777267:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827871.777279:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043194624:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827871.777300:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043194752:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827871.777326:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827871.777341:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827871/real 1787827871] req@ffff9181c35b3100 x1874663043194752/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827896 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827871.777351:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3100 x1874663043194752/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827896 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827871.777370:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827871.777372:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827871.777375:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043194752:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827871.777379:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043194880:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827871.777396:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827871.777409:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827871/real 1787827871] req@ffff9181c35b1c00 x1874663043194880/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827896 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827871.777425:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1c00 x1874663043194880/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827896 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827871.777436:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827871.777440:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827871.777445:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043194880:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827872.352896:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827873.376938:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827874.401020:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827875.424831:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827876.448928:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827876.896898:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827876.896909:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827876.896914:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827876.896927:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9943 00000100:00080000:3.0:1787827876.896940:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827876.896950:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827876.897003:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043195008/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827876.897037:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827876.897040:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827876.897042:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827876.897045:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9943 00000100:00080000:3.0:1787827876.897048:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827876.897052:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827876.897061:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827876.897072:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043195136/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827876.897081:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827876.897084:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827876.897086:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1787827876.897093:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043195008:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787827876.897093:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9943 00000100:00080000:3.0:1787827876.897104:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827876.897115:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827876.897121:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827876.897131:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634a80 x1874663043195264/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827876.897215:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827876.897252:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827876/real 1787827876] req@ffff9181c5635c00 x1874663043195008/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827901 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827876.897269:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043195008/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827901 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827876.897280:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827876.897284:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827876.897297:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043195008:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827876.897323:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043195136:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827876.897345:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827876.897355:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827876/real 1787827876] req@ffff9181c5634000 x1874663043195136/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827901 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827876.897366:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043195136/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827901 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827876.897373:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827876.897375:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827876.897378:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043195136:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827876.897383:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043195264:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827876.897414:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827876.897421:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827876/real 1787827876] req@ffff9181c5634a80 x1874663043195264/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827901 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827876.897430:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634a80 x1874663043195264/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827901 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827876.897442:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827876.897445:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827876.897447:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043195264:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787827877.472852:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827878.496992:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827879.520919:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827880.544902:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827881.568875:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827882.016820:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827882.016828:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827882.016831:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827882.016839:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9948 00000100:00080000:3.0:1787827882.016848:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827882.016854:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827882.016885:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637b80 x1874663043195392/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827882.016904:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827882.016908:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827882.016909:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827882.016912:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9948 00000100:00080000:3.0:1787827882.016914:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827882.016918:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827882.016925:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827882.016935:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634a80 x1874663043195520/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827882.016943:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827882.016948:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:1.0:1787827882.016949:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043195392:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787827882.016951:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827882.016959:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9948 00000100:00080000:3.0:1787827882.016963:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827882.016965:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827882.016969:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827882.016976:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043195648/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827882.017054:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043195520:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827882.017085:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043195648:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827882.592862:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827883.616853:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827884.640863:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1787827885.089188:0:2379:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:1.0:1787827885.089247:0:2379:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787827885.089270:0:2379:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1787827885.089318:0:2379:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1787827885.089328:0:2379:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:3.0:1787827885.089381:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827882/real 1787827885] req@ffff9181c5634a80 x1874663043195520/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827907 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827885.089416:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634a80 x1874663043195520/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827907 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827885.089432:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827885.089439:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827885.089446:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043195520:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827885.089456:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827882/real 1787827885] req@ffff9181c5634000 x1874663043195648/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827907 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827885.089471:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043195648/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827907 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827885.089480:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827885.089483:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827885.089487:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043195648:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827885.089494:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827882/real 1787827885] req@ffff9181c5637b80 x1874663043195392/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827907 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827885.089513:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637b80 x1874663043195392/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827907 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827885.089521:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827885.089525:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827885.089543:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043195392:192.168.203.147@tcp:250:kworker.0 00010000:00010000:2.0:1787827885.664991:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827886.688867:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:2.0:1787827886.688886:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-client' failed (-108). 10000000:00020000:2.0:1787827886.688892:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:2.0:1787827886.688897:0:159735:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:2.0:1787827886.688900:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff9181c5620000 from 7 10000000:01000000:2.0:1787827886.688903:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00010000:00010000:2.0:1787827886.688962:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1842600/0xbef42c0221d6f6b8 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827886.688972:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181d1842600/0xbef42c0221d6f6b8 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827886.688985:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181d1842600/0xbef42c0221d6f6b8 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827886.689007:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5c94240 going to sleep for 0 seconds 00000100:00100000:2.0:1787827886.689018:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ed80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043195776:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:2.0:1787827886.689031:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181d1842600/0xbef42c0221d6f6b8 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827886.689045:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181d1842600/0xbef42c0221d6f6b8 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827886.689051:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1842600/0xbef42c0221d6f6b8 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827886.689057:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181d1842600/0xbef42c0221d6f6b8 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827886.689062:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181d1842600/0xbef42c0221d6f6b8 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827886.689080:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181d1842600 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:2.0:1787827886.689084:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181d1842600/0xbef42c0221d6f6b8 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827886.689090:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181d1842600/0xbef42c0221d6f6b8 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1787827886.689095:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181d1842600/0xbef42c0221d6f6b8 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787827886.689100:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:2.0:1787827886.689103:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787827886.689110:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181d1842600/0xbef42c0221d6f6b8 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787827886.689115:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181d1842600/0xbef42c0221d6f6b8 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787827886.689132:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:2.0:1787827886.689135:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:3.0:1787827886.689170:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827886.689175:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827886.689178:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827886.689184:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9953 00000100:00080000:3.0:1787827886.689192:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827886.689198:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827886.689217:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637b80 x1874663043195904/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827886.689236:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827886.689239:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827886.689241:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827886.689244:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9953 00000100:00080000:3.0:1787827886.689246:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827886.689250:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827886.689259:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827886.689271:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043196032/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827886.689272:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043195904:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787827886.689295:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827886.689299:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827886.689302:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827886.689307:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9953 00000100:00080000:3.0:1787827886.689311:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827886.689316:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827886.689322:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827886.689337:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634a80 x1874663043196160/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827886.689346:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827886.689385:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827886/real 1787827886] req@ffff9181c5637b80 x1874663043195904/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827911 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827886.689400:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637b80 x1874663043195904/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827911 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827886.689409:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827886.689412:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827886.689421:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043195904:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827886.689436:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043196032:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827886.689454:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827886.689469:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827886/real 1787827886] req@ffff9181c5634000 x1874663043196032/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827911 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827886.689478:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043196032/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827911 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827886.689484:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827886.689486:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827886.689489:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043196032:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827886.689492:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043196160:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827886.689505:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827886.689511:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827886/real 1787827886] req@ffff9181c5634a80 x1874663043196160/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827911 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827886.689519:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634a80 x1874663043196160/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827911 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827886.689524:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827886.689526:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827886.689528:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043196160:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787827887.712905:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827888.736902:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827889.760948:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827890.784908:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827891.744814:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827891.744825:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827891.744830:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827891.744840:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9958 00000100:00080000:3.0:1787827891.744852:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827891.744859:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827891.744891:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043196288/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827891.744918:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827891.744921:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827891.744922:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827891.744925:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9958 00000100:00080000:3.0:1787827891.744929:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827891.744933:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827891.744940:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827891.744950:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043196416/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827891.744958:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827891.744961:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827891.744962:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827891.744966:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9958 00000100:00080000:3.0:1787827891.744968:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827891.744971:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827891.744975:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827891.744986:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043196544/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827891.745014:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043196288:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787827891.745134:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787827891.745185:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827891/real 1787827891] req@ffff9181c5635c00 x1874663043196288/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827916 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827891.745206:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043196288/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827916 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827891.745218:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827891.745221:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787827891.745239:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043196288:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787827891.745247:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043196416:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787827891.745285:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787827891.745304:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827891/real 1787827891] req@ffff9181c5636d80 x1874663043196416/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827916 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827891.745320:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043196416/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827916 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827891.745328:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827891.745332:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787827891.745336:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043196416:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787827891.745365:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043196544:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787827891.745398:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787827891.745413:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827891/real 1787827891] req@ffff9181c5637800 x1874663043196544/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827916 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827891.745428:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043196544/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827916 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827891.745434:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827891.745437:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787827891.745439:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043196544:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787827891.808946:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827892.832921:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827893.856927:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827894.880923:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827895.904888:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827896.864859:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827896.864872:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827896.864877:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827896.864889:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9963 00000100:00080000:3.0:1787827896.864901:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827896.864910:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827896.864957:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043196672/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827896.864992:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827896.865006:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827896.865008:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827896.865013:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9963 00000100:00080000:3.0:1787827896.865017:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827896.865022:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827896.865033:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827896.865049:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043196800/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787827896.865051:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043196672:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787827896.865068:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827896.865072:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827896.865074:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827896.865080:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9963 00000100:00080000:3.0:1787827896.865083:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827896.865087:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827896.865092:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827896.865103:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043196928/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787827896.865128:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787827896.865173:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827896/real 1787827896] req@ffff9181c5637100 x1874663043196672/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827921 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827896.865202:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043196672/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827921 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827896.865211:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827896.865221:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787827896.865234:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043196672:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787827896.865259:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043196800:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787827896.865280:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787827896.865290:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827896/real 1787827896] req@ffff9181c5637480 x1874663043196800/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827921 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827896.865299:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043196800/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827921 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827896.865305:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827896.865307:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787827896.865309:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043196800:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787827896.865313:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043196928:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787827896.865325:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787827896.865331:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827896/real 1787827896] req@ffff9181c5636a00 x1874663043196928/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827921 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787827896.865339:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043196928/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827921 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827896.865349:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787827896.865351:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787827896.865354:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043196928:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787827896.928926:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827897.952906:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827898.976880:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827900.000870:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827901.024882:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827901.984840:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827901.984848:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827901.984851:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827901.984859:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9968 00000100:00080000:2.0:1787827901.984869:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827901.984874:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827901.984906:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ed80 x1874663043197056/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787827901.984930:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827901.984933:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827901.984934:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827901.984937:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9968 00000100:00080000:2.0:1787827901.984939:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827901.984948:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827901.984956:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827901.984983:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007df80 x1874663043197184/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787827901.984985:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043197056:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787827901.985000:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827901.985003:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787827901.985005:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827901.985008:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9968 00000100:00080000:2.0:1787827901.985011:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827901.985014:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827901.985018:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787827901.985026:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d500 x1874663043197312/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827901.985046:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827901.985074:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827901/real 1787827901] req@ffff9181e007ed80 x1874663043197056/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827926 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827901.985087:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ed80 x1874663043197056/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827926 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827901.985094:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827901.985097:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827901.985112:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043197056:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827901.985130:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043197184:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827901.985148:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827901.985161:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827901/real 1787827901] req@ffff9181e007df80 x1874663043197184/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827926 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827901.985170:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007df80 x1874663043197184/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827926 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827901.985175:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827901.985177:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827901.985179:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043197184:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827901.985183:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043197312:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827901.985194:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827901.985200:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827901/real 1787827901] req@ffff9181e007d500 x1874663043197312/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827926 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827901.985213:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d500 x1874663043197312/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827926 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827901.985222:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827901.985225:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827901.985228:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043197312:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787827902.048868:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827903.072834:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827904.096864:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827905.120839:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827906.144853:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787827907.104843:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 10000000:01000000:0.0:1787827907.104855:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'params' failed (-108). 00000100:00080000:2.0:1787827907.104857:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827907.104864:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 10000000:00020000:0.0:1787827907.104867:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00080000:2.0:1787827907.104875:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9973 10000000:01000000:0.0:1787827907.104876:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:0.0:1787827907.104884:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff9181c87b19b8 from 1 10000000:01000000:0.0:1787827907.104889:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00000100:00080000:2.0:1787827907.104890:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787827907.104899:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827907.104946:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d180 x1874663043197568/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:0.0:1787827907.104964:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512600/0xbef42c0221d6f6bf lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787827907.104973:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:0.0:1787827907.104977:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e0512600/0xbef42c0221d6f6bf lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787827907.104977:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787827907.104980:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827907.104993:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9973 00000100:00080000:2.0:1787827907.105002:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00010000:00010000:0.0:1787827907.105003:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e0512600/0xbef42c0221d6f6bf lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787827907.105009:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787827907.105027:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827907.105035:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d01a5000 going to sleep for 0 seconds 00000100:00100000:3.0:1787827907.105037:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043197568:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787827907.105044:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e300 x1874663043197696/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827907.105048:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043197440:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00080000:2.0:1787827907.105081:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827907.105085:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00010000:00010000:0.0:1787827907.105086:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e0512600/0xbef42c0221d6f6bf lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787827907.105088:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787827907.105103:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9973 00010000:00010000:0.0:1787827907.105107:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e0512600/0xbef42c0221d6f6bf lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787827907.105109:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000800:00000100:3.0:1787827907.105122:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1787827907.105125:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:0.0:1787827907.105141:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512600/0xbef42c0221d6f6bf lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000008:00100000:2.0:1787827907.105143:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:0.0:1787827907.105159:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e0512600/0xbef42c0221d6f6bf lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787827907.105167:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c700 x1874663043197824/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:0.0:1787827907.105169:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e0512600/0xbef42c0221d6f6bf lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787827907.105179:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827907/real 1787827907] req@ffff9181e007d180 x1874663043197568/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827932 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:0.0:1787827907.105219:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0512600 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:0.0:1787827907.105223:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512600/0xbef42c0221d6f6bf lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827907.105239:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e0512600/0xbef42c0221d6f6bf lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787827907.105245:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d180 x1874663043197568/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827932 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 10000000:00010000:0.0:1787827907.105256:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512600/0xbef42c0221d6f6bf lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787827907.105261:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827907.105268:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 10000000:01000000:0.0:1787827907.105272:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00000100:00100000:3.0:1787827907.105273:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043197568:192.168.203.147@tcp:250:kworker.0 00010000:00010000:0.0:1787827907.105284:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787827907.105294:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e0512600/0xbef42c0221d6f6bf lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787827907.105302:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043197696:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1787827907.105310:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e0512600/0xbef42c0221d6f6bf lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787827907.105328:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:0.0:1787827907.105331:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000800:00000100:3.0:1787827907.105335:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827907.105347:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827907/real 1787827907] req@ffff9181e007e300 x1874663043197696/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827932 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827907.105357:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e300 x1874663043197696/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827932 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827907.105364:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827907.105367:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827907.105371:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043197696:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827907.105378:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043197824:192.168.203.147@tcp:38:kworker.0 00000100:00080000:2.0:1787827907.105386:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787827907.105391:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787827907.105393:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000800:00000100:3.0:1787827907.105397:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1787827907.105401:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9978 00000100:00080000:2.0:1787827907.105409:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787827907.105414:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827907/real 1787827907] req@ffff9181e007c700 x1874663043197824/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827932 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787827907.105419:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787827907.105448:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ea00 x1874663043197952/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827907.105456:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c700 x1874663043197824/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827932 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827907.105471:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827907.105474:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827907.105479:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043197824:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827907.105492:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043197952:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1787827907.105506:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827907.105512:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827907/real 1787827907] req@ffff9181e007ea00 x1874663043197952/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827932 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827907.105520:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ea00 x1874663043197952/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827932 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827907.105526:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827907.105527:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827907.105538:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043197952:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1787827907.168909:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827908.192966:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827909.216928:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827910.240919:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827911.264964:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827912.224855:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827912.224863:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827912.224866:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827912.224874:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9978 00000100:00080000:3.0:1787827912.224884:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827912.224907:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827912.224940:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043198080/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827912.224964:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827912.224967:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827912.224968:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827912.224971:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9978 00000100:00080000:3.0:1787827912.224977:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827912.224980:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827912.224987:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827912.224995:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634e00 x1874663043198208/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827912.225003:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827912.225005:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827912.225007:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827912.225009:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9978 00000100:00080000:3.0:1787827912.225011:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827912.225013:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827912.225021:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827912.225024:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043198080:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827912.225030:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043198336/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827912.225094:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827912.225125:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827912/real 1787827912] req@ffff9181c5634380 x1874663043198080/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827937 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827912.225139:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043198080/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827937 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827912.225147:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827912.225150:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827912.225164:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043198080:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827912.225170:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043198208:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827912.225205:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827912.225217:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827912/real 1787827912] req@ffff9181c5634e00 x1874663043198208/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827937 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827912.225230:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634e00 x1874663043198208/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827937 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827912.225241:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827912.225244:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827912.225247:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043198208:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827912.225266:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043198336:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827912.225278:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827912.225284:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827912/real 1787827912] req@ffff9181c5635500 x1874663043198336/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827937 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827912.225318:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043198336/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827937 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827912.225323:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827912.225326:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827912.225327:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043198336:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787827912.288916:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827913.312953:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827914.336992:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827915.360830:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827916.384872:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827917.344863:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827917.344895:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827917.344900:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827917.344913:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9983 00000100:00080000:3.0:1787827917.344925:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827917.344934:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827917.344996:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635f80 x1874663043198464/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827917.345021:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827917.345024:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827917.345026:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827917.345030:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9983 00000100:00080000:3.0:1787827917.345032:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827917.345035:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827917.345045:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827917.345057:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043198592/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827917.345068:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1787827917.345069:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043198464:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787827917.345076:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827917.345080:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827917.345084:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9983 00000100:00080000:3.0:1787827917.345087:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827917.345090:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827917.345093:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827917.345108:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634e00 x1874663043198720/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827917.345170:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827917.345236:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827917/real 1787827917] req@ffff9181c5635f80 x1874663043198464/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827942 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827917.345256:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635f80 x1874663043198464/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827942 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827917.345271:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827917.345277:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827917.345298:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043198464:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827917.345352:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043198592:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827917.345384:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827917.345400:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827917/real 1787827917] req@ffff9181c5635500 x1874663043198592/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827942 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827917.345416:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043198592/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827942 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827917.345427:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827917.345431:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827917.345436:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043198592:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827917.345443:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043198720:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827917.345462:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827917.345471:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827917/real 1787827917] req@ffff9181c5634e00 x1874663043198720/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827942 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827917.345480:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634e00 x1874663043198720/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827942 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827917.345486:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827917.345489:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827917.345491:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043198720:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1787827917.408825:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827918.432907:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827919.456895:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827920.480852:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827921.504848:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827922.464936:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827922.464948:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827922.464953:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827922.464963:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9989 00000100:00080000:1.0:1787827922.464976:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827922.464984:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787827922.465047:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1500 x1874663043198848/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827922.465091:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827922.465095:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827922.465097:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827922.465102:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9989 00000100:00080000:1.0:1787827922.465105:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827922.465109:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827922.465119:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827922.465135:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1c00 x1874663043198976/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827922.465148:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043198848:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787827922.465153:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827922.465163:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827922.465165:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827922.465170:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9989 00000100:00080000:1.0:1787827922.465173:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827922.465177:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827922.465182:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827922.465196:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3100 x1874663043199104/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787827922.465235:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827922.465276:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827922/real 1787827922] req@ffff9181c35b1500 x1874663043198848/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827947 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827922.465296:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1500 x1874663043198848/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827947 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827922.465307:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827922.465312:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827922.465330:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043198848:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827922.465354:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043198976:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827922.465386:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827922.465405:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827922/real 1787827922] req@ffff9181c35b1c00 x1874663043198976/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827947 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827922.465419:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1c00 x1874663043198976/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827947 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827922.465428:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827922.465431:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827922.465435:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043198976:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827922.465440:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043199104:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827922.465466:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827922.465475:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827922/real 1787827922] req@ffff9181c35b3100 x1874663043199104/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827947 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827922.465487:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3100 x1874663043199104/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827947 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827922.465494:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827922.465497:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827922.465500:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043199104:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787827922.528956:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827923.552864:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827924.577059:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787827925.600981:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827926.625083:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827927.584917:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827927.584931:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827927.584937:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 10000000:01000000:0.0:1787827927.584947:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-sptlrpc' failed (-108). 00000100:00080000:3.0:1787827927.584953:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9994 10000000:00020000:0.0:1787827927.584965:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00080000:3.0:1787827927.584974:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827927.585033:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827927.585085:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043199232/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827927.585123:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827927.585129:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827927.585133:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827927.585139:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9994 00000100:00080000:3.0:1787827927.585146:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827927.585152:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827927.585163:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827927.585184:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043199360/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827927.585199:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827927.585204:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827927.585208:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827927.585214:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9994 00000100:00080000:3.0:1787827927.585219:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827927.585225:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827927.585245:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827927.585261:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043199232:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827927.585266:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043199488/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827927.585366:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827927.585411:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827927/real 1787827927] req@ffff9181c5634380 x1874663043199232/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827952 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827927.585451:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043199232/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827952 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827927.585467:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827927.585475:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827927.585483:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043199232:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827927.585498:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043199360:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827927.585544:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827927.585571:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827927/real 1787827927] req@ffff9181c5636a00 x1874663043199360/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827952 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827927.585591:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043199360/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827952 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827927.585604:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827927.585609:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827927.585614:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043199360:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827927.585680:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043199488:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827927.585733:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827927.585846:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827927/real 1787827927] req@ffff9181c5637480 x1874663043199488/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827952 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827927.585875:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043199488/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827952 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827927.585892:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827927.585899:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827927.585907:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043199488:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1787827927.648899:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827928.673001:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:0.0:1787827929.120990:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:0.0:1787827929.121004:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff9181c5620000 from 341 10000000:01000000:0.0:1787827929.121026:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:0.0:1787827929.121128:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f6c6 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827929.121150:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f6c6 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827929.121170:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f6c6 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827929.121206:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d01a5000 going to sleep for 0 seconds 00000100:00100000:0.0:1787827929.121225:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043199616:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:0.0:1787827929.121251:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f6c6 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827929.121264:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f6c6 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827929.121274:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f6c6 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827929.121284:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f6c6 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827929.121293:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f6c6 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827929.121302:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0512800 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:0.0:1787827929.121307:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f6c6 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827929.121318:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f6c6 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1787827929.121327:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f6c6 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787827929.121334:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:0.0:1787827929.121338:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787827929.121348:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f6c6 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787827929.121358:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f6c6 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787827929.121400:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:0.0:1787827929.121404:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:3.0:1787827929.121534:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827929.121544:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827929.121549:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827929.121560:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 9999 00000100:00080000:3.0:1787827929.121575:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827929.121585:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827929.121622:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043199744/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827929.121661:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827929.121666:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827929.121670:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827929.121676:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9999 00000100:00080000:3.0:1787827929.121680:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827929.121686:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827929.121700:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827929.121719:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043199872/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787827929.121741:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827929.121822:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827929.121826:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827929.121835:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 9999 00000100:00080000:3.0:1787827929.121843:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827929.121850:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827929.121859:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827929.121878:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043199744:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827929.121890:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043200000/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787827929.122033:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827929.122106:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827929/real 1787827929] req@ffff9181c5637100 x1874663043199744/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827954 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827929.122142:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043199744/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827954 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827929.122163:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827929.122171:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827929.122203:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043199744:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827929.122229:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043199872:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827929.122282:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827929.122310:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827929/real 1787827929] req@ffff9181c5637800 x1874663043199872/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827954 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827929.122344:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043199872/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827954 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827929.122376:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827929.122383:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827929.122391:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043199872:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787827929.122430:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043200000:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787827929.122467:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787827929.122484:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827929/real 1787827929] req@ffff9181c5636d80 x1874663043200000/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827954 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787827929.122509:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043200000/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827954 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787827929.122520:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787827929.122524:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787827929.122528:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043200000:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1787827929.696957:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827930.720996:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827931.744895:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827932.769033:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787827933.793020:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827934.240959:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827934.240976:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827934.240983:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827934.240999:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10000 00000100:00080000:1.0:1787827934.241018:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827934.241027:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787827934.241077:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0380 x1874663043200128/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827934.241120:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827934.241126:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827934.241129:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827934.241134:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10000 00000100:00080000:1.0:1787827934.241139:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827934.241145:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827934.241158:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827934.241175:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1180 x1874663043200256/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827934.241189:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043200128:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787827934.241197:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827934.241204:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827934.241207:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827934.241213:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10000 00000100:00080000:1.0:1787827934.241218:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827934.241223:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827934.241229:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827934.241248:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0000 x1874663043200384/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787827934.241290:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827934.241340:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827934/real 1787827934] req@ffff9181c35b0380 x1874663043200128/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827959 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827934.241364:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0380 x1874663043200128/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827959 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827934.241380:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827934.241385:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827934.241411:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043200128:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827934.241442:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043200256:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827934.241469:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827934.241485:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827934/real 1787827934] req@ffff9181c35b1180 x1874663043200256/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827959 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827934.241502:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1180 x1874663043200256/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827959 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827934.241516:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827934.241521:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827934.241525:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043200256:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827934.241536:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043200384:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827934.241561:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827934.241573:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827934/real 1787827934] req@ffff9181c35b0000 x1874663043200384/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827959 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827934.241587:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0000 x1874663043200384/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827959 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827934.241597:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827934.241601:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827934.241604:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043200384:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787827934.817011:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827935.840911:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827936.864898:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827937.888948:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827938.912852:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827939.360847:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827939.360861:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827939.360867:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827939.360882:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10005 00000100:00080000:1.0:1787827939.360897:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827939.360908:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787827939.360966:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0e00 x1874663043200512/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827939.361003:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827939.361010:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827939.361012:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827939.361019:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10005 00000100:00080000:1.0:1787827939.361024:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827939.361031:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827939.361047:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827939.361067:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3480 x1874663043200640/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827939.361080:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043200512:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787827939.361082:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827939.361088:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827939.361090:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827939.361095:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10005 00000100:00080000:1.0:1787827939.361100:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827939.361105:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827939.361112:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827939.361125:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2680 x1874663043200768/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787827939.361188:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827939.361242:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827939/real 1787827939] req@ffff9181c35b0e00 x1874663043200512/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827964 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827939.361263:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0e00 x1874663043200512/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827964 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827939.361275:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827939.361280:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827939.361298:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043200512:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827939.361330:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043200640:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827939.361359:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827939.361376:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827939/real 1787827939] req@ffff9181c35b3480 x1874663043200640/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827964 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827939.361388:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3480 x1874663043200640/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827964 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827939.361397:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827939.361400:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827939.361403:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043200640:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827939.361408:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043200768:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827939.361425:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827939.361433:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827939/real 1787827939] req@ffff9181c35b2680 x1874663043200768/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827964 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827939.361456:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2680 x1874663043200768/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827964 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827939.361464:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827939.361467:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827939.361470:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043200768:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787827939.936938:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827940.960927:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827941.984972:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827943.009110:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827944.032849:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827944.480871:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827944.480884:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827944.480888:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827944.480898:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10011 00000100:00080000:0.0:1787827944.480908:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827944.480915:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787827944.480954:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5c00 x1874663043200896/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827944.480981:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827944.480986:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827944.480988:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827944.480992:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10011 00000100:00080000:0.0:1787827944.480997:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827944.481002:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787827944.481010:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827944.481022:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5f80 x1874663043201024/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827944.481030:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043200896:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787827944.481032:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827944.481039:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827944.481042:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827944.481046:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10011 00000100:00080000:0.0:1787827944.481049:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827944.481051:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787827944.481055:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827944.481063:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4700 x1874663043201152/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827944.481144:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043201024:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827944.481173:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043201152:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787827945.056888:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827946.080898:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827947.104861:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1787827947.553125:0:2376:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:1.0:1787827947.553184:0:2376:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787827947.553210:0:2376:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1787827947.553276:0:2376:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1787827947.553294:0:2376:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:3.0:1787827947.553361:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827944/real 1787827947] req@ffff9181c55a5f80 x1874663043201024/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827969 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827947.553408:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5f80 x1874663043201024/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827969 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827947.553445:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827947.553455:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827947.553462:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043201024:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827947.553478:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827944/real 1787827947] req@ffff9181c55a4700 x1874663043201152/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827969 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827947.553505:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4700 x1874663043201152/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827969 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827947.553521:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827947.553529:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827947.553536:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043201152:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827947.553551:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827944/real 1787827947] req@ffff9181c55a5c00 x1874663043200896/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827969 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827947.553577:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5c00 x1874663043200896/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827969 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827947.553595:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827947.553602:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827947.553673:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043200896:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1787827948.128970:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787827949.152877:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-cliir' failed (-108). 00010000:00010000:3.0:1787827949.152884:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:00000400:1.0:1787827949.152886:0:159735:0:(mgc_request.c:1900:mgc_process_log()) MGC192.168.203.147@tcp: IR log lustre-cliir failed, not fatal: rc = -108 10000000:01000000:1.0:1787827949.152897:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:1.0:1787827949.152900:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff9181c5620000 from 52 10000000:01000000:1.0:1787827949.152905:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:1.0:1787827949.152946:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f6cd lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827949.152955:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f6cd lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827949.152964:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f6cd lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787827949.152980:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215dc00 going to sleep for 0 seconds 00000100:00100000:1.0:1787827949.152989:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043201280:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787827949.152999:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f6cd lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827949.153004:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f6cd lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827949.153009:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f6cd lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827949.153014:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f6cd lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827949.153020:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f6cd lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827949.153025:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4200 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787827949.153028:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f6cd lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827949.153034:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f6cd lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787827949.153041:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f6cd lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787827949.153047:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1787827949.153050:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787827949.153057:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f6cd lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827949.153065:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f6cd lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787827949.153093:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787827949.153096:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:0.0:1787827949.153139:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827949.153143:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827949.153145:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827949.153150:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10016 00000100:00080000:0.0:1787827949.153157:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827949.153162:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787827949.153178:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4380 x1874663043201408/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827949.153195:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827949.153198:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827949.153199:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827949.153202:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10016 00000100:00080000:0.0:1787827949.153205:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827949.153207:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787827949.153214:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827949.153221:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5180 x1874663043201536/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827949.153232:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827949.153235:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827949.153236:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827949.153239:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10016 00000100:00080000:0.0:1787827949.153241:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827949.153244:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787827949.153247:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827949.153253:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5500 x1874663043201664/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827949.153265:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043201408:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1787827949.153329:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827949.153358:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827949/real 1787827949] req@ffff9181c55a4380 x1874663043201408/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827974 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827949.153370:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4380 x1874663043201408/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827974 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827949.153378:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827949.153381:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827949.153392:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043201408:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827949.153397:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043201536:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827949.153417:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827949.153428:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827949/real 1787827949] req@ffff9181c55a5180 x1874663043201536/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827974 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827949.153441:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5180 x1874663043201536/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827974 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827949.153450:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827949.153453:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827949.153457:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043201536:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827949.153476:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043201664:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827949.153494:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827949.153503:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827949/real 1787827949] req@ffff9181c55a5500 x1874663043201664/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827974 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827949.153511:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5500 x1874663043201664/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827974 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827949.153521:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827949.153523:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827949.153525:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043201664:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787827950.176902:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827951.200910:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827952.225117:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827953.248982:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827954.208882:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827954.208898:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827954.208903:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827954.208918:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10020 00000100:00080000:0.0:1787827954.208941:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827954.208950:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787827954.209008:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4e00 x1874663043201792/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827954.209052:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827954.209055:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827954.209057:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827954.209061:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10020 00000100:00080000:0.0:1787827954.209064:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827954.209068:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787827954.209079:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827954.209092:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7800 x1874663043201920/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827954.209102:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827954.209105:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827954.209107:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827954.209111:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10020 00000100:00080000:0.0:1787827954.209114:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827954.209119:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787827954.209122:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043201792:192.168.203.147@tcp:250:kworker.0 00000008:00100000:0.0:1787827954.209124:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827954.209139:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5500 x1874663043202048/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787827954.209257:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827954.209300:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827954/real 1787827954] req@ffff9181c55a4e00 x1874663043201792/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827979 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827954.209320:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4e00 x1874663043201792/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827979 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827954.209331:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827954.209342:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827954.209356:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043201792:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827954.209385:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043201920:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827954.209407:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827954.209430:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827954/real 1787827954] req@ffff9181c55a7800 x1874663043201920/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827979 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827954.209446:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7800 x1874663043201920/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827979 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827954.209461:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827954.209465:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827954.209468:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043201920:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827954.209474:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043202048:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827954.209495:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827954.209503:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827954/real 1787827954] req@ffff9181c55a5500 x1874663043202048/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827979 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827954.209513:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5500 x1874663043202048/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827979 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827954.209519:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827954.209522:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827954.209524:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043202048:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787827954.273000:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827955.296911:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827956.320863:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827957.345027:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827958.368984:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827959.328857:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827959.328869:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827959.328874:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827959.328886:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10025 00000100:00080000:0.0:1787827959.328906:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827959.328915:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787827959.328982:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5180 x1874663043202176/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827959.329020:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827959.329024:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827959.329026:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827959.329033:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10025 00000100:00080000:0.0:1787827959.329037:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827959.329041:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787827959.329053:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827959.329065:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4380 x1874663043202304/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787827959.329079:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827959.329084:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787827959.329086:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827959.329093:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10025 00000100:00080000:0.0:1787827959.329099:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827959.329106:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787827959.329114:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827959.329134:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5c00 x1874663043202432/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827959.329159:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043202176:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1787827959.329296:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827959.329364:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827959/real 1787827959] req@ffff9181c55a5180 x1874663043202176/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827984 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827959.329399:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5180 x1874663043202176/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827984 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827959.329415:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827959.329421:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827959.329459:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043202176:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827959.329470:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043202304:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827959.329507:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827959.329535:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827959/real 1787827959] req@ffff9181c55a4380 x1874663043202304/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827984 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827959.329552:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4380 x1874663043202304/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827984 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827959.329596:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827959.329601:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827959.329622:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043202304:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827959.329666:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043202432:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827959.329692:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827959.329704:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827959/real 1787827959] req@ffff9181c55a5c00 x1874663043202432/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827984 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827959.329718:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5c00 x1874663043202432/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827984 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827959.329727:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827959.329731:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827959.329735:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043202432:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827959.392921:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827960.416980:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827961.441012:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827962.465037:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827963.488992:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827964.448949:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827964.448964:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827964.448970:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827964.448986:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10031 00000100:00080000:1.0:1787827964.449029:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827964.449038:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787827964.449133:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2a00 x1874663043202560/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827964.449194:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827964.449197:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827964.449199:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827964.449203:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10031 00000100:00080000:1.0:1787827964.449206:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827964.449210:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827964.449221:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827964.449234:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0a80 x1874663043202688/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827964.449244:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043202560:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787827964.449245:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827964.449254:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827964.449255:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827964.449259:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10031 00000100:00080000:1.0:1787827964.449262:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827964.449265:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827964.449268:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827964.449277:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2d80 x1874663043202816/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787827964.449340:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827964.449391:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827964/real 1787827964] req@ffff9181c35b2a00 x1874663043202560/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827989 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827964.449409:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2a00 x1874663043202560/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827989 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827964.449419:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827964.449439:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827964.449465:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043202560:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827964.449505:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043202688:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827964.449592:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827964.449602:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827964/real 1787827964] req@ffff9181c35b0a80 x1874663043202688/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827989 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827964.449611:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0a80 x1874663043202688/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827989 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827964.449618:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827964.449620:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827964.449623:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043202688:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827964.449627:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043202816:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827964.449642:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827964.449649:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827964/real 1787827964] req@ffff9181c35b2d80 x1874663043202816/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827989 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827964.449657:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2d80 x1874663043202816/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827989 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827964.449679:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827964.449681:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827964.449683:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043202816:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787827964.512939:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827965.536999:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827966.561004:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827967.584962:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827968.608909:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787827969.568946:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827969.568965:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787827969.568973:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827969.568992:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10036 00000100:00080000:0.0:1787827969.569010:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 10000000:01000000:1.0:1787827969.569013:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-client' failed (-108). 10000000:00020000:1.0:1787827969.569022:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:1.0:1787827969.569031:0:159735:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:1.0:1787827969.569036:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff9181c5620000 from 7 00000100:00080000:0.0:1787827969.569037:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 10000000:01000000:1.0:1787827969.569043:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00000100:00100000:0.0:1787827969.569108:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4700 x1874663043202944/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787827969.569137:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f6d4 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827969.569158:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f6d4 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787827969.569179:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:1.0:1787827969.569183:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f6d4 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787827969.569187:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787827969.569192:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827969.569202:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10036 00000100:00080000:0.0:1787827969.569210:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827969.569219:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787827969.569220:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215dc00 going to sleep for 0 seconds 00000100:00100000:1.0:1787827969.569236:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043203072:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000008:00100000:0.0:1787827969.569237:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787827969.569252:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043202944:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787827969.569271:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5f80 x1874663043203200/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787827969.569271:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f6d4 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827969.569293:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f6d4 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787827969.569299:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787827969.569309:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00010000:00010000:1.0:1787827969.569312:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f6d4 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787827969.569314:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787827969.569324:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10036 00010000:00010000:1.0:1787827969.569328:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f6d4 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787827969.569331:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827969.569339:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787827969.569351:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:1.0:1787827969.569358:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f6d4 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000800:00000100:3.0:1787827969.569376:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827969.569380:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6d80 x1874663043203328/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787827969.569381:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4200 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787827969.569393:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f6d4 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827969.569409:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f6d4 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787827969.569425:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f6d4 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787827969.569442:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827969/real 1787827969] req@ffff9181c55a4700 x1874663043202944/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827994 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 10000000:01000000:1.0:1787827969.569443:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:1.0:1787827969.569470:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787827969.569520:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f6d4 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787827969.569524:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4700 x1874663043202944/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827994 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:1.0:1787827969.569559:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f6d4 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787827969.569569:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827969.569578:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827969.569587:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043202944:192.168.203.147@tcp:250:kworker.0 10000000:01000000:1.0:1787827969.569604:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787827969.569609:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00100000:3.0:1787827969.569625:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043203200:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827969.569665:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:0.0:1787827969.569686:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787827969.569688:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827969/real 1787827969] req@ffff9181c55a5f80 x1874663043203200/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827994 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827969.569706:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00100000:3.0:1787827969.569719:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5f80 x1874663043203200/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827994 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827969.569723:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827969.569738:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827969.569739:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10041 00000100:00080000:3.0:1787827969.569776:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827969.569785:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043203200:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827969.569795:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043203328:192.168.203.147@tcp:38:kworker.0 00000100:00080000:0.0:1787827969.569815:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787827969.569830:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:3.0:1787827969.569840:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827969.569866:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827969/real 1787827969] req@ffff9181c55a6d80 x1874663043203328/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827994 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827969.569876:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7480 x1874663043203456/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787827969.569886:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6d80 x1874663043203328/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827994 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827969.569953:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827969.569967:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827969.569976:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043203328:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827969.570005:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043203456:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1787827969.570045:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827969.570075:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827969/real 1787827969] req@ffff9181c55a7480 x1874663043203456/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827994 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827969.570098:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7480 x1874663043203456/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827994 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827969.570109:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827969.570113:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827969.570139:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043203456:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787827969.633015:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827970.656956:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827971.681044:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827972.705043:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827973.728963:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827974.688868:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827974.688879:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827974.688884:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827974.688896:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10041 00000100:00080000:1.0:1787827974.688918:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827974.688927:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787827974.688968:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0700 x1874663043203584/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827974.688999:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827974.689003:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827974.689005:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827974.689011:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10041 00000100:00080000:1.0:1787827974.689014:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827974.689018:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827974.689029:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827974.689043:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1f80 x1874663043203712/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827974.689057:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827974.689060:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827974.689062:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827974.689069:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10041 00000100:00100000:3.0:1787827974.689073:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043203584:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787827974.689090:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827974.689099:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827974.689114:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827974.689130:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3800 x1874663043203840/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787827974.689213:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827974.689255:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827974/real 1787827974] req@ffff9181c35b0700 x1874663043203584/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827999 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827974.689275:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0700 x1874663043203584/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787827999 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827974.689302:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827974.689307:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827974.689323:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043203584:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827974.689354:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043203712:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827974.689405:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827974.689423:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827974/real 1787827974] req@ffff9181c35b1f80 x1874663043203712/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827999 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827974.689452:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1f80 x1874663043203712/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827999 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827974.689507:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827974.689512:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827974.689516:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043203712:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827974.689524:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043203840:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827974.689550:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827974.689581:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827974/real 1787827974] req@ffff9181c35b3800 x1874663043203840/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827999 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827974.689597:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3800 x1874663043203840/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787827999 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827974.689621:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827974.689626:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827974.689630:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043203840:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787827974.752958:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827975.776902:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827976.800946:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827977.824989:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827978.848924:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827979.808884:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827979.808895:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827979.808900:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827979.808912:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10046 00000100:00080000:1.0:1787827979.808925:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827979.808934:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787827979.808988:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3b80 x1874663043203968/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827979.809023:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827979.809028:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827979.809030:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827979.809034:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10046 00000100:00080000:1.0:1787827979.809038:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827979.809042:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827979.809052:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827979.809068:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3800 x1874663043204096/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827979.809080:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827979.809084:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827979.809086:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827979.809090:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10046 00000100:00080000:1.0:1787827979.809095:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787827979.809103:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043203968:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787827979.809104:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827979.809116:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827979.809131:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1f80 x1874663043204224/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787827979.809198:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827979.809251:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827979/real 1787827979] req@ffff9181c35b3b80 x1874663043203968/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828004 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827979.809298:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3b80 x1874663043203968/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828004 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827979.809319:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827979.809325:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827979.809345:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043203968:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787827979.809374:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043204096:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827979.809416:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827979.809494:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827979/real 1787827979] req@ffff9181c35b3800 x1874663043204096/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828004 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827979.809526:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3800 x1874663043204096/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828004 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827979.809542:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827979.809549:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827979.809555:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043204096:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827979.809591:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043204224:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787827979.809625:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787827979.809639:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827979/real 1787827979] req@ffff9181c35b1f80 x1874663043204224/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828004 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787827979.809652:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1f80 x1874663043204224/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828004 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827979.809701:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787827979.809705:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787827979.809710:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043204224:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787827979.873064:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827980.896951:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827981.920936:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827982.945029:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827983.968981:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827984.928837:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827984.928850:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827984.928855:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827984.928868:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10051 00000100:00080000:1.0:1787827984.928881:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827984.928890:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787827984.928925:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0700 x1874663043204352/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827984.928956:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827984.928960:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827984.928962:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827984.928966:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10051 00000100:00080000:1.0:1787827984.928970:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827984.928975:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827984.928984:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827984.928997:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043204352:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827984.929001:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2d80 x1874663043204480/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827984.929021:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827984.929027:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827984.929030:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827984.929036:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10051 00000100:00080000:1.0:1787827984.929043:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827984.929050:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827984.929057:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827984.929077:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0a80 x1874663043204608/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827984.929082:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827984.929130:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827984/real 1787827984] req@ffff9181c35b0700 x1874663043204352/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828009 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827984.929149:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0700 x1874663043204352/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828009 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827984.929159:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827984.929162:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827984.929185:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043204352:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787827984.929211:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043204480:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827984.929237:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827984.929249:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827984/real 1787827984] req@ffff9181c35b2d80 x1874663043204480/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828009 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827984.929261:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2d80 x1874663043204480/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828009 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827984.929268:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827984.929271:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827984.929274:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043204480:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827984.929279:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043204608:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827984.929292:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827984.929300:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827984/real 1787827984] req@ffff9181c35b0a80 x1874663043204608/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828009 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827984.929310:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0a80 x1874663043204608/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828009 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827984.929316:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827984.929318:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827984.929321:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043204608:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787827984.992944:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827986.016935:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827987.040930:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827988.064844:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827989.088944:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787827990.048904:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827990.048920:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827990.048926:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827990.048943:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10056 10000000:01000000:1.0:1787827990.048953:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'params' failed (-108). 00000100:00080000:3.0:1787827990.048960:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 10000000:00020000:1.0:1787827990.048961:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:1.0:1787827990.048969:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 00000100:00080000:3.0:1787827990.048972:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 10000000:01000000:1.0:1787827990.048974:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff9181c87b19b8 from 1 10000000:01000000:1.0:1787827990.048979:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00000100:00100000:3.0:1787827990.049021:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043204864/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787827990.049039:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f6db lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827990.049058:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f6db lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787827990.049066:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787827990.049073:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787827990.049076:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00010000:00010000:1.0:1787827990.049080:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f6db lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787827990.049084:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10056 00000100:00080000:3.0:1787827990.049091:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827990.049099:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787827990.049110:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215dc00 going to sleep for 0 seconds 00000008:00100000:3.0:1787827990.049113:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827990.049123:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043204736:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00100000:3.0:1787827990.049135:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634a80 x1874663043204992/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787827990.049145:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f6db lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787827990.049156:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:1.0:1787827990.049161:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f6db lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787827990.049163:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787827990.049168:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00010000:00010000:1.0:1787827990.049177:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f6db lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787827990.049177:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10056 00000100:00080000:3.0:1787827990.049184:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1787827990.049198:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043204864:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787827990.049199:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f6db lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787827990.049199:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787827990.049215:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:1.0:1787827990.049229:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f6db lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787827990.049236:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043205120/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787827990.049244:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4200 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787827990.049253:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f6db lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827990.049268:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f6db lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787827990.049285:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f6db lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787827990.049298:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1787827990.049305:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00000800:00000100:0.0:1787827990.049315:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00010000:00010000:1.0:1787827990.049319:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f6db lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787827990.049345:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f6db lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787827990.049393:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827990/real 1787827990] req@ffff9181c5635c00 x1874663043204864/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828015 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 10000000:01000000:1.0:1787827990.049408:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787827990.049416:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:CONNECTING 00000100:00100000:0.0:1787827990.049429:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043204864/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828015 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827990.049445:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827990.049470:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827990.049483:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043204864:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787827990.049493:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043204992:192.168.203.147@tcp:38:kworker.0 00000100:00080000:3.0:1787827990.049522:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:0.0:1787827990.049537:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1787827990.049539:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787827990.049547:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787827990.049560:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10061 00000100:00100000:0.0:1787827990.049562:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827990/real 1787827990] req@ffff9181c5634a80 x1874663043204992/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828015 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827990.049585:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634a80 x1874663043204992/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828015 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787827990.049586:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787827990.049603:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00080000:0.0:1787827990.049609:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827990.049617:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827990.049622:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043204992:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787827990.049636:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637b80 x1874663043205248/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787827990.049656:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043205120:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827990.049686:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827990.049700:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827990/real 1787827990] req@ffff9181c5634000 x1874663043205120/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828015 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827990.049716:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043205120/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828015 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827990.049726:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827990.049730:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827990.049734:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043205120:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827990.049804:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043205248:192.168.203.147@tcp:250:kworker.0 00000800:00000100:0.0:1787827990.049830:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827990.049856:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827990/real 1787827990] req@ffff9181c5637b80 x1874663043205248/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828015 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827990.049875:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637b80 x1874663043205248/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828015 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827990.049895:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827990.049900:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827990.049917:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043205248:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787827990.112994:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827991.136966:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827992.160882:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827993.184868:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827994.208926:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787827995.168916:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827995.168928:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787827995.168933:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827995.168945:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10061 00000100:00080000:1.0:1787827995.168977:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827995.168985:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787827995.169027:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2a00 x1874663043205376/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827995.169063:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827995.169087:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787827995.169091:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827995.169100:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10061 00000100:00080000:1.0:1787827995.169109:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827995.169119:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827995.169135:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787827995.169160:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043205376:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787827995.169167:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2680 x1874663043205504/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787827995.169199:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787827995.169203:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787827995.169206:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787827995.169211:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10061 00000100:00080000:1.0:1787827995.169217:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787827995.169222:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787827995.169229:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787827995.169245:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3480 x1874663043205632/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787827995.169272:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827995.169380:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827995/real 1787827995] req@ffff9181c35b2a00 x1874663043205376/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828020 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827995.169422:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2a00 x1874663043205376/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828020 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827995.169440:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827995.169447:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827995.169465:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043205376:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787827995.169503:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043205504:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827995.169543:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827995.169565:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827995/real 1787827995] req@ffff9181c35b2680 x1874663043205504/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828020 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827995.169587:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2680 x1874663043205504/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828020 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827995.169600:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827995.169606:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827995.169612:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043205504:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787827995.169624:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043205632:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787827995.169655:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787827995.169670:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787827995/real 1787827995] req@ffff9181c35b3480 x1874663043205632/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828020 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787827995.169683:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3480 x1874663043205632/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828020 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787827995.169692:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787827995.169695:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787827995.169699:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043205632:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787827995.232956:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827996.256978:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827997.281030:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787827998.304911:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787827999.329001:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828000.288870:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828000.288879:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828000.288883:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828000.288894:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10066 00000100:00080000:0.0:1787828000.288906:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828000.288913:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787828000.288956:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5880 x1874663043205760/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828000.288985:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828000.288989:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828000.288992:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828000.288996:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10066 00000100:00080000:0.0:1787828000.289000:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828000.289005:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828000.289017:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828000.289031:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7b80 x1874663043205888/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828000.289035:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043205760:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787828000.289041:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828000.289045:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828000.289047:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828000.289051:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10066 00000100:00080000:0.0:1787828000.289054:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828000.289058:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828000.289062:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828000.289093:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6680 x1874663043206016/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828000.289132:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828000.289176:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828000/real 1787828000] req@ffff9181c55a5880 x1874663043205760/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828025 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828000.289191:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5880 x1874663043205760/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828025 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828000.289200:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828000.289205:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828000.289217:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043205760:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828000.289245:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043205888:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828000.289264:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828000.289274:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828000/real 1787828000] req@ffff9181c55a7b80 x1874663043205888/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828025 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828000.289309:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7b80 x1874663043205888/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828025 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828000.289315:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828000.289318:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828000.289320:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043205888:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828000.289325:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043206016:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828000.289338:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828000.289345:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828000/real 1787828000] req@ffff9181c55a6680 x1874663043206016/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828025 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828000.289353:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6680 x1874663043206016/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828025 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828000.289358:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828000.289360:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828000.289362:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043206016:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828000.352894:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828001.376933:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828002.400963:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828003.424935:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828004.449060:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828005.408903:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828005.408917:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828005.408923:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828005.408938:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10072 00000100:00080000:3.0:1787828005.408952:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828005.408962:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828005.409032:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637b80 x1874663043206144/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828005.409070:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828005.409076:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828005.409078:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828005.409084:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10072 00000100:00080000:3.0:1787828005.409088:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828005.409093:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828005.409105:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828005.409121:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043206272/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828005.409138:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828005.409143:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828005.409146:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1787828005.409157:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043206144:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787828005.409159:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10072 00000100:00080000:3.0:1787828005.409170:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828005.409177:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828005.409185:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828005.409201:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634a80 x1874663043206400/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787828005.409282:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787828005.409350:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828005/real 1787828005] req@ffff9181c5637b80 x1874663043206144/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828030 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828005.409378:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637b80 x1874663043206144/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828030 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828005.409397:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828005.409405:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787828005.409433:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043206144:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787828005.409487:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043206272:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787828005.409529:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787828005.409564:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828005/real 1787828005] req@ffff9181c5634000 x1874663043206272/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828030 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828005.409582:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043206272/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828030 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828005.409594:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828005.409598:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787828005.409603:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043206272:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787828005.409611:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043206400:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787828005.409634:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787828005.409659:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828005/real 1787828005] req@ffff9181c5634a80 x1874663043206400/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828030 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828005.409674:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634a80 x1874663043206400/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828030 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828005.409684:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828005.409687:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787828005.409691:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043206400:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828005.472995:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828006.496942:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828007.521010:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828008.545000:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828009.568948:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828010.528839:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828010.528847:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828010.528850:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 10000000:01000000:1.0:1787828010.528859:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-sptlrpc' failed (-108). 10000000:00020000:1.0:1787828010.528865:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00080000:0.0:1787828010.528868:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10077 00000100:00080000:0.0:1787828010.528881:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828010.528890:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787828010.528924:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7100 x1874663043206528/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828010.528947:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828010.528950:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828010.528951:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828010.528955:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10077 00000100:00080000:0.0:1787828010.528959:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828010.528962:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828010.528970:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828010.528981:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6680 x1874663043206656/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828010.528990:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828010.528993:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828010.528994:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828010.528997:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10077 00000100:00100000:3.0:1787828010.529006:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043206528:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787828010.529007:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828010.529021:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828010.529026:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828010.529038:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7b80 x1874663043206784/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828010.529172:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043206656:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828010.529203:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043206784:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828010.592848:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828011.617134:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787828011.872893:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:1.0:1787828011.872902:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff9181c5620000 from 341 10000000:01000000:1.0:1787828011.872909:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:1.0:1787828011.872995:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f6e2 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828011.873011:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f6e2 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828011.873034:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f6e2 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787828011.873077:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215dc00 going to sleep for 0 seconds 00000100:00100000:1.0:1787828011.873090:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043206912:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787828011.873126:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f6e2 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828011.873136:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f6e2 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828011.873145:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f6e2 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828011.873155:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f6e2 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828011.873164:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f6e2 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828011.873172:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4e00 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787828011.873177:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f6e2 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828011.873186:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f6e2 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787828011.873224:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f6e2 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787828011.873234:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:1.0:1787828011.873239:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787828011.873248:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f6e2 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828011.873257:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f6e2 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787828011.873285:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787828011.873289:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:CONNECTING 00010000:00010000:1.0:1787828012.640903:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1787828013.601024:0:2377:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:1.0:1787828013.601072:0:2377:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787828013.601091:0:2377:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1787828013.601138:0:2377:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1787828013.601153:0:2377:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:2.0:1787828013.601205:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828010/real 1787828013] req@ffff9181c55a6680 x1874663043206656/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828035 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828013.601239:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6680 x1874663043206656/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828035 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828013.601254:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828013.601259:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828013.601265:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043206656:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828013.601275:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828010/real 1787828013] req@ffff9181c55a7b80 x1874663043206784/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828035 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828013.601287:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7b80 x1874663043206784/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828035 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828013.601294:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828013.601299:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828013.601304:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043206784:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828013.601314:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828010/real 1787828013] req@ffff9181c55a7100 x1874663043206528/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828035 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828013.601334:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7100 x1874663043206528/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828035 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828013.601350:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828013.601402:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828013.601453:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043206528:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787828013.601493:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828013.601502:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828013.601507:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828013.601517:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10082 00000100:00080000:0.0:1787828013.601526:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828013.601536:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787828013.601592:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5880 x1874663043207040/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828013.601633:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828013.601654:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828013.601656:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828013.601661:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10082 00000100:00080000:0.0:1787828013.601667:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828013.601673:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828013.601684:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828013.601702:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7480 x1874663043207168/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828013.601717:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828013.601721:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828013.601724:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828013.601729:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10082 00000100:00080000:0.0:1787828013.601733:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828013.601739:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828013.601747:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828013.601786:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6d80 x1874663043207296/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828013.601924:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043207040:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787828013.602008:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828013.602045:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828013/real 1787828013] req@ffff9181c55a5880 x1874663043207040/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828038 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828013.602064:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5880 x1874663043207040/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828038 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828013.602077:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828013.602081:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828013.602100:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043207040:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828013.602109:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043207168:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828013.602147:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828013.602166:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828013/real 1787828013] req@ffff9181c55a7480 x1874663043207168/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828038 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828013.602199:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7480 x1874663043207168/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828038 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828013.602211:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828013.602216:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828013.602221:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043207168:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828013.602231:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043207296:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828013.602260:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828013.602278:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828013/real 1787828013] req@ffff9181c55a6d80 x1874663043207296/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828038 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828013.602293:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6d80 x1874663043207296/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828038 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828013.602303:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828013.602307:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828013.602313:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043207296:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828013.664958:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828014.688907:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828015.712914:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828016.736999:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828017.760855:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828018.720867:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828018.720878:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828018.720882:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828018.720893:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10085 00000100:00080000:3.0:1787828018.720905:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828018.720912:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828018.720956:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043207424/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828018.720988:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828018.720992:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828018.720993:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828018.720997:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10085 00000100:00080000:3.0:1787828018.721000:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828018.721003:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828018.721012:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828018.721021:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043207552/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828018.721028:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828018.721031:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828018.721032:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828018.721034:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10085 00000100:00080000:3.0:1787828018.721036:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828018.721038:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828018.721041:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828018.721047:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043207680/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828018.721075:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043207424:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787828018.721172:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828018.721217:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828018/real 1787828018] req@ffff9181c5635c00 x1874663043207424/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828043 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828018.721234:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043207424/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828043 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828018.721244:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828018.721247:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828018.721262:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043207424:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828018.721269:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043207552:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828018.721298:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828018.721309:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828018/real 1787828018] req@ffff9181c5636d80 x1874663043207552/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828043 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828018.721320:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043207552/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828043 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828018.721331:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828018.721334:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828018.721337:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043207552:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828018.721341:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043207680:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828018.721352:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828018.721358:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828018/real 1787828018] req@ffff9181c5637800 x1874663043207680/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828043 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828018.721366:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043207680/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828043 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828018.721371:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828018.721373:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828018.721375:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043207680:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828018.784884:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828019.808989:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828020.833228:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828021.857005:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828022.881002:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828023.840898:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828023.840911:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828023.840916:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828023.840928:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10090 00000100:00080000:2.0:1787828023.840943:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828023.840952:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787828023.841002:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f800 x1874663043207808/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828023.841068:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828023.841074:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828023.841077:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828023.841082:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10090 00000100:00080000:2.0:1787828023.841087:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828023.841093:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828023.841105:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828023.841152:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f100 x1874663043207936/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828023.841153:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043207808:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787828023.841171:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828023.841174:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828023.841193:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828023.841198:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10090 00000100:00080000:2.0:1787828023.841202:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828023.841207:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828023.841214:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828023.841232:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f480 x1874663043208064/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828023.841292:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828023.841334:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828023/real 1787828023] req@ffff9181e007f800 x1874663043207808/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828048 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828023.841379:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f800 x1874663043207808/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828048 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828023.841394:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828023.841399:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828023.841439:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043207808:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828023.841491:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043207936:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828023.841539:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828023.841552:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828023/real 1787828023] req@ffff9181e007f100 x1874663043207936/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828048 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828023.841571:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f100 x1874663043207936/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828048 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828023.841598:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828023.841602:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828023.841606:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043207936:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828023.841614:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043208064:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828023.841638:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828023.841670:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828023/real 1787828023] req@ffff9181e007f480 x1874663043208064/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828048 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828023.841686:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f480 x1874663043208064/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828048 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828023.841696:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828023.841700:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828023.841704:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043208064:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828023.904933:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828024.928983:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828025.953067:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828026.976925:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828028.000987:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828028.960948:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828028.960963:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828028.960970:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828028.960985:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10095 00000100:00080000:0.0:1787828028.961002:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828028.961013:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787828028.961075:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5f80 x1874663043208192/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828028.961150:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828028.961156:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828028.961158:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828028.961164:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10095 00000100:00080000:0.0:1787828028.961169:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828028.961176:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828028.961191:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828028.961215:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4700 x1874663043208320/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828028.961231:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828028.961235:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828028.961238:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787828028.961247:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043208192:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787828028.961253:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10095 00000100:00080000:0.0:1787828028.961264:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828028.961271:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828028.961278:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828028.961295:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5c00 x1874663043208448/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828028.961371:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828028.961430:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828028/real 1787828028] req@ffff9181c55a5f80 x1874663043208192/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828053 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828028.961456:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5f80 x1874663043208192/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828053 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828028.961472:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828028.961477:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828028.961503:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043208192:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828028.961549:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043208320:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828028.961605:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828028.961631:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828028/real 1787828028] req@ffff9181c55a4700 x1874663043208320/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828053 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828028.961657:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4700 x1874663043208320/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828053 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828028.961669:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828028.961673:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828028.961687:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043208320:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828028.961696:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043208448:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828028.961725:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828028.961739:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828028/real 1787828028] req@ffff9181c55a5c00 x1874663043208448/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828053 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828028.961825:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5c00 x1874663043208448/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828053 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828028.961853:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828028.961860:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828028.961867:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043208448:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828029.025058:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828030.049044:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828031.072895:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787828031.904934:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-cliir' failed (-108). 10000000:00000400:1.0:1787828031.904965:0:159735:0:(mgc_request.c:1900:mgc_process_log()) MGC192.168.203.147@tcp: IR log lustre-cliir failed, not fatal: rc = -108 10000000:01000000:1.0:1787828031.904975:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:1.0:1787828031.904979:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff9181c5620000 from 52 10000000:01000000:1.0:1787828031.904984:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:1.0:1787828031.905040:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f6e9 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828031.905080:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f6e9 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828031.905131:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f6e9 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787828031.905175:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215dc00 going to sleep for 0 seconds 00000100:00100000:1.0:1787828031.905190:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043208576:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787828031.905210:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f6e9 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828031.905227:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f6e9 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828031.905242:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f6e9 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828031.905258:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f6e9 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828031.905275:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f6e9 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828031.905289:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4e00 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787828031.905297:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f6e9 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828031.905310:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f6e9 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787828031.905320:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f6e9 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787828031.905329:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1787828031.905334:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787828031.905343:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f6e9 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828031.905353:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f6e9 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787828031.905380:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787828031.905384:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:0.0:1787828031.905564:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828031.905571:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828031.905575:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828031.905594:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10100 00000100:00080000:0.0:1787828031.905606:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828031.905614:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787828031.905660:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4380 x1874663043208704/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828031.905690:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828031.905694:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828031.905696:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828031.905701:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10100 00000100:00080000:0.0:1787828031.905705:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828031.905709:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828031.905719:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828031.905735:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5180 x1874663043208832/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828031.905837:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043208704:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787828031.905868:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828031.905876:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828031.905880:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828031.905893:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10100 00000100:00080000:0.0:1787828031.905902:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828031.905912:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828031.905924:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1787828031.905929:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787828031.905977:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5500 x1874663043208960/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828031.906003:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828031/real 1787828031] req@ffff9181c55a4380 x1874663043208704/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828056 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828031.906027:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4380 x1874663043208704/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828056 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828031.906040:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828031.906045:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828031.906111:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043208704:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828031.906141:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043208832:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828031.906178:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828031.906196:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828031/real 1787828031] req@ffff9181c55a5180 x1874663043208832/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828056 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828031.906221:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5180 x1874663043208832/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828056 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828031.906237:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828031.906244:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828031.906251:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043208832:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828031.906264:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043208960:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828031.906305:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828031.906331:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828031/real 1787828031] req@ffff9181c55a5500 x1874663043208960/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828056 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828031.906348:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5500 x1874663043208960/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828056 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828031.906359:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828031.906363:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828031.906368:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043208960:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828032.096955:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828033.120968:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828034.144981:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828035.168945:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828036.192966:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828037.152840:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828037.152849:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828037.152854:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828037.152864:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10103 00000100:00080000:3.0:1787828037.152876:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828037.152883:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828037.152917:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043209088/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828037.152949:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828037.152953:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828037.152955:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828037.152960:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10103 00000100:00080000:3.0:1787828037.152966:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828037.152971:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828037.152980:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828037.152995:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043209216/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828037.153008:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828037.153014:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:2.0:1787828037.153015:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043209088:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787828037.153017:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828037.153048:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10103 00000100:00080000:3.0:1787828037.153054:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828037.153059:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828037.153065:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828037.153085:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043209344/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828037.153150:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828037.153199:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828037/real 1787828037] req@ffff9181c5637100 x1874663043209088/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828062 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828037.153221:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043209088/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828062 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828037.153241:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828037.153246:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828037.153271:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043209088:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828037.153302:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043209216:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828037.153326:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828037.153340:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828037/real 1787828037] req@ffff9181c5637480 x1874663043209216/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828062 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828037.153353:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043209216/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828062 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828037.153363:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828037.153370:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828037.153377:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043209216:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828037.153390:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043209344:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828037.153421:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828037.153439:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828037/real 1787828037] req@ffff9181c5636a00 x1874663043209344/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828062 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828037.153456:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043209344/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828062 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828037.153464:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828037.153468:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828037.153471:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043209344:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828037.216916:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828038.240941:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828039.264995:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828040.289069:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828041.312965:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828042.272887:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828042.272903:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828042.272910:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828042.272926:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10108 00000100:00080000:3.0:1787828042.272942:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828042.272955:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828042.273038:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043209472/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828042.273096:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828042.273103:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828042.273107:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828042.273116:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10108 00000100:00080000:3.0:1787828042.273123:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828042.273130:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828042.273148:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828042.273177:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043209472:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828042.273181:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634e00 x1874663043209600/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828042.273210:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828042.273217:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828042.273220:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828042.273227:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10108 00000100:00080000:3.0:1787828042.273233:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828042.273240:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828042.273248:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828042.273267:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043209728/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828042.273293:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828042.273358:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828042/real 1787828042] req@ffff9181c5634380 x1874663043209472/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828067 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828042.273391:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043209472/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828067 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828042.273414:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828042.273423:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828042.273457:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043209472:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828042.273506:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043209600:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828042.273559:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828042.273585:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828042/real 1787828042] req@ffff9181c5634e00 x1874663043209600/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828067 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828042.273606:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634e00 x1874663043209600/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828067 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828042.273623:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828042.273632:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828042.273640:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043209600:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828042.273654:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043209728:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828042.273705:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828042.273734:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828042/real 1787828042] req@ffff9181c5635500 x1874663043209728/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828067 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828042.273801:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043209728/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828067 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828042.273813:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828042.273817:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828042.273822:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043209728:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828042.337034:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828043.361039:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828044.384949:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828045.409004:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828046.432943:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828047.392865:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828047.392880:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828047.392885:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828047.392899:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10114 00000100:00080000:3.0:1787828047.392913:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828047.392923:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828047.393004:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635f80 x1874663043209856/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828047.393042:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828047.393048:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828047.393050:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828047.393056:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10114 00000100:00080000:3.0:1787828047.393061:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828047.393067:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828047.393080:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828047.393101:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043209984/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828047.393117:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043209856:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787828047.393122:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828047.393133:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828047.393136:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828047.393143:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10114 00000100:00080000:3.0:1787828047.393147:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828047.393152:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828047.393158:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828047.393177:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634e00 x1874663043210112/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828047.393211:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828047.393260:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828047/real 1787828047] req@ffff9181c5635f80 x1874663043209856/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828072 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828047.393283:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635f80 x1874663043209856/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828072 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828047.393296:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828047.393302:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828047.393322:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043209856:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828047.393352:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043209984:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828047.393387:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828047.393409:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828047/real 1787828047] req@ffff9181c5635500 x1874663043209984/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828072 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828047.393425:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043209984/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828072 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828047.393444:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828047.393449:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828047.393453:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043209984:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828047.393460:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043210112:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828047.393480:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828047.393491:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828047/real 1787828047] req@ffff9181c5634e00 x1874663043210112/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828072 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828047.393505:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634e00 x1874663043210112/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828072 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828047.393515:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828047.393519:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828047.393523:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043210112:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828047.457027:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828048.480976:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828049.505071:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828050.529103:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828051.553005:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787828052.000912:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-client' failed (-108). 10000000:00020000:1.0:1787828052.000937:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:1.0:1787828052.000943:0:159735:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:1.0:1787828052.000945:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff9181c5620000 from 7 10000000:01000000:1.0:1787828052.000947:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00010000:00010000:1.0:1787828052.000988:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f6f0 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828052.000997:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f6f0 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828052.001013:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f6f0 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787828052.001034:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215dc00 going to sleep for 0 seconds 00000100:00100000:1.0:1787828052.001043:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043210240:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787828052.001054:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f6f0 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828052.001060:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f6f0 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828052.001064:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f6f0 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828052.001069:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f6f0 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828052.001074:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f6f0 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828052.001079:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4e00 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787828052.001081:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f6f0 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828052.001086:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f6f0 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787828052.001090:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f6f0 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787828052.001094:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:1.0:1787828052.001097:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787828052.001101:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f6f0 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828052.001106:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f6f0 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787828052.001121:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787828052.001123:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:0.0:1787828052.001217:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828052.001223:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828052.001225:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828052.001231:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10119 00000100:00080000:0.0:1787828052.001240:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828052.001244:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787828052.001263:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7800 x1874663043210368/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828052.001287:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828052.001290:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828052.001292:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828052.001295:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10119 00000100:00080000:0.0:1787828052.001297:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828052.001300:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828052.001307:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828052.001316:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4e00 x1874663043210496/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828052.001324:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828052.001327:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828052.001328:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828052.001331:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10119 00000100:00080000:0.0:1787828052.001332:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1787828052.001336:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043210368:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787828052.001338:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828052.001346:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828052.001354:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5500 x1874663043210624/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828052.001404:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828052.001436:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828052/real 1787828052] req@ffff9181c55a7800 x1874663043210368/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828077 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828052.001448:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7800 x1874663043210368/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828077 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828052.001455:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828052.001459:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828052.001468:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043210368:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828052.001473:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043210496:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828052.001489:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828052.001498:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828052/real 1787828052] req@ffff9181c55a4e00 x1874663043210496/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828077 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828052.001506:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4e00 x1874663043210496/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828077 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828052.001512:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828052.001514:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828052.001517:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043210496:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828052.001535:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043210624:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828052.001547:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828052.001553:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828052/real 1787828052] req@ffff9181c55a5500 x1874663043210624/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828077 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828052.001561:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5500 x1874663043210624/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828077 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828052.001566:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828052.001568:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828052.001570:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043210624:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828052.576865:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828053.601038:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828054.625054:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828055.649031:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828056.673001:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828057.120973:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828057.120985:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828057.120991:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828057.121004:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10123 00000100:00080000:0.0:1787828057.121019:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828057.121029:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787828057.121081:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5180 x1874663043210752/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828057.121120:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828057.121126:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828057.121129:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828057.121137:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10123 00000100:00080000:0.0:1787828057.121141:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828057.121146:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828057.121158:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828057.121175:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4380 x1874663043210880/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828057.121192:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828057.121200:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:2.0:1787828057.121202:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043210752:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787828057.121204:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828057.121220:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10123 00000100:00080000:0.0:1787828057.121225:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828057.121230:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828057.121237:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828057.121251:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5c00 x1874663043211008/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828057.121325:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828057.121402:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828057/real 1787828057] req@ffff9181c55a5180 x1874663043210752/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828082 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828057.121434:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5180 x1874663043210752/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828082 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828057.121448:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828057.121454:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828057.121477:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043210752:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828057.121529:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043210880:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828057.121568:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828057.121586:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828057/real 1787828057] req@ffff9181c55a4380 x1874663043210880/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828082 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828057.121603:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4380 x1874663043210880/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828082 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828057.121614:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828057.121619:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828057.121623:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043210880:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828057.121631:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043211008:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828057.121652:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828057.121664:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828057/real 1787828057] req@ffff9181c55a5c00 x1874663043211008/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828082 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828057.121692:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5c00 x1874663043211008/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828082 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828057.121712:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828057.121716:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828057.121721:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043211008:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828057.697009:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828058.720961:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828059.745064:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828060.769066:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828061.793018:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828062.240943:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828062.240959:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828062.240965:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828062.240992:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10128 00000100:00080000:3.0:1787828062.241006:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828062.241015:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828062.241074:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043211136/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828062.241102:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828062.241107:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828062.241109:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828062.241113:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10128 00000100:00080000:3.0:1787828062.241117:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828062.241122:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828062.241132:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828062.241162:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043211264/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828062.241172:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043211136:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787828062.241177:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828062.241182:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828062.241184:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828062.241188:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10128 00000100:00080000:3.0:1787828062.241191:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828062.241195:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828062.241199:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828062.241210:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043211392/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828062.241256:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828062.241305:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828062/real 1787828062] req@ffff9181c5634380 x1874663043211136/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828087 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828062.241323:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043211136/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828087 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828062.241334:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828062.241338:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828062.241352:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043211136:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828062.241382:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043211264:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828062.241417:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828062.241440:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828062/real 1787828062] req@ffff9181c5636a00 x1874663043211264/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828087 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828062.241454:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043211264/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828087 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828062.241462:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828062.241465:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828062.241469:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043211264:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828062.241475:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043211392:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828062.241496:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828062.241505:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828062/real 1787828062] req@ffff9181c5637480 x1874663043211392/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828087 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828062.241516:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043211392/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828087 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828062.241523:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828062.241526:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828062.241529:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043211392:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828062.817011:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828063.841002:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828064.865003:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828065.888935:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828066.912998:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828067.360925:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828067.360948:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828067.360954:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828067.360967:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10133 00000100:00080000:3.0:1787828067.360985:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828067.360998:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828067.361068:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043211520/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828067.361123:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828067.361128:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828067.361131:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828067.361137:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10133 00000100:00080000:3.0:1787828067.361143:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828067.361149:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828067.361161:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828067.361183:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043211648/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828067.361197:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828067.361202:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828067.361204:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828067.361209:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10133 00000100:00080000:3.0:1787828067.361212:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1787828067.361222:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043211520:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787828067.361224:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828067.361236:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828067.361261:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043211776/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828067.361345:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828067.361416:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828067/real 1787828067] req@ffff9181c5637100 x1874663043211520/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828092 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828067.361452:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043211520/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828092 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828067.361468:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828067.361474:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828067.361505:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043211520:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828067.361516:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043211648:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828067.361560:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828067.361596:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828067/real 1787828067] req@ffff9181c5637800 x1874663043211648/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828092 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828067.361625:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043211648/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828092 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828067.361643:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828067.361651:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828067.361658:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043211648:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828067.361700:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043211776:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828067.361792:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828067.361843:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828067/real 1787828067] req@ffff9181c5636d80 x1874663043211776/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828092 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828067.361881:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043211776/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828092 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828067.361898:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828067.361905:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828067.361912:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043211776:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828067.937063:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828068.961004:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828069.985905:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828071.008842:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828072.033101:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787828072.480976:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'params' failed (-108). 10000000:00020000:1.0:1787828072.480989:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:1.0:1787828072.480996:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:1.0:1787828072.480999:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff9181c87b19b8 from 1 10000000:01000000:1.0:1787828072.481003:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00010000:00010000:1.0:1787828072.481066:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f6f7 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828072.481089:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f6f7 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828072.481111:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f6f7 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787828072.481147:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215dc00 going to sleep for 0 seconds 00000100:00100000:1.0:1787828072.481165:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043211904:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787828072.481185:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f6f7 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828072.481196:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f6f7 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828072.481213:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f6f7 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828072.481223:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f6f7 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828072.481232:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f6f7 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828072.481240:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4e00 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787828072.481245:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f6f7 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828072.481255:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f6f7 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787828072.481263:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f6f7 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787828072.481271:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1787828072.481275:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787828072.481283:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f6f7 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828072.481291:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f6f7 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787828072.481316:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787828072.481320:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:1.0:1787828072.481348:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828072.481354:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828072.481359:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828072.481365:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10139 00000100:00080000:1.0:1787828072.481378:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828072.481385:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787828072.481409:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0e00 x1874663043212032/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828072.481448:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828072.481453:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828072.481455:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828072.481461:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10139 00000100:00080000:1.0:1787828072.481465:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828072.481469:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828072.481480:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828072.481495:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0000 x1874663043212160/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828072.481510:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828072.481515:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828072.481518:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828072.481527:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10139 00000100:00100000:2.0:1787828072.481530:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043212032:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787828072.481532:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828072.481546:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828072.481554:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828072.481572:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1180 x1874663043212288/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828072.481641:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828072.481697:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828072/real 1787828072] req@ffff9181c35b0e00 x1874663043212032/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828097 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828072.481720:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0e00 x1874663043212032/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828097 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828072.481810:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828072.481819:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828072.481838:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043212032:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828072.481853:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043212160:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828072.481894:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828072.481918:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828072/real 1787828072] req@ffff9181c35b0000 x1874663043212160/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828097 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828072.481944:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0000 x1874663043212160/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828097 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828072.481962:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828072.481969:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828072.481976:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043212160:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828072.482013:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043212288:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828072.482056:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828072.482074:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828072/real 1787828072] req@ffff9181c35b1180 x1874663043212288/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828097 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828072.482090:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1180 x1874663043212288/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828097 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828072.482100:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828072.482105:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828072.482109:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043212288:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828073.056932:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828074.081006:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828075.105061:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828076.128977:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828077.153037:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828077.600930:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828077.600945:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828077.600951:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828077.600967:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10144 00000100:00080000:1.0:1787828077.600982:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828077.600994:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787828077.601063:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0380 x1874663043212416/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828077.601130:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828077.601139:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828077.601143:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828077.601152:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10144 00000100:00080000:1.0:1787828077.601160:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828077.601167:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828077.601183:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828077.601205:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3100 x1874663043212544/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828077.601224:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043212416:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787828077.601228:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828077.601251:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828077.601255:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828077.601274:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10144 00000100:00080000:1.0:1787828077.601281:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828077.601289:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828077.601297:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828077.601316:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1c00 x1874663043212672/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828077.601393:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043212544:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828077.601432:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043212672:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828078.176969:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828079.201037:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828080.225041:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1787828080.673273:0:2378:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:0.0:1787828080.673365:0:2378:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1787828080.673397:0:2378:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1787828080.673473:0:2378:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1787828080.673496:0:2378:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:2.0:1787828080.673572:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828077/real 1787828080] req@ffff9181c35b3100 x1874663043212544/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828102 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828080.673620:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3100 x1874663043212544/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828102 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828080.673642:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828080.673652:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828080.673662:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043212544:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828080.673678:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828077/real 1787828080] req@ffff9181c35b1c00 x1874663043212672/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828102 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828080.673704:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1c00 x1874663043212672/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828102 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828080.673789:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828080.673800:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828080.673810:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043212672:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828080.673833:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828077/real 1787828080] req@ffff9181c35b0380 x1874663043212416/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828102 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828080.673866:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0380 x1874663043212416/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828102 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828080.673892:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828080.673902:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828080.673942:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043212416:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1787828081.248991:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828082.273018:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828082.721031:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828082.721045:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828082.721068:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828082.721083:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10149 00000100:00080000:0.0:1787828082.721098:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828082.721109:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787828082.721172:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4700 x1874663043212800/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828082.721224:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828082.721232:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828082.721253:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828082.721270:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10149 00000100:00080000:0.0:1787828082.721277:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828082.721284:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828082.721301:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828082.721327:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5f80 x1874663043212928/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828082.721348:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828082.721362:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:2.0:1787828082.721363:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043212800:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787828082.721368:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828082.721392:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10149 00000100:00080000:0.0:1787828082.721401:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828082.721412:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828082.721424:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828082.721466:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6d80 x1874663043213056/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828082.721534:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828082.721598:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828082/real 1787828082] req@ffff9181c55a4700 x1874663043212800/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828107 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828082.721627:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4700 x1874663043212800/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828107 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828082.721660:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828082.721667:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828082.721695:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043212800:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828082.721807:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043212928:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828082.721864:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828082.721887:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828082/real 1787828082] req@ffff9181c55a5f80 x1874663043212928/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828107 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828082.721916:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5f80 x1874663043212928/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828107 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828082.721929:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828082.721934:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828082.721941:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043212928:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828082.721951:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043213056:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828082.721985:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828082.722004:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828082/real 1787828082] req@ffff9181c55a6d80 x1874663043213056/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828107 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828082.722023:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6d80 x1874663043213056/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828107 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828082.722036:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828082.722041:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828082.722046:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043213056:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828083.296884:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828084.320850:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828085.344968:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828086.368976:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828087.392993:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828087.840943:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828087.840965:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828087.840973:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828087.840993:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10154 00000100:00080000:3.0:1787828087.841011:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828087.841023:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828087.841087:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043213184/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828087.841130:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828087.841135:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828087.841138:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828087.841146:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10154 00000100:00080000:3.0:1787828087.841151:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828087.841157:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828087.841173:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828087.841200:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043213184:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828087.841202:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634a80 x1874663043213312/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828087.841234:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828087.841241:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828087.841246:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828087.841257:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10154 00000100:00080000:3.0:1787828087.841265:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828087.841274:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828087.841302:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000800:00000100:2.0:1787828087.841319:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828087.841328:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043213440/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828087.841389:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828087/real 1787828087] req@ffff9181c5635c00 x1874663043213184/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828112 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828087.841418:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043213184/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828112 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828087.841434:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828087.841440:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828087.841466:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043213184:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828087.841499:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043213312:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828087.841528:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828087.841546:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828087/real 1787828087] req@ffff9181c5634a80 x1874663043213312/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828112 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828087.841563:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634a80 x1874663043213312/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828112 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828087.841576:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828087.841581:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828087.841587:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043213312:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828087.841596:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043213440:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828087.841634:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828087.841649:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828087/real 1787828087] req@ffff9181c5634000 x1874663043213440/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828112 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828087.841702:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043213440/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828112 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828087.841713:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828087.841717:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828087.841722:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043213440:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787828088.416949:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828089.441038:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828090.464995:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828091.488975:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828092.512895:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828092.960869:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828092.960880:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828092.960886:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828092.960902:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10159 00000100:00080000:3.0:1787828092.960916:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828092.960925:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 10000000:01000000:1.0:1787828092.960947:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-sptlrpc' failed (-108). 10000000:00020000:1.0:1787828092.960956:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00100000:3.0:1787828092.960974:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637b80 x1874663043213568/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828092.961015:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828092.961020:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828092.961022:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828092.961027:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10159 00000100:00080000:3.0:1787828092.961031:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828092.961037:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828092.961048:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828092.961064:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043213696/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828092.961077:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828092.961081:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828092.961083:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828092.961096:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10159 00000100:00080000:3.0:1787828092.961099:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828092.961103:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828092.961109:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828092.961119:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634a80 x1874663043213824/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828092.961153:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043213568:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787828092.961300:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828092.961377:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828092/real 1787828092] req@ffff9181c5637b80 x1874663043213568/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828117 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828092.961416:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637b80 x1874663043213568/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828117 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828092.961433:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828092.961440:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828092.961462:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043213568:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828092.961474:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043213696:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828092.961532:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828092.961560:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828092/real 1787828092] req@ffff9181c5634000 x1874663043213696/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828117 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828092.961588:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043213696/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828117 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828092.961615:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828092.961620:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828092.961625:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043213696:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828092.961713:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043213824:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828092.961822:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828092.961841:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828092/real 1787828092] req@ffff9181c5634a80 x1874663043213824/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828117 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828092.961875:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634a80 x1874663043213824/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828117 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828092.961891:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828092.961895:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828092.961899:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043213824:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828093.536973:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:0.0:1787828094.240992:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:0.0:1787828094.241006:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff9181c5620000 from 341 10000000:01000000:0.0:1787828094.241033:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:0.0:1787828094.241139:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f6fe lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828094.241158:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f6fe lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828094.241177:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f6fe lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787828094.241210:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d01a5180 going to sleep for 0 seconds 00000100:00100000:0.0:1787828094.241230:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043213952:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:0.0:1787828094.241256:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f6fe lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828094.241268:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f6fe lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828094.241277:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f6fe lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828094.241286:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f6fe lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828094.241296:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f6fe lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828094.241313:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0512800 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:0.0:1787828094.241319:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f6fe lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828094.241329:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f6fe lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1787828094.241338:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f6fe lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787828094.241346:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:0.0:1787828094.241350:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787828094.241360:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f6fe lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828094.241368:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f6fe lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787828094.241403:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:0.0:1787828094.241407:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:3.0:1787828094.241523:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828094.241533:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828094.241539:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828094.241550:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10164 00000100:00080000:3.0:1787828094.241567:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828094.241576:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828094.241613:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043214080/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828094.241676:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828094.241682:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828094.241685:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828094.241691:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10164 00000100:00080000:3.0:1787828094.241696:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828094.241701:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828094.241717:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828094.241739:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043214208/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828094.241805:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828094.241811:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828094.241814:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828094.241821:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10164 00000100:00080000:3.0:1787828094.241827:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828094.241836:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787828094.241840:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043214080:192.168.203.147@tcp:250:kworker.0 00000008:00100000:3.0:1787828094.241844:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828094.241868:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043214336/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828094.241981:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828094.242035:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828094/real 1787828094] req@ffff9181c5635c00 x1874663043214080/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828119 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828094.242059:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043214080/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828119 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828094.242074:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828094.242080:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828094.242101:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043214080:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828094.242133:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043214208:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828094.242174:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828094.242200:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828094/real 1787828094] req@ffff9181c5636d80 x1874663043214208/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828119 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828094.242224:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043214208/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828119 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828094.242241:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828094.242247:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828094.242254:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043214208:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828094.242267:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043214336:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828094.242304:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828094.242325:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828094/real 1787828094] req@ffff9181c5637800 x1874663043214336/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828119 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828094.242368:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043214336/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828119 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828094.242384:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828094.242390:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828094.242396:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043214336:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787828094.560989:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828095.585134:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828096.608980:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828097.632966:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828098.656845:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828099.616894:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828099.616906:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828099.616911:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828099.616925:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10165 00000100:00080000:0.0:1787828099.616939:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828099.616948:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787828099.616993:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7480 x1874663043214464/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828099.617056:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828099.617060:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828099.617063:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828099.617068:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10165 00000100:00080000:0.0:1787828099.617073:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828099.617077:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828099.617089:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828099.617106:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5880 x1874663043214592/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828099.617118:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828099.617122:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828099.617125:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828099.617129:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10165 00000100:00080000:0.0:1787828099.617132:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828099.617137:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828099.617143:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828099.617155:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7100 x1874663043214720/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828099.617188:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043214464:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787828099.617279:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828099.617338:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828099/real 1787828099] req@ffff9181c55a7480 x1874663043214464/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828124 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828099.617361:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7480 x1874663043214464/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828124 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828099.617377:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828099.617384:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828099.617413:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043214464:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828099.617425:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043214592:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828099.617489:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828099.617527:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828099/real 1787828099] req@ffff9181c55a5880 x1874663043214592/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828124 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828099.617563:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5880 x1874663043214592/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828124 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828099.617581:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828099.617619:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828099.617629:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043214592:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828099.617682:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043214720:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828099.617728:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828099.617802:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828099/real 1787828099] req@ffff9181c55a7100 x1874663043214720/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828124 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828099.617822:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7100 x1874663043214720/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828124 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828099.617835:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828099.617841:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828099.617849:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043214720:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828099.680997:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828100.704883:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828101.728924:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828102.752978:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828103.776920:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828104.736902:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828104.736918:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828104.736925:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828104.736941:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10171 00000100:00080000:0.0:1787828104.736959:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828104.736972:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787828104.737029:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7b80 x1874663043214848/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828104.737087:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828104.737092:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828104.737095:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828104.737101:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10171 00000100:00080000:0.0:1787828104.737107:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828104.737112:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828104.737124:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828104.737143:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6680 x1874663043214976/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828104.737156:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828104.737161:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828104.737163:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828104.737167:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10171 00000100:00080000:0.0:1787828104.737171:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828104.737176:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828104.737195:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828104.737211:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7100 x1874663043215104/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828104.737276:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043214848:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787828104.737373:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828104.737431:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828104/real 1787828104] req@ffff9181c55a7b80 x1874663043214848/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828129 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828104.737457:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7b80 x1874663043214848/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828129 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828104.737484:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828104.737489:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828104.737509:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043214848:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828104.737519:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043214976:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828104.737547:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828104.737602:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828104/real 1787828104] req@ffff9181c55a6680 x1874663043214976/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828129 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828104.737631:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6680 x1874663043214976/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828129 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828104.737643:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828104.737648:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828104.737652:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043214976:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828104.737659:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043215104:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828104.737685:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828104.737697:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828104/real 1787828104] req@ffff9181c55a7100 x1874663043215104/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828129 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828104.737715:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7100 x1874663043215104/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828129 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828104.737796:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828104.737805:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828104.737812:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043215104:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787828104.800960:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828105.825122:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828106.848905:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828107.873000:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828108.896946:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828109.856800:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828109.856808:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828109.856811:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828109.856819:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10176 00000100:00080000:0.0:1787828109.856827:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828109.856833:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787828109.856864:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5880 x1874663043215232/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828109.856883:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828109.856886:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828109.856887:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828109.856890:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10176 00000100:00080000:0.0:1787828109.856893:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828109.856895:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828109.856903:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828109.856911:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7480 x1874663043215360/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828109.856919:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828109.856921:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828109.856922:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828109.856925:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10176 00000100:00080000:0.0:1787828109.856927:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1787828109.856932:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043215232:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787828109.856933:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828109.856940:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828109.856948:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6d80 x1874663043215488/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828109.857001:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828109.857035:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828109/real 1787828109] req@ffff9181c55a5880 x1874663043215232/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828134 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828109.857054:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5880 x1874663043215232/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828134 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828109.857066:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828109.857071:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828109.857085:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043215232:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828109.857093:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043215360:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828109.857120:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828109.857135:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828109/real 1787828109] req@ffff9181c55a7480 x1874663043215360/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828134 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828109.857149:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7480 x1874663043215360/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828134 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828109.857165:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828109.857168:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828109.857171:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043215360:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828109.857196:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043215488:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828109.857217:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828109.857230:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828109/real 1787828109] req@ffff9181c55a6d80 x1874663043215488/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828134 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828109.857243:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6d80 x1874663043215488/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828134 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828109.857251:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828109.857255:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828109.857258:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043215488:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828109.920960:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828110.945017:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828111.968895:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828112.992942:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828114.017033:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:3.0:1787828114.464942:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-cliir' failed (-108). 10000000:00000400:3.0:1787828114.464959:0:159735:0:(mgc_request.c:1900:mgc_process_log()) MGC192.168.203.147@tcp: IR log lustre-cliir failed, not fatal: rc = -108 10000000:01000000:3.0:1787828114.464972:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:3.0:1787828114.464976:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff9181c5620000 from 52 10000000:01000000:3.0:1787828114.464997:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:3.0:1787828114.465098:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f705 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787828114.465117:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f705 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787828114.465136:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f705 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787828114.465171:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d225f180 going to sleep for 0 seconds 00000100:00100000:3.0:1787828114.465190:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043215616:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:3.0:1787828114.465211:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f705 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787828114.465224:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f705 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787828114.465243:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f705 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787828114.465255:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f705 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787828114.465265:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f705 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787828114.465272:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3480000 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:3.0:1787828114.465277:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f705 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787828114.465285:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f705 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1787828114.465291:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f705 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787828114.465297:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:3.0:1787828114.465300:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787828114.465307:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f705 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787828114.465314:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f705 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787828114.465357:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:3.0:1787828114.465361:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:0.0:1787828114.465471:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828114.465478:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828114.465495:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828114.465504:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10181 00000100:00080000:0.0:1787828114.465515:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828114.465522:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787828114.465559:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5f80 x1874663043215744/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828114.465593:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828114.465597:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828114.465602:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828114.465608:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10181 00000100:00080000:0.0:1787828114.465611:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828114.465616:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828114.465628:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828114.465641:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4700 x1874663043215872/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828114.465656:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043215744:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787828114.465657:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828114.465675:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828114.465678:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828114.465684:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10181 00000100:00080000:0.0:1787828114.465687:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828114.465692:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828114.465698:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828114.465710:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5c00 x1874663043216000/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828114.465811:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828114.465868:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828114/real 1787828114] req@ffff9181c55a5f80 x1874663043215744/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828139 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828114.465908:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5f80 x1874663043215744/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828139 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828114.465920:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828114.465926:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828114.465952:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043215744:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828114.465993:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043215872:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828114.466025:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828114.466038:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828114/real 1787828114] req@ffff9181c55a4700 x1874663043215872/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828139 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828114.466051:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4700 x1874663043215872/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828139 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828114.466071:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828114.466074:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828114.466078:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043215872:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828114.466083:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043216000:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828114.466099:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828114.466108:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828114/real 1787828114] req@ffff9181c55a5c00 x1874663043216000/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828139 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828114.466120:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5c00 x1874663043216000/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828139 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828114.466127:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828114.466130:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828114.466133:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043216000:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787828115.040976:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828116.064847:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828117.088985:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828118.113070:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828119.136971:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828119.584952:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828119.584969:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828119.584976:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828119.584993:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10186 00000100:00080000:3.0:1787828119.585010:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828119.585021:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828119.585085:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043216128/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828119.585130:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828119.585136:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828119.585139:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828119.585144:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10186 00000100:00080000:3.0:1787828119.585149:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828119.585154:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828119.585167:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828119.585183:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043216256/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828119.585197:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828119.585202:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828119.585205:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828119.585210:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10186 00000100:00080000:3.0:1787828119.585216:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828119.585221:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828119.585228:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828119.585241:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043216384/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828119.585263:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043216128:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787828119.585363:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828119.585413:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828119/real 1787828119] req@ffff9181c5637100 x1874663043216128/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828144 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828119.585437:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043216128/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828144 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828119.585491:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828119.585498:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828119.585520:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043216128:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828119.585531:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043216256:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828119.585579:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828119.585606:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828119/real 1787828119] req@ffff9181c5637480 x1874663043216256/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828144 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828119.585621:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043216256/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828144 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828119.585632:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828119.585636:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828119.585641:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043216256:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828119.585678:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043216384:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828119.585715:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828119.585738:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828119/real 1787828119] req@ffff9181c5636a00 x1874663043216384/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828144 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828119.585822:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043216384/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828144 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828119.585833:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828119.585837:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828119.585842:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043216384:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828120.161030:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828121.184949:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828122.208963:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828123.233038:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828124.256903:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828124.704915:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828124.704933:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828124.704939:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828124.704957:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10191 00000100:00080000:0.0:1787828124.704976:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828124.704986:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787828124.705038:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4380 x1874663043216512/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828124.705089:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828124.705098:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828124.705102:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828124.705110:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10191 00000100:00080000:0.0:1787828124.705119:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828124.705128:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828124.705143:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828124.705168:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5180 x1874663043216640/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828124.705185:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828124.705190:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828124.705194:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828124.705201:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10191 00000100:00080000:0.0:1787828124.705207:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828124.705213:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828124.705221:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828124.705240:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043216512:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787828124.705257:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5500 x1874663043216768/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828124.705399:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828124.705496:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828124/real 1787828124] req@ffff9181c55a4380 x1874663043216512/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828149 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828124.705523:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4380 x1874663043216512/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828149 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828124.705540:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828124.705547:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828124.705574:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043216512:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828124.705585:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043216640:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828124.705623:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828124.705651:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828124/real 1787828124] req@ffff9181c55a5180 x1874663043216640/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828149 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828124.705686:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5180 x1874663043216640/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828149 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828124.705696:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828124.705701:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828124.705705:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043216640:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828124.705739:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043216768:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828124.705817:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828124.705833:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828124/real 1787828124] req@ffff9181c55a5500 x1874663043216768/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828149 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828124.705849:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5500 x1874663043216768/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828149 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828124.705859:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828124.705864:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828124.705868:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043216768:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787828125.280937:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828126.304895:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828127.328886:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828128.352845:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828129.376999:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828129.824877:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828129.824893:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828129.824899:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828129.824916:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10196 00000100:00080000:0.0:1787828129.824933:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828129.824945:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787828129.825023:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4e00 x1874663043216896/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828129.825071:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828129.825079:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828129.825083:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828129.825092:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10196 00000100:00080000:0.0:1787828129.825100:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828129.825108:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828129.825128:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828129.825152:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043216896:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787828129.825156:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7800 x1874663043217024/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828129.825176:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828129.825181:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828129.825185:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828129.825192:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10196 00000100:00080000:0.0:1787828129.825198:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828129.825204:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828129.825213:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828129.825233:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5500 x1874663043217152/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828129.825270:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828129.825338:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828129/real 1787828129] req@ffff9181c55a4e00 x1874663043216896/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828154 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828129.825396:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4e00 x1874663043216896/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828154 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828129.825419:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828129.825428:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828129.825453:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043216896:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828129.825502:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043217024:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828129.825557:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828129.825585:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828129/real 1787828129] req@ffff9181c55a7800 x1874663043217024/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828154 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828129.825602:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7800 x1874663043217024/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828154 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828129.825613:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828129.825617:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828129.825621:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043217024:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828129.825629:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043217152:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828129.825655:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828129.825668:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828129/real 1787828129] req@ffff9181c55a5500 x1874663043217152/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828154 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828129.825682:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5500 x1874663043217152/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828154 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828129.825700:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828129.825704:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828129.825708:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043217152:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1787828130.400894:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828131.424995:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828132.448972:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828133.473110:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828134.496868:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828134.944834:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828134.944842:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828134.944845:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828134.944855:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10201 00000100:00080000:3.0:1787828134.944865:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828134.944871:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 10000000:01000000:0.0:1787828134.944899:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-client' failed (-108). 10000000:00020000:0.0:1787828134.944904:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:0.0:1787828134.944908:0:159735:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:0.0:1787828134.944910:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff9181c5620000 from 7 10000000:01000000:0.0:1787828134.944915:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00000100:00100000:3.0:1787828134.944915:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043217280/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828134.944938:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828134.944941:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828134.944943:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828134.944957:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10201 00010000:00010000:0.0:1787828134.944958:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f70c lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787828134.944962:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828134.944969:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828134.944976:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:0.0:1787828134.944980:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f70c lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787828134.944989:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634e00 x1874663043217536/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:0.0:1787828134.944991:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f70c lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787828134.945022:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043217280:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787828134.945022:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828134.945032:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828134.945035:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1787828134.945036:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d01a5180 going to sleep for 0 seconds 00000100:00080000:3.0:1787828134.945040:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10201 00000100:00080000:3.0:1787828134.945044:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1787828134.945046:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043217408:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00080000:3.0:1787828134.945051:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828134.945057:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:0.0:1787828134.945062:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f70c lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787828134.945068:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043217664/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:0.0:1787828134.945071:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f70c lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828134.945090:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f70c lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000800:00000100:2.0:1787828134.945095:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00010000:00010000:0.0:1787828134.945100:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f70c lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828134.945107:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f70c lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828134.945112:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0513400 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:0.0:1787828134.945114:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f70c lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828134.945120:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f70c lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1787828134.945135:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f70c lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787828134.945138:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:0.0:1787828134.945142:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787828134.945148:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f70c lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787828134.945151:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828134/real 1787828134] req@ffff9181c5634380 x1874663043217280/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828159 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:0.0:1787828134.945156:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f70c lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787828134.945186:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043217280/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828159 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 10000000:01000000:0.0:1787828134.945201:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:0.0:1787828134.945206:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:CONNECTING 00000100:00080000:2.0:1787828134.945209:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828134.945228:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828134.945242:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043217280:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787828134.945248:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828134.945252:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828134.945254:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828134.945270:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10206 00000100:00080000:3.0:1787828134.945274:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828134.945277:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787828134.945284:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043217536:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828134.945291:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635f80 x1874663043217792/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828134.945309:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828134.945323:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828134/real 1787828134] req@ffff9181c5634e00 x1874663043217536/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828159 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828134.945338:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634e00 x1874663043217536/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828159 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828134.945359:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828134.945375:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828134.945377:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043217536:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828134.945381:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043217664:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828134.945401:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828134.945413:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828134/real 1787828134] req@ffff9181c5635500 x1874663043217664/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828159 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828134.945426:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043217664/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828159 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828134.945445:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828134.945448:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828134.945452:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043217664:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828134.945465:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043217792:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787828134.945479:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828134.945486:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828134/real 1787828134] req@ffff9181c5635f80 x1874663043217792/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828159 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828134.945494:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635f80 x1874663043217792/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828159 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828134.945499:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828134.945501:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828134.945510:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043217792:192.168.203.147@tcp:250:kworker.0 00010000:00010000:2.0:1787828135.520896:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828136.544949:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828137.568928:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828138.593078:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828139.616924:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828140.064887:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828140.064895:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828140.064899:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828140.064906:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10206 00000100:00080000:2.0:1787828140.064916:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828140.064922:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787828140.064954:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ce00 x1874663043217920/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828140.064979:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828140.064981:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828140.064983:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828140.064986:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10206 00000100:00080000:2.0:1787828140.064988:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828140.064991:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828140.065000:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828140.065009:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d880 x1874663043218048/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828140.065030:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828140.065032:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828140.065033:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828140.065037:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10206 00000100:00080000:2.0:1787828140.065040:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828140.065046:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828140.065058:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828140.065060:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043217920:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828140.065071:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007fb80 x1874663043218176/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828140.065177:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828140.065233:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828140/real 1787828140] req@ffff9181e007ce00 x1874663043217920/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828165 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828140.065259:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ce00 x1874663043217920/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828165 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828140.065275:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828140.065281:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828140.065335:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043217920:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828140.065371:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043218048:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828140.065398:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828140.065409:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828140/real 1787828140] req@ffff9181e007d880 x1874663043218048/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828165 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828140.065421:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d880 x1874663043218048/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828165 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828140.065428:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828140.065432:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828140.065435:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043218048:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828140.065440:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043218176:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828140.065454:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828140.065461:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828140/real 1787828140] req@ffff9181e007fb80 x1874663043218176/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828165 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828140.065471:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007fb80 x1874663043218176/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828165 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828140.065478:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828140.065480:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828140.065483:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043218176:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828140.640842:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828141.664928:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828142.688977:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828143.712951:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828144.736919:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828145.184921:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828145.184933:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828145.184938:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828145.184964:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10211 00000100:00080000:1.0:1787828145.184975:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828145.184983:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787828145.185020:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1500 x1874663043218304/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828145.185049:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828145.185053:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828145.185055:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828145.185059:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10211 00000100:00080000:1.0:1787828145.185064:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828145.185069:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828145.185080:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828145.185096:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0380 x1874663043218432/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828145.185108:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787828145.185111:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043218304:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787828145.185129:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828145.185134:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828145.185144:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10211 00000100:00080000:1.0:1787828145.185148:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828145.185153:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828145.185158:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828145.185172:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1c00 x1874663043218560/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828145.185310:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043218432:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828145.185344:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043218560:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787828145.760883:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828146.784957:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828147.808903:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1787828148.257361:0:2379:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:0.0:1787828148.257464:0:2379:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1787828148.257492:0:2379:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1787828148.257556:0:2379:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1787828148.257568:0:2379:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:3.0:1787828148.257649:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828145/real 1787828148] req@ffff9181c35b0380 x1874663043218432/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828170 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828148.257702:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0380 x1874663043218432/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828170 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828148.257727:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828148.257738:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828148.257795:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043218432:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828148.257816:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828145/real 1787828148] req@ffff9181c35b1c00 x1874663043218560/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828170 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828148.257848:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1c00 x1874663043218560/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828170 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828148.257865:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828148.257874:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828148.257882:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043218560:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828148.257902:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828145/real 1787828148] req@ffff9181c35b1500 x1874663043218304/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828170 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828148.257929:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1500 x1874663043218304/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828170 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828148.257948:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828148.257956:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828148.257996:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043218304:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1787828148.832948:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828149.856921:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828150.304960:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828150.304978:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828150.304986:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828150.305007:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10216 00000100:00080000:0.0:1787828150.305027:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828150.305042:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787828150.305110:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5180 x1874663043218688/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828150.305164:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828150.305171:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828150.305176:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828150.305184:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10216 00000100:00080000:0.0:1787828150.305192:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828150.305203:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828150.305221:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828150.305277:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043218688:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787828150.305281:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4380 x1874663043218816/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828150.305312:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828150.305319:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828150.305323:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828150.305332:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10216 00000100:00080000:0.0:1787828150.305340:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828150.305348:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828150.305360:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828150.305383:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5c00 x1874663043218944/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828150.305415:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828150.305485:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828150/real 1787828150] req@ffff9181c55a5180 x1874663043218688/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828175 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828150.305523:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5180 x1874663043218688/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828175 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828150.305543:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828150.305551:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828150.305592:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043218688:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828150.305649:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043218816:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828150.305699:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828150.305733:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828150/real 1787828150] req@ffff9181c55a4380 x1874663043218816/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828175 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828150.305821:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4380 x1874663043218816/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828175 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828150.305840:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828150.305847:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828150.305858:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043218816:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828150.305871:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043218944:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828150.305921:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828150.305946:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828150/real 1787828150] req@ffff9181c55a5c00 x1874663043218944/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828175 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828150.305972:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5c00 x1874663043218944/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828175 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828150.305990:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828150.305997:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828150.306004:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043218944:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828150.880993:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828151.905022:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828152.929053:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828153.952953:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828154.976991:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828155.424972:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 10000000:01000000:0.0:1787828155.424974:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'params' failed (-108). 00000100:00080000:2.0:1787828155.424991:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 10000000:00020000:0.0:1787828155.424993:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00080000:2.0:1787828155.424998:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 10000000:01000000:0.0:1787828155.425006:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:0.0:1787828155.425012:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff9181c87b19b8 from 1 00000100:00080000:2.0:1787828155.425032:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10222 10000000:01000000:0.0:1787828155.425036:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00000100:00080000:2.0:1787828155.425052:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828155.425065:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787828155.425114:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ca80 x1874663043219200/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:0.0:1787828155.425140:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f713 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828155.425159:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f713 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787828155.425160:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828155.425173:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828155.425179:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828155.425188:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10222 00000100:00080000:2.0:1787828155.425193:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00010000:00010000:0.0:1787828155.425194:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f713 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787828155.425249:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828155.425271:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828155.425284:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d01a5180 going to sleep for 0 seconds 00000100:00100000:2.0:1787828155.425296:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e680 x1874663043219328/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828155.425302:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043219072:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00100000:3.0:1787828155.425306:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043219200:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787828155.425353:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:0.0:1787828155.425355:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f713 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787828155.425364:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828155.425370:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00010000:00010000:0.0:1787828155.425386:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f713 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787828155.425387:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10222 00010000:00010000:0.0:1787828155.425405:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f713 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787828155.425422:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00010000:00010000:0.0:1787828155.425428:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f713 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787828155.425435:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:3.0:1787828155.425448:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:2.0:1787828155.425451:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:0.0:1787828155.425471:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f713 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787828155.425484:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c000 x1874663043219456/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:0.0:1787828155.425485:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0513400 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:0.0:1787828155.425499:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f713 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787828155.425514:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828155/real 1787828155] req@ffff9181e007ca80 x1874663043219200/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828180 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:0.0:1787828155.425521:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f713 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1787828155.425553:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f713 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787828155.425564:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:0.0:1787828155.425572:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00000100:00100000:3.0:1787828155.425596:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ca80 x1874663043219200/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828180 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:0.0:1787828155.425602:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f713 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828155.425627:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f713 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787828155.425641:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828155.425652:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828155.425658:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043219200:192.168.203.147@tcp:250:kworker.0 10000000:01000000:0.0:1787828155.425662:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:0.0:1787828155.425670:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00100000:3.0:1787828155.425721:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043219328:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828155.425792:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1787828155.425798:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828155.425805:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828155.425808:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828155.425816:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10227 00000100:00080000:2.0:1787828155.425822:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828155.425828:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787828155.425847:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007dc00 x1874663043219584/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828155.425896:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828155/real 1787828155] req@ffff9181e007e680 x1874663043219328/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828180 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828155.425925:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e680 x1874663043219328/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828180 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828155.425966:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828155.425971:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828155.425975:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043219328:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828155.425983:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043219456:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828155.426010:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828155.426041:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828155/real 1787828155] req@ffff9181e007c000 x1874663043219456/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828180 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828155.426056:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c000 x1874663043219456/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828180 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828155.426066:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828155.426069:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828155.426073:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043219456:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828155.426092:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043219584:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1787828155.426112:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828155.426130:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828155/real 1787828155] req@ffff9181e007dc00 x1874663043219584/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828180 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828155.426145:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007dc00 x1874663043219584/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828180 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828155.426155:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828155.426158:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828155.426178:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043219584:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787828156.001109:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828157.024959:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828158.049072:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828159.073031:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828160.097046:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828160.544900:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828160.544909:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828160.544913:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828160.544932:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10227 00000100:00080000:0.0:1787828160.544941:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828160.544948:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787828160.544986:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4700 x1874663043219712/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828160.545024:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828160.545027:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828160.545028:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828160.545032:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10227 00000100:00080000:0.0:1787828160.545035:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828160.545039:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828160.545047:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828160.545061:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5f80 x1874663043219840/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828160.545061:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043219712:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787828160.545080:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828160.545083:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828160.545085:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828160.545087:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10227 00000100:00080000:0.0:1787828160.545090:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828160.545093:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828160.545096:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828160.545105:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6d80 x1874663043219968/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828160.545123:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828160.545152:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828160/real 1787828160] req@ffff9181c55a4700 x1874663043219712/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828185 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828160.545179:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4700 x1874663043219712/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828185 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828160.545198:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828160.545201:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828160.545211:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043219712:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828160.545245:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043219840:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828160.545268:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828160.545279:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828160/real 1787828160] req@ffff9181c55a5f80 x1874663043219840/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828185 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828160.545293:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5f80 x1874663043219840/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828185 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828160.545299:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828160.545301:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828160.545303:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043219840:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828160.545308:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043219968:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828160.545319:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828160.545325:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828160/real 1787828160] req@ffff9181c55a6d80 x1874663043219968/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828185 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828160.545333:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6d80 x1874663043219968/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828185 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828160.545338:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828160.545340:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828160.545342:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043219968:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828161.120903:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828162.144977:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828163.168973:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828164.192986:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828165.217084:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828165.664893:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828165.664906:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828165.664911:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828165.664924:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10232 00000100:00080000:1.0:1787828165.664937:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828165.664946:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787828165.664992:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3100 x1874663043220096/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828165.665028:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828165.665031:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828165.665033:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828165.665038:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10232 00000100:00080000:1.0:1787828165.665041:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828165.665046:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828165.665054:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828165.665066:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1180 x1874663043220224/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828165.665075:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828165.665079:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828165.665080:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828165.665083:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10232 00000100:00080000:1.0:1787828165.665086:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828165.665089:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828165.665093:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828165.665102:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0000 x1874663043220352/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828165.665146:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043220096:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1787828165.665218:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828165.665258:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828165/real 1787828165] req@ffff9181c35b3100 x1874663043220096/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828190 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828165.665277:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3100 x1874663043220096/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828190 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828165.665288:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828165.665292:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828165.665309:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043220096:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828165.665316:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043220224:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828165.665351:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828165.665370:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828165/real 1787828165] req@ffff9181c35b1180 x1874663043220224/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828190 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828165.665382:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1180 x1874663043220224/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828190 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828165.665390:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828165.665393:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828165.665396:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043220224:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828165.665401:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043220352:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828165.665416:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828165.665425:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828165/real 1787828165] req@ffff9181c35b0000 x1874663043220352/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828190 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828165.665435:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0000 x1874663043220352/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828190 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828165.665448:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828165.665451:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828165.665453:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043220352:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828166.240993:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828167.264853:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828168.288849:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828169.312849:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828170.336974:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828170.784863:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828170.784873:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828170.784877:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828170.784897:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10237 00000100:00080000:1.0:1787828170.784911:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828170.784919:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787828170.784959:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0e00 x1874663043220480/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828170.784986:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828170.784990:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828170.784992:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828170.784996:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10237 00000100:00080000:1.0:1787828170.784999:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828170.785003:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828170.785012:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828170.785022:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3480 x1874663043220608/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828170.785032:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828170.785035:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828170.785036:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828170.785040:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10237 00000100:00080000:1.0:1787828170.785043:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828170.785046:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828170.785051:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828170.785060:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2680 x1874663043220736/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828170.785081:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043220480:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1787828170.785189:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828170.785229:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828170/real 1787828170] req@ffff9181c35b0e00 x1874663043220480/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828195 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828170.785247:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0e00 x1874663043220480/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828195 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828170.785259:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828170.785264:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828170.785283:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043220480:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828170.785291:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043220608:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828170.785318:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828170.785353:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828170/real 1787828170] req@ffff9181c35b3480 x1874663043220608/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828195 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828170.785366:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3480 x1874663043220608/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828195 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828170.785375:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828170.785379:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828170.785382:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043220608:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828170.785406:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043220736:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828170.785423:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828170.785432:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828170/real 1787828170] req@ffff9181c35b2680 x1874663043220736/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828195 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828170.785443:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2680 x1874663043220736/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828195 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828170.785451:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828170.785453:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828170.785456:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043220736:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828171.360871:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828172.384858:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828173.408914:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828174.432931:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828175.457054:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828175.904980:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 10000000:01000000:0.0:1787828175.904981:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-sptlrpc' failed (-108). 00000100:00080000:1.0:1787828175.905012:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 10000000:00020000:0.0:1787828175.905014:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00080000:1.0:1787828175.905030:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828175.905047:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10242 00000100:00080000:1.0:1787828175.905092:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828175.905103:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787828175.905154:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2a00 x1874663043220864/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828175.905195:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828175.905200:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828175.905203:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828175.905208:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10242 00000100:00080000:1.0:1787828175.905212:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828175.905217:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828175.905229:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828175.905245:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0a80 x1874663043220992/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828175.905261:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828175.905266:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828175.905268:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828175.905277:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10242 00000100:00080000:1.0:1787828175.905282:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787828175.905290:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043220864:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787828175.905291:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828175.905310:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828175.905327:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2d80 x1874663043221120/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828175.905429:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828175.905501:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828175/real 1787828175] req@ffff9181c35b2a00 x1874663043220864/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828200 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828175.905529:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2a00 x1874663043220864/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828200 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828175.905548:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828175.905554:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828175.905560:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043220864:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828175.905596:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043220992:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828175.905626:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828175.905643:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828175/real 1787828175] req@ffff9181c35b0a80 x1874663043220992/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828200 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828175.905659:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0a80 x1874663043220992/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828200 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828175.905670:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828175.905674:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828175.905678:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043220992:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828175.905686:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043221120:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828175.905706:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828175.905718:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828175/real 1787828175] req@ffff9181c35b2d80 x1874663043221120/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828200 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828175.905732:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2d80 x1874663043221120/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828200 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828175.905787:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828175.905792:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828175.905796:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043221120:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828176.480958:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:0.0:1787828177.120809:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:0.0:1787828177.120815:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff9181c5620000 from 341 10000000:01000000:0.0:1787828177.120820:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:0.0:1787828177.120872:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f71a lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828177.120881:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f71a lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828177.120890:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f71a lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787828177.120908:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d01a5180 going to sleep for 0 seconds 00000100:00100000:0.0:1787828177.120932:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043221248:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:0.0:1787828177.120945:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f71a lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828177.120952:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f71a lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828177.120957:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f71a lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828177.120962:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f71a lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828177.120974:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f71a lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828177.120979:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0512800 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:0.0:1787828177.120982:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f71a lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828177.120988:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f71a lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1787828177.120993:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f71a lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787828177.120997:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:0.0:1787828177.120999:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787828177.121005:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f71a lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828177.121009:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f71a lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787828177.121032:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:0.0:1787828177.121035:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:1.0:1787828177.121148:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828177.121159:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828177.121165:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828177.121177:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10247 00000100:00080000:1.0:1787828177.121191:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828177.121200:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787828177.121245:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0700 x1874663043221376/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828177.121281:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828177.121286:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828177.121288:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828177.121293:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10247 00000100:00080000:1.0:1787828177.121297:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828177.121301:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828177.121313:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828177.121328:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1f80 x1874663043221504/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828177.121341:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828177.121345:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828177.121346:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828177.121351:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10247 00000100:00080000:1.0:1787828177.121354:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828177.121358:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828177.121366:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828177.121374:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043221376:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787828177.121397:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3800 x1874663043221632/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828177.121492:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828177.121555:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828177/real 1787828177] req@ffff9181c35b0700 x1874663043221376/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828202 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828177.121587:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0700 x1874663043221376/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828202 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828177.121607:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828177.121614:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828177.121641:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043221376:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828177.121655:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043221504:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828177.121709:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828177.121732:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828177/real 1787828177] req@ffff9181c35b1f80 x1874663043221504/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828202 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828177.121818:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1f80 x1874663043221504/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828202 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828177.121833:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828177.121838:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828177.121845:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043221504:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828177.121884:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043221632:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828177.121922:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828177.121945:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828177/real 1787828177] req@ffff9181c35b3800 x1874663043221632/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828202 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828177.121967:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3800 x1874663043221632/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828202 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828177.121982:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828177.121988:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828177.121994:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043221632:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828177.505017:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828178.528866:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828179.552991:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828180.576906:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828181.600964:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828182.560976:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828182.560989:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828182.560995:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828182.561029:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10248 00000100:00080000:2.0:1787828182.561057:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828182.561067:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787828182.561129:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c380 x1874663043221760/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828182.561167:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828182.561190:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828182.561193:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828182.561199:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10248 00000100:00080000:2.0:1787828182.561203:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828182.561209:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828182.561221:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828182.561240:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007dc00 x1874663043221888/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828182.561255:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828182.561261:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828182.561267:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787828182.561267:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043221760:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787828182.561308:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10248 00000100:00080000:2.0:1787828182.561314:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828182.561319:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828182.561326:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828182.561345:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c000 x1874663043222016/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828182.561394:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828182.561440:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828182/real 1787828182] req@ffff9181e007c380 x1874663043221760/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828207 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828182.561491:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c380 x1874663043221760/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828207 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828182.561509:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828182.561516:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828182.561542:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043221760:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828182.561580:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043221888:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828182.561637:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828182.561661:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828182/real 1787828182] req@ffff9181e007dc00 x1874663043221888/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828207 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828182.561680:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007dc00 x1874663043221888/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828207 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828182.561691:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828182.561695:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828182.561700:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043221888:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828182.561708:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043222016:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828182.561729:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828182.561742:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828182/real 1787828182] req@ffff9181e007c000 x1874663043222016/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828207 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828182.561756:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c000 x1874663043222016/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828207 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828182.561803:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828182.561809:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828182.561816:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043222016:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828182.625066:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828183.648962:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828184.672941:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828185.696892:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828186.720836:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828187.680845:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828187.680853:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828187.680856:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828187.680864:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10254 00000100:00080000:1.0:1787828187.680873:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828187.680880:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787828187.680913:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3b80 x1874663043222144/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828187.680934:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828187.680937:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828187.680938:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828187.680942:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10254 00000100:00080000:1.0:1787828187.680944:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828187.680947:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828187.680954:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828187.680963:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3800 x1874663043222272/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828187.680971:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828187.680987:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828187.680988:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828187.680994:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10254 00000100:00080000:1.0:1787828187.680997:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828187.681000:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828187.681006:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828187.681006:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043222144:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787828187.681020:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1f80 x1874663043222400/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828187.681081:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828187.681118:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828187/real 1787828187] req@ffff9181c35b3b80 x1874663043222144/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828212 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828187.681138:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3b80 x1874663043222144/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828212 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828187.681150:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828187.681155:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828187.681171:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043222144:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828187.681180:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043222272:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828187.681207:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828187.681226:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828187/real 1787828187] req@ffff9181c35b3800 x1874663043222272/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828212 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828187.681241:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3800 x1874663043222272/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828212 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828187.681250:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828187.681254:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828187.681257:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043222272:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828187.681280:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043222400:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828187.681302:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828187.681315:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828187/real 1787828187] req@ffff9181c35b1f80 x1874663043222400/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828212 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828187.681328:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1f80 x1874663043222400/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828212 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828187.681344:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828187.681347:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828187.681351:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043222400:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828187.744915:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828188.768920:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828189.793022:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828190.816927:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828191.841001:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828192.800829:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828192.800839:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828192.800843:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828192.800853:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10259 00000100:00080000:1.0:1787828192.800863:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828192.800870:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787828192.800907:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0700 x1874663043222528/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828192.800929:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828192.800933:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828192.800935:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828192.800957:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10259 00000100:00100000:2.0:1787828192.800959:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043222528:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787828192.800961:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828192.800965:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828192.800975:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828192.800988:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2d80 x1874663043222656/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828192.801005:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828192.801008:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828192.801011:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828192.801016:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10259 00000100:00080000:1.0:1787828192.801020:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828192.801025:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:2.0:1787828192.801027:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:1.0:1787828192.801031:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828192.801042:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0a80 x1874663043222784/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828192.801062:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828192/real 1787828192] req@ffff9181c35b0700 x1874663043222528/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828217 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828192.801083:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0700 x1874663043222528/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828217 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828192.801094:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828192.801099:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828192.801112:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043222528:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828192.801134:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043222656:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828192.801165:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828192.801179:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828192/real 1787828192] req@ffff9181c35b2d80 x1874663043222656/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828217 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828192.801193:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2d80 x1874663043222656/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828217 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828192.801204:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828192.801208:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828192.801213:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043222656:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828192.801219:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043222784:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828192.801235:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828192.801245:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828192/real 1787828192] req@ffff9181c35b0a80 x1874663043222784/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828217 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828192.801258:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0a80 x1874663043222784/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828217 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828192.801272:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828192.801276:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828192.801280:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043222784:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828192.864998:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828193.888904:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828194.913000:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828195.936905:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828196.960896:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:0.0:1787828197.408866:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-cliir' failed (-108). 10000000:00000400:0.0:1787828197.408877:0:159735:0:(mgc_request.c:1900:mgc_process_log()) MGC192.168.203.147@tcp: IR log lustre-cliir failed, not fatal: rc = -108 10000000:01000000:0.0:1787828197.412802:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:0.0:1787828197.412806:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff9181c5620000 from 52 10000000:01000000:0.0:1787828197.412809:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:0.0:1787828197.412859:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f721 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828197.412869:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f721 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828197.412883:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f721 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787828197.412904:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d01a5180 going to sleep for 0 seconds 00000100:00100000:0.0:1787828197.412930:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043222912:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:0.0:1787828197.412944:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f721 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828197.412951:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f721 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828197.412960:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f721 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828197.412967:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f721 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828197.412974:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f721 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828197.412979:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0512800 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:0.0:1787828197.412982:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f721 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828197.412988:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f721 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1787828197.412993:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f721 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787828197.412997:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:0.0:1787828197.412999:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787828197.413006:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f721 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828197.413011:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e0512800/0xbef42c0221d6f721 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787828197.413029:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:0.0:1787828197.413031:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:1.0:1787828197.413071:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828197.413077:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828197.413080:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828197.413087:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10264 00000100:00080000:1.0:1787828197.413097:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828197.413104:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787828197.413135:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2a00 x1874663043223040/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828197.413156:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828197.413160:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828197.413161:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828197.413165:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10264 00000100:00080000:1.0:1787828197.413169:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828197.413172:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828197.413183:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828197.413185:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043223040:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787828197.413196:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2680 x1874663043223168/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828197.413206:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828197.413209:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828197.413211:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828197.413214:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10264 00000100:00080000:1.0:1787828197.413217:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828197.413219:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828197.413223:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828197.413232:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3480 x1874663043223296/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828197.413259:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828197.413289:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828197/real 1787828197] req@ffff9181c35b2a00 x1874663043223040/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828222 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828197.413303:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2a00 x1874663043223040/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828222 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828197.413312:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828197.413316:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828197.413327:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043223040:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828197.413342:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043223168:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828197.413358:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828197.413367:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828197/real 1787828197] req@ffff9181c35b2680 x1874663043223168/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828222 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828197.413376:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2680 x1874663043223168/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828222 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828197.413381:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828197.413384:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828197.413386:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043223168:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828197.413389:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043223296:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828197.413399:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828197.413405:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828197/real 1787828197] req@ffff9181c35b3480 x1874663043223296/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828222 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828197.413412:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3480 x1874663043223296/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828222 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828197.413417:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828197.413419:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828197.413421:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043223296:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828197.984869:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828199.008888:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828200.032866:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828201.056916:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828202.080862:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828202.528820:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828202.528828:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828202.528832:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828202.528840:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10269 00000100:00080000:3.0:1787828202.528849:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828202.528856:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828202.528906:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635f80 x1874663043223424/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828202.528928:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828202.528931:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828202.528933:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828202.528936:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10269 00000100:00080000:3.0:1787828202.528939:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828202.528942:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828202.528951:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828202.528961:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043223552/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828202.528969:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043223424:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787828202.528972:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828202.528977:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828202.528979:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828202.528982:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10269 00000100:00080000:3.0:1787828202.528984:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828202.528987:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828202.528990:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828202.528998:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634e00 x1874663043223680/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828202.529029:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828202.529058:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828202/real 1787828202] req@ffff9181c5635f80 x1874663043223424/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828227 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828202.529071:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635f80 x1874663043223424/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828227 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828202.529079:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828202.529082:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828202.529096:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043223424:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828202.529115:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043223552:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828202.529133:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828202.529144:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828202/real 1787828202] req@ffff9181c5635500 x1874663043223552/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828227 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828202.529153:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043223552/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828227 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828202.529159:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828202.529161:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828202.529163:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043223552:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828202.529168:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043223680:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828202.529178:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828202.529184:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828202/real 1787828202] req@ffff9181c5634e00 x1874663043223680/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828227 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828202.529191:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634e00 x1874663043223680/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828227 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828202.529196:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828202.529198:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828202.529200:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043223680:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828203.104879:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828204.128883:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828205.152891:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828206.176876:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828207.200913:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828207.648872:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828207.648882:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828207.648885:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828207.648895:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10274 00000100:00080000:2.0:1787828207.648906:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828207.648923:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787828207.648956:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e680 x1874663043223808/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828207.648984:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828207.648988:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828207.648990:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828207.648994:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10274 00000100:00080000:2.0:1787828207.648998:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828207.649004:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828207.649012:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828207.649026:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043223808:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828207.649027:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ca80 x1874663043223936/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828207.649042:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828207.649046:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828207.649048:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828207.649053:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10274 00000100:00080000:2.0:1787828207.649057:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828207.649061:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828207.649066:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828207.649077:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007fb80 x1874663043224064/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828207.649115:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043223936:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828207.649136:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043224064:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828208.224891:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828209.248910:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828210.272918:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1787828210.721040:0:2376:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:0.0:1787828210.721087:0:2376:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1787828210.721105:0:2376:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1787828210.721145:0:2376:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1787828210.721154:0:2376:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:3.0:1787828210.721195:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828207/real 1787828210] req@ffff9181e007ca80 x1874663043223936/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828232 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828210.721222:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ca80 x1874663043223936/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828232 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828210.721235:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828210.721240:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828210.721244:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043223936:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828210.721252:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828207/real 1787828210] req@ffff9181e007fb80 x1874663043224064/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828232 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828210.721264:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007fb80 x1874663043224064/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828232 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828210.721278:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828210.721282:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828210.721285:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043224064:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828210.721291:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828207/real 1787828210] req@ffff9181e007e680 x1874663043223808/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828232 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828210.721303:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e680 x1874663043223808/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828232 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828210.721310:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828210.721313:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828210.721329:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043223808:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1787828211.296918:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828212.320917:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828212.768981:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828212.768991:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828212.768995:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828212.769012:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10279 00000100:00080000:1.0:1787828212.769024:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828212.769034:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787828212.769071:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0e00 x1874663043224192/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828212.769095:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828212.769098:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828212.769100:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828212.769105:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10279 00000100:00080000:1.0:1787828212.769107:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828212.769111:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828212.769120:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828212.769130:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0000 x1874663043224320/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828212.769138:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828212.769141:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828212.769142:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828212.769145:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10279 00000100:00080000:1.0:1787828212.769147:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828212.769150:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828212.769153:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828212.769166:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1180 x1874663043224448/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828212.769166:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043224192:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1787828212.769295:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828212.769354:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828212/real 1787828212] req@ffff9181c35b0e00 x1874663043224192/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828237 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828212.769378:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0e00 x1874663043224192/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828237 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828212.769390:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828212.769395:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828212.769416:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043224192:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828212.769431:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043224320:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828212.769459:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828212.769473:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828212/real 1787828212] req@ffff9181c35b0000 x1874663043224320/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828237 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828212.769486:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0000 x1874663043224320/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828237 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828212.769494:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828212.769497:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828212.769501:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043224320:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828212.769530:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043224448:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828212.769551:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828212.769568:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828212/real 1787828212] req@ffff9181c35b1180 x1874663043224448/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828237 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828212.769590:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1180 x1874663043224448/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828237 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828212.769602:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828212.769608:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828212.769618:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043224448:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828213.344904:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828214.368915:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828215.392904:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828216.416924:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828217.440894:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828217.888874:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828217.888885:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828217.888889:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 10000000:01000000:0.0:1787828217.888898:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-client' failed (-108). 00000100:00080000:1.0:1787828217.888900:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10284 10000000:00020000:0.0:1787828217.888906:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:0.0:1787828217.888912:0:159735:0:(mgc_request.c:611:do_requeue()) updating log params 00000100:00080000:1.0:1787828217.888913:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 10000000:01000000:0.0:1787828217.888914:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff9181c5620000 from 7 10000000:01000000:0.0:1787828217.888920:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00000100:00080000:1.0:1787828217.888925:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787828217.888956:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3100 x1874663043224704/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:0.0:1787828217.888972:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512600/0xbef42c0221d6f728 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828217.888985:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e0512600/0xbef42c0221d6f728 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787828217.888995:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828217.889001:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00010000:00010000:0.0:1787828217.889002:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e0512600/0xbef42c0221d6f728 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787828217.889005:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828217.889015:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10284 00000100:00080000:1.0:1787828217.889021:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828217.889027:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787828217.889033:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d01a5180 going to sleep for 0 seconds 00000008:00100000:1.0:1787828217.889038:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828217.889044:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043224704:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787828217.889047:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043224576:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00100000:1.0:1787828217.889055:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1500 x1874663043224832/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828217.889066:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828217.889069:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00010000:00010000:0.0:1787828217.889070:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e0512600/0xbef42c0221d6f728 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787828217.889071:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828217.889079:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10284 00000100:00080000:1.0:1787828217.889084:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00010000:00010000:0.0:1787828217.889085:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e0512600/0xbef42c0221d6f728 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787828217.889089:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828217.889097:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:0.0:1787828217.889099:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512600/0xbef42c0221d6f728 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828217.889106:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e0512600/0xbef42c0221d6f728 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787828217.889110:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1c00 x1874663043224960/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:0.0:1787828217.889121:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e0512600/0xbef42c0221d6f728 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828217.889128:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0512600 already unused, calling callback (ffffffffc1273c20) 00000800:00000100:3.0:1787828217.889134:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 10000000:00010000:0.0:1787828217.889135:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512600/0xbef42c0221d6f728 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828217.889144:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e0512600/0xbef42c0221d6f728 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1787828217.889151:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512600/0xbef42c0221d6f728 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787828217.889156:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:0.0:1787828217.889159:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787828217.889165:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e0512600/0xbef42c0221d6f728 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828217.889169:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e0512600/0xbef42c0221d6f728 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787828217.889177:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828217/real 1787828217] req@ffff9181c35b3100 x1874663043224704/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828242 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 10000000:01000000:0.0:1787828217.889190:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:0.0:1787828217.889194:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:CONNECTING 00000100:00100000:3.0:1787828217.889201:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3100 x1874663043224704/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828242 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828217.889215:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828217.889233:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828217.889244:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043224704:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787828217.889256:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828217.889260:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828217.889262:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828217.889268:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10289 00000100:00080000:1.0:1787828217.889272:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828217.889276:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828217.889280:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043224832:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787828217.889290:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0380 x1874663043225088/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828217.889313:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828217.889330:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828217/real 1787828217] req@ffff9181c35b1500 x1874663043224832/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828242 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828217.889344:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1500 x1874663043224832/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828242 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828217.889354:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828217.889359:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828217.889362:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043224832:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828217.889370:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043224960:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828217.889393:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828217.889407:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828217/real 1787828217] req@ffff9181c35b1c00 x1874663043224960/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828242 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828217.889421:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1c00 x1874663043224960/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828242 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828217.889430:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828217.889433:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828217.889437:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043224960:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828217.889453:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043225088:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1787828217.889475:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828217.889487:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828217/real 1787828217] req@ffff9181c35b0380 x1874663043225088/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828242 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828217.889499:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0380 x1874663043225088/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828242 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828217.889508:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828217.889511:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828217.889525:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043225088:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1787828218.464946:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828219.488907:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828220.512940:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828221.536862:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828222.560928:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828223.008842:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828223.008849:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828223.008853:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828223.008861:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10289 00000100:00080000:1.0:1787828223.008870:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828223.008876:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787828223.008909:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0380 x1874663043225216/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828223.008932:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828223.008934:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828223.008936:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828223.008939:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10289 00000100:00080000:1.0:1787828223.008941:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828223.008944:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828223.008951:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828223.008962:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1c00 x1874663043225344/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828223.008970:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828223.008972:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828223.008974:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828223.008978:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10289 00000100:00080000:1.0:1787828223.008981:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787828223.008985:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043225216:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787828223.008987:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828223.008994:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828223.009004:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1500 x1874663043225472/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828223.009049:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828223.009080:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828223/real 1787828223] req@ffff9181c35b0380 x1874663043225216/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828248 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828223.009095:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0380 x1874663043225216/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828248 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828223.009103:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828223.009106:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828223.009119:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043225216:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828223.009124:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043225344:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828223.009142:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828223.009156:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828223/real 1787828223] req@ffff9181c35b1c00 x1874663043225344/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828248 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828223.009165:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1c00 x1874663043225344/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828248 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828223.009170:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828223.009172:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828223.009175:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043225344:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828223.009196:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043225472:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828223.009216:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828223.009228:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828223/real 1787828223] req@ffff9181c35b1500 x1874663043225472/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828248 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828223.009242:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1500 x1874663043225472/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828248 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828223.009253:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828223.009256:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828223.009258:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043225472:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828223.584897:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828224.608877:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828225.632877:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828226.656937:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828227.680899:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828228.128808:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828228.128817:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828228.128820:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828228.128828:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10294 00000100:00080000:1.0:1787828228.128837:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828228.128843:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787828228.128873:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3100 x1874663043225600/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828228.128893:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828228.128896:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828228.128897:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828228.128900:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10294 00000100:00080000:1.0:1787828228.128903:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828228.128906:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828228.128912:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828228.128921:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1180 x1874663043225728/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828228.128932:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043225600:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787828228.128938:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828228.128942:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828228.128943:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828228.128947:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10294 00000100:00080000:1.0:1787828228.128949:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828228.128951:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828228.128955:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828228.128962:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0000 x1874663043225856/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828228.128994:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828228.129022:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828228/real 1787828228] req@ffff9181c35b3100 x1874663043225600/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828253 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828228.129040:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3100 x1874663043225600/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828253 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828228.129050:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828228.129055:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828228.129069:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043225600:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828228.129094:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043225728:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828228.129124:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828228.129137:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828228/real 1787828228] req@ffff9181c35b1180 x1874663043225728/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828253 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828228.129146:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1180 x1874663043225728/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828253 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828228.129153:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828228.129155:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828228.129157:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043225728:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828228.129161:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043225856:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828228.129174:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828228.129180:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828228/real 1787828228] req@ffff9181c35b0000 x1874663043225856/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828253 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828228.129187:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0000 x1874663043225856/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828253 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828228.129192:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828228.129194:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828228.129196:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043225856:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828228.704850:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828229.728949:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828230.752851:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828231.776897:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828232.800856:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828233.248825:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828233.248835:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828233.248839:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828233.248849:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10299 00000100:00080000:1.0:1787828233.248860:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828233.248867:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787828233.248902:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0e00 x1874663043225984/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828233.248928:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828233.248932:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828233.248933:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828233.248938:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10299 00000100:00080000:1.0:1787828233.248941:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828233.248945:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828233.248955:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828233.248967:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3480 x1874663043226112/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828233.248974:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043225984:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787828233.248985:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828233.248989:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828233.248991:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828233.248996:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10299 00000100:00080000:1.0:1787828233.249000:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828233.249004:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828233.249009:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828233.249021:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2680 x1874663043226240/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828233.249038:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828233.249070:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828233/real 1787828233] req@ffff9181c35b0e00 x1874663043225984/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828258 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828233.249084:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0e00 x1874663043225984/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828258 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828233.249093:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828233.249096:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828233.249109:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043225984:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828233.249128:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043226112:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828233.249146:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828233.249156:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828233/real 1787828233] req@ffff9181c35b3480 x1874663043226112/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828258 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828233.249167:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3480 x1874663043226112/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828258 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828233.249175:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828233.249179:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828233.249182:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043226112:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828233.249189:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043226240:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828233.249209:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828233.249221:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828233/real 1787828233] req@ffff9181c35b2680 x1874663043226240/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828258 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828233.249230:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2680 x1874663043226240/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828258 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828233.249236:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828233.249238:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828233.249240:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043226240:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828233.824863:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828234.848976:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828235.872968:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828236.896850:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828237.920880:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:0.0:1787828238.369006:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'params' failed (-108). 00000100:00080000:2.0:1787828238.369015:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 10000000:00020000:0.0:1787828238.369023:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00080000:2.0:1787828238.369026:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828238.369033:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 10000000:01000000:0.0:1787828238.369035:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:0.0:1787828238.369042:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff9181c87b19b8 from 1 00000100:00080000:2.0:1787828238.369063:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10304 10000000:01000000:0.0:1787828238.369072:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00000100:00080000:2.0:1787828238.369081:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828238.369101:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787828238.369139:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d880 x1874663043226496/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:0.0:1787828238.369169:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512200/0xbef42c0221d6f72f lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828238.369185:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e0512200/0xbef42c0221d6f72f lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787828238.369189:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828238.369199:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828238.369204:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00010000:00010000:0.0:1787828238.369206:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e0512200/0xbef42c0221d6f72f lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787828238.369212:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10304 00000100:00080000:2.0:1787828238.369222:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828238.369235:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828238.369251:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828238.369259:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d01a5180 going to sleep for 0 seconds 00000100:00100000:3.0:1787828238.369273:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043226496:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828238.369278:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ce00 x1874663043226624/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828238.369279:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043226368:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:0.0:1787828238.369309:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e0512200/0xbef42c0221d6f72f lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787828238.369312:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828238.369350:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00010000:00010000:0.0:1787828238.369353:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e0512200/0xbef42c0221d6f72f lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787828238.369356:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00010000:00010000:0.0:1787828238.369377:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512200/0xbef42c0221d6f72f lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787828238.369385:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10304 00000800:00000100:3.0:1787828238.369394:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00010000:00010000:0.0:1787828238.369395:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e0512200/0xbef42c0221d6f72f lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787828238.369400:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00010000:00010000:0.0:1787828238.369417:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e0512200/0xbef42c0221d6f72f lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828238.369425:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0512200 already unused, calling callback (ffffffffc1273c20) 00000100:00080000:2.0:1787828238.369445:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 10000000:00010000:0.0:1787828238.369460:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512200/0xbef42c0221d6f72f lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000008:00100000:2.0:1787828238.369466:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:0.0:1787828238.369479:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e0512200/0xbef42c0221d6f72f lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787828238.369479:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828238/real 1787828238] req@ffff9181e007d880 x1874663043226496/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828263 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828238.369495:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f480 x1874663043226752/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 10000000:00010000:0.0:1787828238.369523:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512200/0xbef42c0221d6f72f lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787828238.369551:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d880 x1874663043226496/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828263 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 10000000:01000000:0.0:1787828238.369556:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:0.0:1787828238.369572:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00000100:00080000:3.0:1787828238.369584:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00010000:00010000:0.0:1787828238.369587:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e0512200/0xbef42c0221d6f72f lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787828238.369596:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00010000:00010000:0.0:1787828238.369611:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e0512200/0xbef42c0221d6f72f lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787828238.369612:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043226496:192.168.203.147@tcp:250:kworker.0 10000000:01000000:0.0:1787828238.369687:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00100000:3.0:1787828238.369689:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043226624:192.168.203.147@tcp:38:kworker.0 00000100:00080000:0.0:1787828238.369695:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:2.0:1787828238.369778:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828238.369786:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828238.369789:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000800:00000100:3.0:1787828238.369800:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1787828238.369802:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10310 00000100:00080000:2.0:1787828238.369815:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828238.369822:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828238.369845:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828238/real 1787828238] req@ffff9181e007ce00 x1874663043226624/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828263 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828238.369857:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f100 x1874663043226880/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828238.369925:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ce00 x1874663043226624/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828263 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828238.369948:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828238.369954:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828238.369959:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043226624:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828238.369969:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043226752:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828238.370033:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828238.370049:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828238/real 1787828238] req@ffff9181e007f480 x1874663043226752/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828263 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828238.370063:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f480 x1874663043226752/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828263 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828238.370073:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828238.370077:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828238.370080:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043226752:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828238.370109:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043226880:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1787828238.370128:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828238.370145:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828238/real 1787828238] req@ffff9181e007f100 x1874663043226880/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828263 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828238.370181:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f100 x1874663043226880/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828263 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828238.370190:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828238.370193:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828238.370210:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043226880:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1787828238.944849:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828239.968984:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828240.992860:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828242.017025:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828243.040906:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828243.488802:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828243.488810:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828243.488814:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828243.488823:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10310 00000100:00080000:2.0:1787828243.488833:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828243.488839:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787828243.488897:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f800 x1874663043227008/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828243.488926:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828243.488930:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828243.488932:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828243.488937:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10310 00000100:00080000:2.0:1787828243.488941:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828243.488945:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828243.488954:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828243.488968:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ea00 x1874663043227136/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828243.488979:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828243.488982:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828243.488984:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828243.488988:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10310 00000100:00080000:2.0:1787828243.488991:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828243.488995:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828243.489002:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828243.489012:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c700 x1874663043227264/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828243.489030:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043227008:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1787828243.489123:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828243.489167:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828243/real 1787828243] req@ffff9181e007f800 x1874663043227008/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828268 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828243.489188:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f800 x1874663043227008/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828268 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828243.489202:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828243.489209:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828243.489255:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043227008:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828243.489271:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043227136:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828243.489343:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828243.489362:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828243/real 1787828243] req@ffff9181e007ea00 x1874663043227136/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828268 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828243.489380:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ea00 x1874663043227136/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828268 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828243.489389:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828243.489393:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828243.489396:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043227136:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828243.489422:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043227264:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828243.489441:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828243.489470:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828243/real 1787828243] req@ffff9181e007c700 x1874663043227264/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828268 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828243.489490:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c700 x1874663043227264/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828268 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828243.489499:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828243.489502:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828243.489505:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043227264:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1787828244.064903:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828245.089003:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828246.112930:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828247.136974:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828248.160943:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828248.608913:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828248.608928:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828248.608934:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828248.608949:0:160087:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10315 00000100:00080000:1.0:1787828248.608964:0:160087:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828248.608976:0:160087:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787828248.609027:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2a00 x1874663043227392/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828248.609061:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828248.609066:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828248.609068:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828248.609074:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10315 00000100:00080000:1.0:1787828248.609080:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828248.609086:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828248.609098:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828248.609112:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0a80 x1874663043227520/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828248.609143:0:160087:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828248.609147:0:160087:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828248.609150:0:160087:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828248.609155:0:160087:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10315 00000100:00080000:1.0:1787828248.609160:0:160087:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787828248.609167:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043227392:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787828248.609169:0:160087:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828248.609181:0:160087:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828248.609195:0:160087:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2d80 x1874663043227648/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828248.609396:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828248.609446:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828248/real 1787828248] req@ffff9181c35b2a00 x1874663043227392/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828273 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828248.609464:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2a00 x1874663043227392/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828273 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828248.609475:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828248.609479:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828248.609509:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043227392:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828248.609542:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043227520:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828248.609584:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828248.609610:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828248/real 1787828248] req@ffff9181c35b0a80 x1874663043227520/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828273 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828248.609638:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0a80 x1874663043227520/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828273 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828248.609645:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828248.609650:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828248.609658:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043227520:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828248.609667:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043227648:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828248.609691:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828248.609703:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828248/real 1787828248] req@ffff9181c35b2d80 x1874663043227648/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828273 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828248.609715:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2d80 x1874663043227648/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828273 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828248.609721:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828248.609723:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828248.609726:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043227648:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787828249.184934:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828250.208898:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828251.232917:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828252.256861:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828253.280961:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828253.728911:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828253.728921:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828253.728925:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828253.728936:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10320 00000100:00080000:2.0:1787828253.728947:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828253.728954:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787828253.728994:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e300 x1874663043227776/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828253.729024:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828253.729029:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828253.729031:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828253.729035:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10320 00000100:00080000:2.0:1787828253.729038:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828253.729043:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828253.729052:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828253.729064:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d180 x1874663043227904/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828253.729077:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043227776:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787828253.729078:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828253.729090:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828253.729092:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828253.729099:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10320 00000100:00080000:2.0:1787828253.729105:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828253.729111:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828253.729120:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828253.729139:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d500 x1874663043228032/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828253.729167:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828253.729208:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828253/real 1787828253] req@ffff9181e007e300 x1874663043227776/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828278 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828253.729226:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e300 x1874663043227776/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828278 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828253.729244:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828253.729249:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828253.729263:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043227776:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828253.729288:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043227904:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828253.729311:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828253.729324:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828253/real 1787828253] req@ffff9181e007d180 x1874663043227904/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828278 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828253.729337:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d180 x1874663043227904/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828278 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828253.729346:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828253.729352:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828253.729358:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043227904:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828253.729365:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043228032:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828253.729382:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828253.729391:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828253/real 1787828253] req@ffff9181e007d500 x1874663043228032/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828278 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828253.729402:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d500 x1874663043228032/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828278 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828253.729410:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828253.729412:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828253.729415:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043228032:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828254.304876:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828255.328843:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828256.352884:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828257.376912:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828258.400866:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828258.848849:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828258.848858:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 10000000:01000000:0.0:1787828258.848859:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-sptlrpc' failed (-108). 00000100:00080000:2.0:1787828258.848863:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 10000000:00020000:0.0:1787828258.848870:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00080000:2.0:1787828258.848878:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10325 00000100:00080000:2.0:1787828258.848891:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828258.848899:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787828258.848939:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007df80 x1874663043228160/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828258.848965:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828258.848968:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828258.848970:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828258.848973:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10325 00000100:00080000:2.0:1787828258.848976:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828258.848979:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828258.848988:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828258.848999:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ed80 x1874663043228288/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828258.849008:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828258.849012:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828258.849014:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787828258.849014:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043228160:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787828258.849021:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10325 00000100:00080000:2.0:1787828258.849025:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828258.849028:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828258.849032:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828258.849041:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d500 x1874663043228416/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828258.849087:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828258.849117:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828258/real 1787828258] req@ffff9181e007df80 x1874663043228160/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828283 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828258.849130:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007df80 x1874663043228160/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828283 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828258.849138:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828258.849141:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828258.849144:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043228160:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828258.849163:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043228288:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828258.849179:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828258.849187:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828258/real 1787828258] req@ffff9181e007ed80 x1874663043228288/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828283 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828258.849195:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ed80 x1874663043228288/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828283 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828258.849201:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828258.849204:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828258.849206:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043228288:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828258.849210:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043228416:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828258.849223:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828258.849229:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828258/real 1787828258] req@ffff9181e007d500 x1874663043228416/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828283 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828258.849236:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d500 x1874663043228416/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828283 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828258.849241:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828258.849243:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828258.849245:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043228416:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787828259.424900:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:0.0:1787828260.000874:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:0.0:1787828260.000885:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff9181c5620000 from 341 10000000:01000000:0.0:1787828260.000893:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:0.0:1787828260.000971:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512c00/0xbef42c0221d6f736 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828260.000989:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e0512c00/0xbef42c0221d6f736 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828260.001007:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e0512c00/0xbef42c0221d6f736 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787828260.001040:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d01a5180 going to sleep for 0 seconds 00000100:00100000:0.0:1787828260.001058:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043228544:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:0.0:1787828260.001081:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e0512c00/0xbef42c0221d6f736 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828260.001099:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e0512c00/0xbef42c0221d6f736 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828260.001108:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512c00/0xbef42c0221d6f736 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828260.001118:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e0512c00/0xbef42c0221d6f736 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828260.001127:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e0512c00/0xbef42c0221d6f736 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828260.001136:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0512c00 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:0.0:1787828260.001141:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512c00/0xbef42c0221d6f736 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828260.001150:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e0512c00/0xbef42c0221d6f736 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1787828260.001160:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512c00/0xbef42c0221d6f736 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787828260.001167:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:0.0:1787828260.001171:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787828260.001180:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e0512c00/0xbef42c0221d6f736 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828260.001189:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e0512c00/0xbef42c0221d6f736 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787828260.001227:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:0.0:1787828260.001231:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:2.0:1787828260.001366:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828260.001375:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828260.001378:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828260.001387:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10330 00000100:00080000:2.0:1787828260.001399:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828260.001406:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787828260.001442:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d180 x1874663043228672/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828260.001493:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828260.001496:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828260.001499:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828260.001502:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10330 00000100:00080000:2.0:1787828260.001505:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828260.001508:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828260.001517:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828260.001528:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e300 x1874663043228800/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828260.001537:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828260.001539:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828260.001541:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828260.001547:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10330 00000100:00080000:2.0:1787828260.001549:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828260.001552:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828260.001555:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828260.001562:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c700 x1874663043228928/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828260.001590:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043228672:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1787828260.001699:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828260.001799:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828260/real 1787828260] req@ffff9181e007d180 x1874663043228672/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828285 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828260.001835:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d180 x1874663043228672/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828285 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828260.001858:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828260.001867:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828260.001890:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043228672:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828260.001906:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043228800:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828260.001956:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828260.001982:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828260/real 1787828260] req@ffff9181e007e300 x1874663043228800/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828285 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828260.002009:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e300 x1874663043228800/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828285 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828260.002020:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828260.002025:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828260.002029:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043228800:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828260.002037:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043228928:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828260.002065:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828260.002078:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828260/real 1787828260] req@ffff9181e007c700 x1874663043228928/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828285 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828260.002094:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c700 x1874663043228928/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828285 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828260.002103:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828260.002107:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828260.002110:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043228928:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828260.448881:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828261.473006:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828262.496936:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828263.521024:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828264.544984:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828265.504897:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828265.504912:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828265.504919:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828265.504934:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10331 00000100:00080000:0.0:1787828265.504949:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828265.504960:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787828265.505004:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7480 x1874663043229056/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828265.505034:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828265.505038:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828265.505040:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828265.505044:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10331 00000100:00080000:0.0:1787828265.505048:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828265.505052:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828265.505060:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828265.505092:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5880 x1874663043229184/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828265.505104:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828265.505108:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828265.505110:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828265.505116:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10331 00000100:00080000:0.0:1787828265.505120:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828265.505125:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828265.505147:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828265.505156:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7100 x1874663043229312/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828265.505163:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043229056:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1787828265.505270:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828265.505372:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828265/real 1787828265] req@ffff9181c55a7480 x1874663043229056/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828290 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828265.505404:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7480 x1874663043229056/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828290 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828265.505456:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828265.505464:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828265.505513:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043229056:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828265.505529:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043229184:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828265.505580:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828265.505607:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828265/real 1787828265] req@ffff9181c55a5880 x1874663043229184/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828290 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828265.505649:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5880 x1874663043229184/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828290 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828265.505664:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828265.505671:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828265.505677:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043229184:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828265.505725:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043229312:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828265.505778:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828265.505794:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828265/real 1787828265] req@ffff9181c55a7100 x1874663043229312/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828290 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828265.505863:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7100 x1874663043229312/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828290 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828265.505875:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828265.505880:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828265.505885:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043229312:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787828265.568937:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828266.592889:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828267.616876:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828268.641001:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828269.664887:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828270.624973:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828270.624985:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828270.624990:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828270.625003:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10337 00000100:00080000:1.0:1787828270.625018:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828270.625026:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787828270.625073:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0700 x1874663043229440/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828270.625129:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828270.625134:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828270.625136:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828270.625142:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10337 00000100:00080000:1.0:1787828270.625147:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828270.625152:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828270.625164:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828270.625182:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1f80 x1874663043229568/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828270.625198:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828270.625208:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:3.0:1787828270.625208:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043229440:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787828270.625214:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828270.625230:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10337 00000100:00080000:1.0:1787828270.625234:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828270.625239:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828270.625249:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828270.625274:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3800 x1874663043229696/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828270.625380:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043229568:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828270.625448:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043229696:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828270.688993:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828271.712955:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828272.736865:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1787828273.697040:0:2377:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:2.0:1787828273.697088:0:2377:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1787828273.697133:0:2377:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:2.0:1787828273.697190:0:2377:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:2.0:1787828273.697212:0:2377:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:3.0:1787828273.697251:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828270/real 1787828273] req@ffff9181c35b1f80 x1874663043229568/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828295 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828273.697308:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1f80 x1874663043229568/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828295 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828273.697326:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828273.697332:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828273.697338:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043229568:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828273.697350:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828270/real 1787828273] req@ffff9181c35b3800 x1874663043229696/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828295 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828273.697368:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3800 x1874663043229696/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828295 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828273.697402:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828273.697407:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828273.697412:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043229696:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828273.697423:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828270/real 1787828273] req@ffff9181c35b0700 x1874663043229440/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828295 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828273.697442:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0700 x1874663043229440/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828295 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828273.697456:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828273.697461:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828273.697483:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043229440:192.168.203.147@tcp:250:kworker.0 00010000:00010000:2.0:1787828273.760969:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828274.784978:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828275.744852:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828275.744860:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828275.744863:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828275.744873:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10342 00000100:00080000:0.0:1787828275.744883:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828275.744889:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787828275.744923:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6680 x1874663043229824/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828275.744951:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828275.744957:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828275.744958:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828275.744962:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10342 00000100:00080000:0.0:1787828275.744966:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828275.744970:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828275.744979:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828275.744995:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7b80 x1874663043229952/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828275.745005:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043229824:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787828275.745011:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828275.745017:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828275.745019:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828275.745024:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10342 00000100:00080000:0.0:1787828275.745029:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828275.745033:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828275.745039:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828275.745055:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7100 x1874663043230080/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828275.745075:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828275.745117:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828275/real 1787828275] req@ffff9181c55a6680 x1874663043229824/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828300 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828275.745137:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6680 x1874663043229824/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828300 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828275.745148:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828275.745152:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828275.745177:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043229824:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828275.745207:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043229952:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828275.745234:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828275.745248:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828275/real 1787828275] req@ffff9181c55a7b80 x1874663043229952/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828300 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828275.745262:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7b80 x1874663043229952/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828300 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828275.745271:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828275.745275:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828275.745278:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043229952:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828275.745285:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043230080:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828275.745308:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828275.745321:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828275/real 1787828275] req@ffff9181c55a7100 x1874663043230080/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828300 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828275.745334:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7100 x1874663043230080/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828300 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828275.745343:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828275.745347:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828275.745350:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043230080:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787828275.808880:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828276.832900:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828277.856868:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828278.880922:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828279.904872:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:0.0:1787828280.352866:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-cliir' failed (-108). 10000000:00000400:0.0:1787828280.352874:0:159735:0:(mgc_request.c:1900:mgc_process_log()) MGC192.168.203.147@tcp: IR log lustre-cliir failed, not fatal: rc = -108 10000000:01000000:0.0:1787828280.352881:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:0.0:1787828280.352883:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff9181c5620000 from 52 10000000:01000000:0.0:1787828280.352886:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:0.0:1787828280.352937:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512a00/0xbef42c0221d6f73d lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828280.352947:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e0512a00/0xbef42c0221d6f73d lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828280.352966:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e0512a00/0xbef42c0221d6f73d lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787828280.352983:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d01a5180 going to sleep for 0 seconds 00000100:00100000:0.0:1787828280.353004:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043230208:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:0.0:1787828280.353015:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e0512a00/0xbef42c0221d6f73d lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828280.353027:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e0512a00/0xbef42c0221d6f73d lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828280.353032:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512a00/0xbef42c0221d6f73d lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828280.353037:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e0512a00/0xbef42c0221d6f73d lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828280.353042:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e0512a00/0xbef42c0221d6f73d lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828280.353046:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0512a00 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:0.0:1787828280.353049:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512a00/0xbef42c0221d6f73d lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828280.353057:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e0512a00/0xbef42c0221d6f73d lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1787828280.353062:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512a00/0xbef42c0221d6f73d lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787828280.353069:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:0.0:1787828280.353072:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787828280.353079:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e0512a00/0xbef42c0221d6f73d lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828280.353086:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e0512a00/0xbef42c0221d6f73d lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787828280.353111:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:0.0:1787828280.353114:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:2.0:1787828280.353162:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828280.353167:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828280.353170:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828280.353175:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10347 00000100:00080000:2.0:1787828280.353183:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828280.353188:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787828280.353207:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ea00 x1874663043230336/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828280.353225:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828280.353228:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828280.353230:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828280.353233:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10347 00000100:00080000:2.0:1787828280.353235:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828280.353237:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828280.353245:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828280.353253:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f800 x1874663043230464/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828280.353262:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828280.353267:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:3.0:1787828280.353267:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043230336:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787828280.353270:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828280.353278:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10347 00000100:00080000:2.0:1787828280.353280:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828280.353282:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828280.353287:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828280.353295:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f100 x1874663043230592/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828280.353346:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828280.353379:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828280/real 1787828280] req@ffff9181e007ea00 x1874663043230336/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828305 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828280.353395:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ea00 x1874663043230336/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828305 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828280.353405:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828280.353409:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828280.353421:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043230336:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828280.353442:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043230464:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828280.353467:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828280.353481:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828280/real 1787828280] req@ffff9181e007f800 x1874663043230464/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828305 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828280.353496:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f800 x1874663043230464/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828305 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828280.353505:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828280.353507:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828280.353509:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043230464:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828280.353514:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043230592:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828280.353529:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828280.353537:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828280/real 1787828280] req@ffff9181e007f100 x1874663043230592/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828305 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828280.353546:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f100 x1874663043230592/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828305 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828280.353554:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828280.353557:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828280.353560:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043230592:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828280.928880:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828281.952885:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828282.976841:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828284.000868:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828285.024880:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828285.472804:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828285.472812:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828285.472816:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828285.472824:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10352 00000100:00080000:2.0:1787828285.472833:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828285.472839:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787828285.472871:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f480 x1874663043230720/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828285.472893:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828285.472896:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828285.472898:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828285.472901:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10352 00000100:00080000:2.0:1787828285.472903:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828285.472905:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828285.472912:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828285.472922:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ce00 x1874663043230848/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828285.472930:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828285.472932:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828285.472933:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828285.472939:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10352 00000100:00100000:3.0:1787828285.472940:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043230720:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787828285.472941:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828285.472945:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828285.472948:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828285.472955:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d880 x1874663043230976/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828285.473005:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828285.473034:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828285/real 1787828285] req@ffff9181e007f480 x1874663043230720/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828310 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828285.473046:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f480 x1874663043230720/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828310 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828285.473059:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828285.473062:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828285.473073:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043230720:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828285.473093:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043230848:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828285.473110:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828285.473122:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828285/real 1787828285] req@ffff9181e007ce00 x1874663043230848/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828310 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828285.473131:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ce00 x1874663043230848/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828310 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828285.473137:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828285.473139:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828285.473141:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043230848:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828285.473145:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043230976:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828285.473155:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828285.473161:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828285/real 1787828285] req@ffff9181e007d880 x1874663043230976/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828310 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828285.473168:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d880 x1874663043230976/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828310 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828285.473173:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828285.473175:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828285.473177:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043230976:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787828286.048854:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828287.072885:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828288.097001:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828289.120872:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828290.144906:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828290.592826:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828290.592835:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828290.592839:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828290.592848:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10357 00000100:00080000:2.0:1787828290.592859:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828290.592867:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787828290.592905:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e680 x1874663043231104/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828290.592933:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828290.592937:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828290.592938:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828290.592942:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10357 00000100:00080000:2.0:1787828290.592946:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828290.592950:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828290.592958:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828290.592969:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043231104:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828290.592972:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007fb80 x1874663043231232/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828290.592985:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828290.592989:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828290.592991:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828290.592995:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10357 00000100:00080000:2.0:1787828290.593000:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828290.593004:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828290.593010:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828290.593024:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ca80 x1874663043231360/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828290.593034:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828290.593071:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828290/real 1787828290] req@ffff9181e007e680 x1874663043231104/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828315 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828290.593085:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e680 x1874663043231104/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828315 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828290.593093:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828290.593096:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828290.593107:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043231104:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828290.593127:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043231232:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828290.593148:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828290.593158:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828290/real 1787828290] req@ffff9181e007fb80 x1874663043231232/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828315 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828290.593166:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007fb80 x1874663043231232/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828315 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828290.593172:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828290.593174:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828290.593176:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043231232:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828290.593180:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043231360:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828290.593190:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828290.593196:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828290/real 1787828290] req@ffff9181e007ca80 x1874663043231360/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828315 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828290.593204:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ca80 x1874663043231360/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828315 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828290.593209:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828290.593211:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828290.593213:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043231360:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1787828291.168841:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828292.192890:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828293.216896:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828294.240891:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828295.264858:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828295.712835:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828295.712843:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828295.712846:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828295.712855:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10362 00000100:00080000:3.0:1787828295.712864:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828295.712871:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828295.712902:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043231488/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828295.712924:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828295.712928:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828295.712930:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828295.712935:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10362 00000100:00080000:3.0:1787828295.712939:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828295.712944:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828295.712953:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828295.712960:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043231488:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828295.712965:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043231616/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828295.712976:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828295.712979:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828295.712981:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828295.712984:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10362 00000100:00080000:3.0:1787828295.712987:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828295.712991:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828295.712995:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828295.713003:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043231744/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828295.713030:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828295.713067:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828295/real 1787828295] req@ffff9181c5634380 x1874663043231488/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828320 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828295.713080:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043231488/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828320 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828295.713087:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828295.713090:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828295.713101:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043231488:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828295.713121:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043231616:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828295.713143:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828295.713153:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828295/real 1787828295] req@ffff9181c5636a00 x1874663043231616/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828320 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828295.713161:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043231616/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828320 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828295.713167:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828295.713169:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828295.713171:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043231616:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828295.713174:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043231744:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828295.713185:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828295.713190:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828295/real 1787828295] req@ffff9181c5637480 x1874663043231744/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828320 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828295.713197:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043231744/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828320 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828295.713202:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828295.713204:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828295.713206:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043231744:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828296.288856:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828297.312848:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828298.336889:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828299.360870:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828300.384865:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:0.0:1787828300.832878:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-client' failed (-108). 10000000:00020000:0.0:1787828300.832886:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00080000:1.0:1787828300.832886:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 10000000:01000000:0.0:1787828300.832891:0:159735:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:0.0:1787828300.832894:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff9181c5620000 from 7 00000100:00080000:1.0:1787828300.832895:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 10000000:01000000:0.0:1787828300.832898:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00000100:00080000:1.0:1787828300.832901:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828300.832914:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10367 00000100:00080000:1.0:1787828300.832924:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828300.832931:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787828300.832958:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3b80 x1874663043232000/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:0.0:1787828300.832959:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512a00/0xbef42c0221d6f744 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828300.832977:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e0512a00/0xbef42c0221d6f744 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828300.832993:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e0512a00/0xbef42c0221d6f744 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787828300.833005:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1787828300.833007:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d01a5180 going to sleep for 0 seconds 00000100:00080000:1.0:1787828300.833010:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828300.833014:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1787828300.833017:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043231872:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00080000:1.0:1787828300.833023:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10367 00000100:00080000:1.0:1787828300.833029:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00010000:00010000:0.0:1787828300.833030:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e0512a00/0xbef42c0221d6f744 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787828300.833039:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:0.0:1787828300.833040:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e0512a00/0xbef42c0221d6f744 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787828300.833046:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043232000:192.168.203.147@tcp:250:kworker.0 00010000:00010000:0.0:1787828300.833056:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512a00/0xbef42c0221d6f744 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000008:00100000:1.0:1787828300.833056:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:0.0:1787828300.833068:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e0512a00/0xbef42c0221d6f744 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828300.833073:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e0512a00/0xbef42c0221d6f744 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787828300.833079:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0700 x1874663043232128/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:0.0:1787828300.833081:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0512a00 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:0.0:1787828300.833086:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512a00/0xbef42c0221d6f744 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787828300.833096:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:0.0:1787828300.833097:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e0512a00/0xbef42c0221d6f744 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787828300.833105:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000800:00000100:2.0:1787828300.833109:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1787828300.833110:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 10000000:00010000:0.0:1787828300.833112:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512a00/0xbef42c0221d6f744 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787828300.833120:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10367 10000000:01000000:0.0:1787828300.833121:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:0.0:1787828300.833126:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00000100:00080000:1.0:1787828300.833129:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00010000:00010000:0.0:1787828300.833131:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e0512a00/0xbef42c0221d6f744 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787828300.833137:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:0.0:1787828300.833139:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e0512a00/0xbef42c0221d6f744 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000008:00100000:1.0:1787828300.833143:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828300.833145:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828300/real 1787828300] req@ffff9181c35b3b80 x1874663043232000/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828325 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828300.833158:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3800 x1874663043232256/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 10000000:01000000:0.0:1787828300.833168:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:0.0:1787828300.833172:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:CONNECTING 00000100:00100000:2.0:1787828300.833173:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3b80 x1874663043232000/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828325 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828300.833183:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828300.833208:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828300.833217:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043232000:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787828300.833225:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828300.833229:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828300.833231:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828300.833237:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10372 00000100:00100000:2.0:1787828300.833238:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043232128:192.168.203.147@tcp:38:kworker.0 00000100:00080000:1.0:1787828300.833241:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828300.833247:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787828300.833260:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1f80 x1874663043232384/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828300.833268:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828300.833285:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828300/real 1787828300] req@ffff9181c35b0700 x1874663043232128/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828325 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828300.833295:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0700 x1874663043232128/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828325 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828300.833309:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828300.833314:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828300.833316:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043232128:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828300.833321:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043232256:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828300.833334:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828300.833347:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828300/real 1787828300] req@ffff9181c35b3800 x1874663043232256/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828325 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828300.833358:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3800 x1874663043232256/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828325 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828300.833368:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828300.833370:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828300.833372:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043232256:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828300.833381:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043232384:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787828300.833392:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828300.833398:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828300/real 1787828300] req@ffff9181c35b1f80 x1874663043232384/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828325 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828300.833405:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1f80 x1874663043232384/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828325 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828300.833410:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828300.833412:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828300.833420:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043232384:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1787828301.408857:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828302.432866:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828303.456994:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828304.480878:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828305.504864:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828305.952875:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828305.952884:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828305.952887:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828305.952896:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10372 00000100:00080000:0.0:1787828305.952905:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828305.952911:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787828305.952943:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5880 x1874663043232512/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828305.952967:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828305.952969:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828305.952971:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828305.952974:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10372 00000100:00080000:0.0:1787828305.952976:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828305.952979:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828305.952985:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828305.952994:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7480 x1874663043232640/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828305.953001:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828305.953004:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828305.953007:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828305.953012:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10372 00000100:00080000:0.0:1787828305.953014:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828305.953016:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828305.953019:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828305.953026:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6d80 x1874663043232768/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828305.953044:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043232512:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787828305.953125:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828305.953182:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828305/real 1787828305] req@ffff9181c55a5880 x1874663043232512/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828330 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828305.953199:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5880 x1874663043232512/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828330 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828305.953209:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828305.953212:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828305.953230:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043232512:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828305.953238:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043232640:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828305.953259:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828305.953271:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828305/real 1787828305] req@ffff9181c55a7480 x1874663043232640/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828330 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828305.953281:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7480 x1874663043232640/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828330 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828305.953287:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828305.953290:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828305.953293:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043232640:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828305.953324:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043232768:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828305.953346:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828305.953368:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828305/real 1787828305] req@ffff9181c55a6d80 x1874663043232768/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828330 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828305.953377:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6d80 x1874663043232768/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828330 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828305.953383:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828305.953385:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828305.953388:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043232768:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828306.528907:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828307.552911:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828308.576928:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828309.600895:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828310.624995:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828311.072936:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828311.072948:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828311.072952:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828311.072964:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10377 00000100:00080000:3.0:1787828311.072976:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828311.072983:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828311.073045:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043232896/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828311.073080:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828311.073083:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828311.073085:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828311.073088:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10377 00000100:00080000:3.0:1787828311.073091:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828311.073094:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828311.073103:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828311.073113:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043233024/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828311.073155:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828311.073160:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:2.0:1787828311.073162:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043232896:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787828311.073163:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828311.073176:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10377 00000100:00080000:3.0:1787828311.073181:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828311.073184:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828311.073189:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828311.073200:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043233152/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828311.073239:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828311.073274:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828311/real 1787828311] req@ffff9181c5637100 x1874663043232896/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828336 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828311.073289:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043232896/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828336 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828311.073297:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828311.073301:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828311.073313:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043232896:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828311.073334:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043233024:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828311.073353:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828311.073361:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828311/real 1787828311] req@ffff9181c5637800 x1874663043233024/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828336 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828311.073369:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043233024/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828336 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828311.073375:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828311.073377:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828311.073379:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043233024:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828311.073383:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043233152:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828311.073394:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828311.073400:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828311/real 1787828311] req@ffff9181c5636d80 x1874663043233152/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828336 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828311.073407:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043233152/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828336 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828311.073417:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828311.073419:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828311.073421:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043233152:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828311.648871:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828312.672837:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828313.696930:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828314.720928:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828315.744886:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828316.192879:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828316.192891:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828316.192896:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828316.192918:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10382 00000100:00080000:3.0:1787828316.192931:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828316.192940:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828316.193013:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043233280/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828316.193059:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828316.193064:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828316.193067:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828316.193072:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10382 00000100:00080000:3.0:1787828316.193076:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828316.193108:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828316.193154:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828316.193174:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634a80 x1874663043233408/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828316.193190:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043233280:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787828316.193193:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828316.193207:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828316.193210:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828316.193220:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10382 00000100:00080000:3.0:1787828316.193227:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828316.193252:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828316.193265:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828316.193291:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043233536/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828316.193303:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828316.193382:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828316/real 1787828316] req@ffff9181c5635c00 x1874663043233280/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828341 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828316.193424:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043233280/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828341 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828316.193436:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828316.193441:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828316.193459:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043233280:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828316.193487:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043233408:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828316.193515:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828316.193533:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828316/real 1787828316] req@ffff9181c5634a80 x1874663043233408/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828341 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828316.193553:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634a80 x1874663043233408/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828341 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828316.193566:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828316.193570:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828316.193576:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043233408:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828316.193584:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043233536:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828316.193604:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828316.193614:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828316/real 1787828316] req@ffff9181c5634000 x1874663043233536/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828341 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828316.193626:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043233536/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828341 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828316.193635:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828316.193637:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828316.193641:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043233536:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828316.768922:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828317.792898:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828318.816934:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828319.840864:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828320.864907:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828321.312824:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828321.312837:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828321.312843:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828321.312856:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10387 00000100:00080000:3.0:1787828321.312869:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828321.312877:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828321.312928:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637b80 x1874663043233664/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 10000000:01000000:0.0:1787828321.312936:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'params' failed (-108). 10000000:00020000:0.0:1787828321.312944:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00080000:3.0:1787828321.312959:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828321.312965:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828321.312967:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828321.312973:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10387 00000100:00080000:3.0:1787828321.312976:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828321.312980:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828321.312990:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828321.313008:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043233792/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828321.313018:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828321.313022:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828321.313025:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828321.313029:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10387 00000100:00080000:3.0:1787828321.313032:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828321.313036:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828321.313040:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828321.313068:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634a80 x1874663043233920/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828321.313074:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043233664:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787828321.313169:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828321.313229:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828321/real 1787828321] req@ffff9181c5637b80 x1874663043233664/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828346 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828321.313248:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637b80 x1874663043233664/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828346 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828321.313264:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828321.313269:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828321.313273:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043233664:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828321.313282:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043233792:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828321.313306:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828321.313319:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828321/real 1787828321] req@ffff9181c5634000 x1874663043233792/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828346 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828321.313330:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043233792/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828346 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828321.313337:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828321.313341:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828321.313344:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043233792:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828321.313366:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043233920:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828321.313381:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828321.313398:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828321/real 1787828321] req@ffff9181c5634a80 x1874663043233920/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828346 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828321.313409:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634a80 x1874663043233920/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828346 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828321.313415:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828321.313418:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828321.313421:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043233920:192.168.203.147@tcp:38:kworker.0 10000000:01000000:0.0:1787828321.318815:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:0.0:1787828321.318823:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff9181c87b19b8 from 1 10000000:01000000:0.0:1787828321.318830:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00010000:00010000:0.0:1787828321.318911:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0513800/0xbef42c0221d6f74b lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828321.318924:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e0513800/0xbef42c0221d6f74b lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828321.318938:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e0513800/0xbef42c0221d6f74b lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787828321.318975:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d01a5180 going to sleep for 0 seconds 00000100:00100000:0.0:1787828321.318986:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043234048:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:0.0:1787828321.319006:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e0513800/0xbef42c0221d6f74b lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828321.319016:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e0513800/0xbef42c0221d6f74b lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828321.319029:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0513800/0xbef42c0221d6f74b lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828321.319037:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e0513800/0xbef42c0221d6f74b lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828321.319079:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e0513800/0xbef42c0221d6f74b lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828321.319104:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0513800 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:0.0:1787828321.319108:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e0513800/0xbef42c0221d6f74b lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828321.319116:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e0513800/0xbef42c0221d6f74b lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1787828321.319122:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e0513800/0xbef42c0221d6f74b lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787828321.319127:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:0.0:1787828321.319130:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787828321.319138:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e0513800/0xbef42c0221d6f74b lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828321.319144:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e0513800/0xbef42c0221d6f74b lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787828321.319168:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:0.0:1787828321.319171:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:3.0:1787828321.319234:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828321.319240:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828321.319243:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828321.319249:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10393 00000100:00080000:3.0:1787828321.319256:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828321.319261:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828321.319281:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043234176/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828321.319369:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043234176:192.168.203.147@tcp:250:kworker.0 00000800:00000100:0.0:1787828321.319438:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787828321.319479:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828321/real 1787828321] req@ffff9181c5635c00 x1874663043234176/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828346 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828321.319501:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043234176/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828346 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828321.319515:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828321.319520:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787828321.319542:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043234176:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1787828321.889070:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828322.912919:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828323.937005:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828324.961000:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828325.984901:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828326.432924:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828326.432935:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828326.432940:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828326.432952:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10393 00000100:00080000:2.0:1787828326.432965:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828326.432973:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787828326.433043:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c000 x1874663043234304/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828326.433089:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828326.433092:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828326.433093:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828326.433097:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10393 00000100:00080000:2.0:1787828326.433100:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828326.433103:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828326.433112:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828326.433123:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007dc00 x1874663043234432/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828326.433132:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828326.433137:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:0.0:1787828326.433139:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043234304:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787828326.433139:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828326.433148:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10393 00000100:00080000:2.0:1787828326.433151:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828326.433153:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828326.433157:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828326.433165:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c380 x1874663043234560/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787828326.433218:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787828326.433262:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828326/real 1787828326] req@ffff9181e007c000 x1874663043234304/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828351 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828326.433281:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c000 x1874663043234304/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828351 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828326.433291:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828326.433294:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787828326.433307:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043234304:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787828326.433331:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043234432:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787828326.433362:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787828326.433373:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828326/real 1787828326] req@ffff9181e007dc00 x1874663043234432/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828351 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828326.433382:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007dc00 x1874663043234432/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828351 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828326.433389:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828326.433392:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787828326.433394:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043234432:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787828326.433400:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043234560:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787828326.433413:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787828326.433420:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828326/real 1787828326] req@ffff9181e007c380 x1874663043234560/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828351 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828326.433446:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c380 x1874663043234560/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828351 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828326.433455:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828326.433458:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787828326.433461:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043234560:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828327.008956:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828328.033052:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828329.056956:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828330.080974:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828331.105048:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828331.552936:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828331.552953:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828331.552959:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828331.552997:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10398 00000100:00080000:3.0:1787828331.553012:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828331.553040:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828331.553086:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043234688/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828331.553122:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828331.553127:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828331.553129:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828331.553134:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10398 00000100:00080000:3.0:1787828331.553138:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828331.553142:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828331.553153:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828331.553167:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043234816/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828331.553180:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828331.553184:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828331.553186:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828331.553192:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10398 00000100:00080000:3.0:1787828331.553195:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828331.553200:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828331.553206:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828331.553218:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043234688:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828331.553221:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043234944/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787828331.553346:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787828331.553394:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828331/real 1787828331] req@ffff9181c5636d80 x1874663043234688/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828356 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828331.553420:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043234688/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828356 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828331.553441:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828331.553448:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787828331.553477:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043234688:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787828331.553493:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043234816:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787828331.553547:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787828331.553570:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828331/real 1787828331] req@ffff9181c5637800 x1874663043234816/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828356 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828331.553588:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043234816/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828356 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828331.553633:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828331.553638:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787828331.553642:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043234816:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787828331.553671:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043234944:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787828331.553696:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787828331.553710:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828331/real 1787828331] req@ffff9181c5637100 x1874663043234944/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828356 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828331.553724:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043234944/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828356 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828331.553734:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828331.553738:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787828331.553784:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043234944:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828332.129043:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828333.152873:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828334.176907:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828335.201032:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828336.225009:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828336.672964:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828336.672977:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828336.672982:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828336.672997:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10403 00000100:00080000:2.0:1787828336.673012:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828336.673021:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787828336.673074:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c380 x1874663043235072/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828336.673114:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828336.673119:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828336.673122:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828336.673128:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10403 00000100:00080000:2.0:1787828336.673132:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828336.673137:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828336.673148:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828336.673166:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007dc00 x1874663043235200/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828336.673180:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828336.673185:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828336.673187:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828336.673193:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10403 00000100:00080000:2.0:1787828336.673196:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828336.673203:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787828336.673211:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043235072:192.168.203.147@tcp:250:kworker.0 00000008:00100000:2.0:1787828336.673212:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828336.673232:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c000 x1874663043235328/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828336.673371:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043235200:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787828336.673455:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043235328:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828337.249064:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828338.272985:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828339.297020:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1787828339.745083:0:2378:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:2.0:1787828339.745126:0:2378:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1787828339.745142:0:2378:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:2.0:1787828339.745196:0:2378:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:2.0:1787828339.745204:0:2378:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:0.0:1787828339.745287:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828336/real 1787828339] req@ffff9181e007c000 x1874663043235328/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828361 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828339.745345:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c000 x1874663043235328/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828361 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828339.745361:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828339.745368:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787828339.745373:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043235328:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787828339.745385:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828336/real 1787828339] req@ffff9181e007c380 x1874663043235072/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828361 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828339.745404:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c380 x1874663043235072/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828361 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828339.745416:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828339.745421:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787828339.745451:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043235072:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787828339.745461:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828336/real 1787828339] req@ffff9181e007dc00 x1874663043235200/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828361 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828339.745478:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007dc00 x1874663043235200/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828361 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828339.745489:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828339.745494:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787828339.745499:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043235200:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828340.320986:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:2.0:1787828341.344928:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-sptlrpc' failed (-108). 10000000:00020000:2.0:1787828341.344936:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00010000:00010000:3.0:1787828341.345021:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828341.792897:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828341.792920:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828341.792923:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828341.792933:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10408 00000100:00080000:2.0:1787828341.792944:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828341.792952:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787828341.793004:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ca80 x1874663043235456/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828341.793034:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828341.793038:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828341.793039:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828341.793043:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10408 00000100:00080000:2.0:1787828341.793047:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828341.793055:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828341.793065:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828341.793078:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007fb80 x1874663043235584/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828341.793097:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828341.793104:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:0.0:1787828341.793114:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043235456:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787828341.793114:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828341.793134:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10408 00000100:00080000:2.0:1787828341.793141:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828341.793148:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828341.793156:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828341.793174:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e680 x1874663043235712/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787828341.793243:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787828341.793310:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828341/real 1787828341] req@ffff9181e007ca80 x1874663043235456/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828366 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828341.793343:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ca80 x1874663043235456/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828366 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828341.793361:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828341.793369:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787828341.793375:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043235456:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787828341.793419:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043235584:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787828341.793460:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787828341.793482:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828341/real 1787828341] req@ffff9181e007fb80 x1874663043235584/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828366 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828341.793504:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007fb80 x1874663043235584/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828366 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828341.793517:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828341.793522:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787828341.793527:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043235584:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787828341.793539:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043235712:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787828341.793573:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787828341.793592:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828341/real 1787828341] req@ffff9181e007e680 x1874663043235712/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828366 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828341.793619:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e680 x1874663043235712/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828366 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828341.793633:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828341.793638:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787828341.793643:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043235712:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828342.368944:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:2.0:1787828343.264889:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:2.0:1787828343.264933:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff9181c5620000 from 341 10000000:01000000:2.0:1787828343.264940:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:2.0:1787828343.264999:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f752 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787828343.265009:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f752 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787828343.265019:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f752 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787828343.265036:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5c94c00 going to sleep for 0 seconds 00000100:00100000:2.0:1787828343.265046:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043235840:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:2.0:1787828343.265059:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f752 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787828343.265069:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f752 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787828343.265073:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f752 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787828343.265078:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f752 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787828343.265083:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f752 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787828343.265088:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181d1843400 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:2.0:1787828343.265091:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f752 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787828343.265096:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f752 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1787828343.265100:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f752 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787828343.265104:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:2.0:1787828343.265106:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787828343.265111:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f752 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787828343.265115:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181d1843400/0xbef42c0221d6f752 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787828343.265158:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:2.0:1787828343.265161:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:3.0:1787828343.265249:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828343.265278:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828343.265285:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828343.265302:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10413 00000100:00080000:3.0:1787828343.265322:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828343.265343:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828343.265393:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043235968/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828343.265431:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828343.265435:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828343.265438:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828343.265444:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10413 00000100:00080000:3.0:1787828343.265449:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828343.265454:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828343.265468:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828343.265486:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043236096/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828343.265494:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043235968:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787828343.265509:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828343.265515:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828343.265518:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828343.265523:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10413 00000100:00080000:3.0:1787828343.265526:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828343.265531:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828343.265536:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828343.265549:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043236224/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787828343.265602:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787828343.265670:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828343/real 1787828343] req@ffff9181c5637480 x1874663043235968/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828368 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828343.265706:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043235968/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828368 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828343.265727:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828343.265735:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787828343.265832:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043235968:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787828343.265877:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043236096:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787828343.265955:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787828343.265982:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828343/real 1787828343] req@ffff9181c5636a00 x1874663043236096/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828368 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828343.266001:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043236096/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828368 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828343.266020:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828343.266025:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787828343.266029:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043236096:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787828343.266036:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043236224:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787828343.266058:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787828343.266070:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828343/real 1787828343] req@ffff9181c5634380 x1874663043236224/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828368 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828343.266085:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043236224/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828368 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828343.266094:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828343.266097:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787828343.266101:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043236224:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828343.393029:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828344.416955:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828345.440938:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828346.465073:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828347.488928:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828348.448875:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828348.448887:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828348.448892:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828348.448905:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10415 00000100:00080000:3.0:1787828348.448936:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828348.448948:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828348.449013:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634e00 x1874663043236352/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828348.449054:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828348.449059:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828348.449062:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828348.449068:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10415 00000100:00080000:3.0:1787828348.449083:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828348.449089:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828348.449103:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828348.449122:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043236480/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828348.449143:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043236352:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787828348.449145:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828348.449165:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828348.449171:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828348.449196:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10415 00000100:00080000:3.0:1787828348.449200:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828348.449205:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828348.449212:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828348.449226:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635f80 x1874663043236608/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787828348.449271:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787828348.449341:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828348/real 1787828348] req@ffff9181c5634e00 x1874663043236352/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828373 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828348.449366:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634e00 x1874663043236352/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828373 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828348.449380:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828348.449386:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787828348.449412:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043236352:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787828348.449446:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043236480:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787828348.449478:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787828348.449511:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828348/real 1787828348] req@ffff9181c5635500 x1874663043236480/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828373 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828348.449538:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043236480/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828373 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828348.449555:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828348.449562:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787828348.449570:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043236480:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787828348.449584:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043236608:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787828348.449621:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787828348.449637:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828348/real 1787828348] req@ffff9181c5635f80 x1874663043236608/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828373 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828348.449652:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635f80 x1874663043236608/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828373 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828348.449663:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828348.449667:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787828348.449671:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043236608:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828348.513041:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828349.536989:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828350.561046:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828351.584896:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828352.608994:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828353.569004:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828353.569021:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828353.569029:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828353.569045:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10420 00000100:00080000:2.0:1787828353.569063:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828353.569072:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787828353.569135:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d880 x1874663043236736/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828353.569194:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828353.569200:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828353.569202:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828353.569207:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10420 00000100:00080000:2.0:1787828353.569212:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828353.569227:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828353.569238:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828353.569253:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ce00 x1874663043236864/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828353.569269:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828353.569273:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828353.569275:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828353.569280:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10420 00000100:00080000:2.0:1787828353.569287:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828353.569292:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828353.569299:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828353.569306:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043236736:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828353.569319:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f480 x1874663043236992/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787828353.569433:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787828353.569495:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828353/real 1787828353] req@ffff9181e007d880 x1874663043236736/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828378 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828353.569521:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d880 x1874663043236736/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828378 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828353.569535:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828353.569541:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787828353.569562:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043236736:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787828353.569572:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043236864:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787828353.569602:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787828353.569625:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828353/real 1787828353] req@ffff9181e007ce00 x1874663043236864/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828378 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828353.569646:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ce00 x1874663043236864/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828378 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828353.569667:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828353.569672:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787828353.569676:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043236864:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787828353.569705:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043236992:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787828353.569737:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787828353.569817:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828353/real 1787828353] req@ffff9181e007f480 x1874663043236992/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828378 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828353.569870:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f480 x1874663043236992/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828378 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828353.569893:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828353.569897:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787828353.569901:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043236992:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828353.633051:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828354.656964:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828355.681053:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828356.704935:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828357.728972:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828358.688839:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828358.688848:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828358.688852:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828358.688861:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10425 00000100:00080000:2.0:1787828358.688870:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828358.688877:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787828358.688914:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f100 x1874663043237120/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828358.688939:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828358.688942:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828358.688944:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828358.688947:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10425 00000100:00080000:2.0:1787828358.688950:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828358.688953:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828358.688961:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828358.688971:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f800 x1874663043237248/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828358.688979:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828358.688983:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828358.688985:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828358.688988:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10425 00000100:00080000:2.0:1787828358.688992:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1787828358.688995:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043237120:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787828358.688997:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828358.689005:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828358.689013:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ea00 x1874663043237376/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787828358.689071:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787828358.689113:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828358/real 1787828358] req@ffff9181e007f100 x1874663043237120/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828383 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828358.689137:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f100 x1874663043237120/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828383 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828358.689149:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828358.689155:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787828358.689173:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043237120:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787828358.689182:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043237248:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787828358.689214:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787828358.689232:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828358/real 1787828358] req@ffff9181e007f800 x1874663043237248/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828383 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828358.689247:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f800 x1874663043237248/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828383 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828358.689258:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828358.689262:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787828358.689266:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043237248:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787828358.689302:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043237376:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787828358.689329:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787828358.689343:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828358/real 1787828358] req@ffff9181e007ea00 x1874663043237376/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828383 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828358.689358:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ea00 x1874663043237376/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828383 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828358.689369:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828358.689373:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787828358.689377:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043237376:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828358.752907:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828359.776945:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828360.801054:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828361.824979:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828362.848984:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:2.0:1787828363.296878:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-cliir' failed (-108). 10000000:00000400:2.0:1787828363.296886:0:159735:0:(mgc_request.c:1900:mgc_process_log()) MGC192.168.203.147@tcp: IR log lustre-cliir failed, not fatal: rc = -108 10000000:01000000:2.0:1787828363.296893:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:2.0:1787828363.296894:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff9181c5620000 from 52 10000000:01000000:2.0:1787828363.296897:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:2.0:1787828363.296937:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1843e00/0xbef42c0221d6f759 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787828363.296946:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181d1843e00/0xbef42c0221d6f759 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787828363.296956:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181d1843e00/0xbef42c0221d6f759 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787828363.296979:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5c94c00 going to sleep for 0 seconds 00000100:00100000:2.0:1787828363.296989:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043237504:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:2.0:1787828363.296999:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181d1843e00/0xbef42c0221d6f759 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787828363.297009:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181d1843e00/0xbef42c0221d6f759 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787828363.297014:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1843e00/0xbef42c0221d6f759 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787828363.297019:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181d1843e00/0xbef42c0221d6f759 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787828363.297024:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181d1843e00/0xbef42c0221d6f759 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787828363.297028:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181d1843e00 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:2.0:1787828363.297030:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181d1843e00/0xbef42c0221d6f759 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787828363.297035:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181d1843e00/0xbef42c0221d6f759 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1787828363.297040:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181d1843e00/0xbef42c0221d6f759 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787828363.297044:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:2.0:1787828363.297046:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787828363.297051:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181d1843e00/0xbef42c0221d6f759 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787828363.297056:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181d1843e00/0xbef42c0221d6f759 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787828363.297075:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:2.0:1787828363.297077:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:3.0:1787828363.297190:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828363.297200:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828363.297205:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828363.297218:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10430 00000100:00080000:3.0:1787828363.297233:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828363.297243:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828363.297290:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635f80 x1874663043237632/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828363.297329:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828363.297335:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828363.297338:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828363.297343:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10430 00000100:00080000:3.0:1787828363.297347:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828363.297354:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828363.297368:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828363.297385:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043237760/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828363.297400:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043237632:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787828363.297404:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828363.297416:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828363.297419:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828363.297426:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10430 00000100:00080000:3.0:1787828363.297430:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828363.297435:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828363.297442:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828363.297456:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634e00 x1874663043237888/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787828363.297529:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787828363.297607:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828363/real 1787828363] req@ffff9181c5635f80 x1874663043237632/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828388 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828363.297638:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635f80 x1874663043237632/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828388 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828363.297655:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828363.297660:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787828363.297680:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043237632:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787828363.297716:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043237760:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787828363.297851:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787828363.297883:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828363/real 1787828363] req@ffff9181c5635500 x1874663043237760/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828388 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828363.297904:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043237760/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828388 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828363.297916:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828363.297920:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787828363.297924:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043237760:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787828363.297933:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043237888:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787828363.297977:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787828363.297992:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828363/real 1787828363] req@ffff9181c5634e00 x1874663043237888/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828388 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828363.298009:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634e00 x1874663043237888/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828388 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828363.298019:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828363.298023:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787828363.298028:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043237888:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828363.873080:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828364.897075:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828365.920940:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828366.944984:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828367.968987:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828368.416914:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828368.416925:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828368.416930:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828368.416943:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10435 00000100:00080000:2.0:1787828368.416956:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828368.416965:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787828368.417009:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c700 x1874663043238016/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828368.417072:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828368.417077:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828368.417080:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828368.417085:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10435 00000100:00080000:2.0:1787828368.417091:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828368.417098:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828368.417127:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828368.417143:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043238016:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828368.417149:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e300 x1874663043238144/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828368.417165:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828368.417169:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828368.417171:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828368.417176:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10435 00000100:00080000:2.0:1787828368.417182:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828368.417187:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828368.417193:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828368.417206:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d180 x1874663043238272/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787828368.417232:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787828368.417280:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828368/real 1787828368] req@ffff9181e007c700 x1874663043238016/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828393 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828368.417302:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c700 x1874663043238016/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828393 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828368.417323:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828368.417327:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787828368.417346:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043238016:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787828368.417375:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043238144:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787828368.417404:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787828368.417418:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828368/real 1787828368] req@ffff9181e007e300 x1874663043238144/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828393 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828368.417432:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e300 x1874663043238144/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828393 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828368.417441:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828368.417444:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787828368.417448:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043238144:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787828368.417454:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043238272:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787828368.417470:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787828368.417480:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828368/real 1787828368] req@ffff9181e007d180 x1874663043238272/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828393 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828368.417492:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d180 x1874663043238272/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828393 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828368.417501:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828368.417504:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787828368.417507:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043238272:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828368.992886:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828370.017006:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828371.040978:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828372.064875:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828373.088848:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828373.536802:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828373.536810:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828373.536814:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828373.536822:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10440 00000100:00080000:2.0:1787828373.536832:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828373.536844:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787828373.536878:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d500 x1874663043238400/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828373.536900:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828373.536903:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828373.536904:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828373.536907:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10440 00000100:00080000:2.0:1787828373.536909:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828373.536912:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828373.536920:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828373.536929:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ed80 x1874663043238528/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828373.536938:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1787828373.536941:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043238400:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787828373.536941:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828373.536945:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828373.536951:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10440 00000100:00080000:2.0:1787828373.536954:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828373.536957:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828373.536960:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828373.536969:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007df80 x1874663043238656/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787828373.537003:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787828373.537034:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828373/real 1787828373] req@ffff9181e007d500 x1874663043238400/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828398 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828373.537049:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d500 x1874663043238400/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828398 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828373.537057:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828373.537060:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787828373.537072:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043238400:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787828373.537093:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043238528:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787828373.537110:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787828373.537118:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828373/real 1787828373] req@ffff9181e007ed80 x1874663043238528/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828398 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828373.537126:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ed80 x1874663043238528/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828398 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828373.537132:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828373.537134:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787828373.537137:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043238528:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787828373.537140:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043238656:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787828373.537151:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787828373.537156:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828373/real 1787828373] req@ffff9181e007df80 x1874663043238656/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828398 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828373.537164:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007df80 x1874663043238656/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828398 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828373.537169:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828373.537171:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787828373.537173:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043238656:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828374.112874:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828375.136971:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828376.160995:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828377.184855:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828378.208854:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828378.656861:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828378.656869:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828378.656873:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828378.656882:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10445 00000100:00080000:3.0:1787828378.656892:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828378.656899:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828378.656933:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043238784/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828378.656957:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828378.656960:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828378.656962:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828378.656966:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10445 00000100:00080000:3.0:1787828378.656968:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828378.656971:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828378.656979:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828378.656990:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043238912/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828378.656998:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828378.657001:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828378.657002:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828378.657004:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10445 00000100:00080000:3.0:1787828378.657006:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1787828378.657011:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043238784:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787828378.657013:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828378.657019:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828378.657030:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043239040/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787828378.657076:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787828378.657118:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828378/real 1787828378] req@ffff9181c5634380 x1874663043238784/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828403 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828378.657139:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043238784/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828403 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828378.657148:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828378.657151:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787828378.657164:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043238784:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787828378.657170:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043238912:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787828378.657202:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787828378.657215:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828378/real 1787828378] req@ffff9181c5636a00 x1874663043238912/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828403 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828378.657224:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043238912/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828403 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828378.657235:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828378.657237:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787828378.657240:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043238912:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787828378.657259:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043239040:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787828378.657271:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787828378.657281:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828378/real 1787828378] req@ffff9181c5637480 x1874663043239040/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828403 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828378.657295:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043239040/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828403 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828378.657304:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828378.657323:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787828378.657327:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043239040:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828379.232953:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828380.257006:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828381.280884:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828382.304958:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828383.328914:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828383.776882:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 10000000:01000000:2.0:1787828383.776884:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-client' failed (-108). 10000000:00020000:2.0:1787828383.776897:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00080000:3.0:1787828383.776900:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 10000000:01000000:2.0:1787828383.776907:0:159735:0:(mgc_request.c:611:do_requeue()) updating log params 00000100:00080000:3.0:1787828383.776909:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 10000000:01000000:2.0:1787828383.776916:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff9181c5620000 from 7 10000000:01000000:2.0:1787828383.776923:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00000100:00080000:3.0:1787828383.776925:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10450 00000100:00080000:3.0:1787828383.776953:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828383.776963:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828383.777003:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043239296/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:2.0:1787828383.777018:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1843e00/0xbef42c0221d6f760 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787828383.777058:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181d1843e00/0xbef42c0221d6f760 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787828383.777075:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181d1843e00/0xbef42c0221d6f760 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787828383.777099:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5c94c00 going to sleep for 0 seconds 00000100:00080000:3.0:1787828383.777110:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1787828383.777112:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007df80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043239168:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00080000:3.0:1787828383.777129:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828383.777137:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00010000:00010000:2.0:1787828383.777140:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181d1843e00/0xbef42c0221d6f760 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787828383.777152:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10450 00010000:00010000:2.0:1787828383.777157:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181d1843e00/0xbef42c0221d6f760 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787828383.777165:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828383.777184:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:2.0:1787828383.777187:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1843e00/0xbef42c0221d6f760 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787828383.777205:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043239296:192.168.203.147@tcp:250:kworker.0 00000008:00100000:3.0:1787828383.777206:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:2.0:1787828383.777230:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181d1843e00/0xbef42c0221d6f760 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787828383.777247:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181d1843e00/0xbef42c0221d6f760 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787828383.777254:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043239424/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:2.0:1787828383.777256:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181d1843e00 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:2.0:1787828383.777268:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181d1843e00/0xbef42c0221d6f760 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787828383.777288:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:2.0:1787828383.777302:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181d1843e00/0xbef42c0221d6f760 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787828383.777305:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828383.777312:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:0.0:1787828383.777322:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1787828383.777331:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10450 10000000:00010000:2.0:1787828383.777336:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181d1843e00/0xbef42c0221d6f760 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787828383.777342:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 10000000:01000000:2.0:1787828383.777353:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00000100:00080000:3.0:1787828383.777358:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:2.0:1787828383.777362:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00000008:00100000:3.0:1787828383.777370:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:2.0:1787828383.777377:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181d1843e00/0xbef42c0221d6f760 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787828383.777388:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828383/real 1787828383] req@ffff9181c5637100 x1874663043239296/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828408 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828383.777401:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043239552/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:2.0:1787828383.777402:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181d1843e00/0xbef42c0221d6f760 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787828383.777427:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043239296/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828408 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 10000000:01000000:2.0:1787828383.777474:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:0.0:1787828383.777478:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828383.777480:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:CONNECTING 00000100:00080000:0.0:1787828383.777489:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787828383.777500:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043239296:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787828383.777543:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1787828383.777552:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043239424:192.168.203.147@tcp:38:kworker.0 00000100:00080000:3.0:1787828383.777554:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828383.777562:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828383.777571:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10455 00000100:00080000:3.0:1787828383.777578:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828383.777585:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:0.0:1787828383.777600:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828383.777635:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043239680/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828383.777662:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828383/real 1787828383] req@ffff9181c5637800 x1874663043239424/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828408 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828383.777689:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043239424/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828408 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828383.777708:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828383.777715:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787828383.777722:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043239424:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787828383.777735:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043239552:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787828383.777850:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787828383.777881:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828383/real 1787828383] req@ffff9181c5636d80 x1874663043239552/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828408 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828383.777899:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043239552/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828408 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828383.777910:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828383.777914:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787828383.777918:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043239552:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787828383.777938:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043239680:192.168.203.147@tcp:250:kworker.0 00000800:00000100:0.0:1787828383.777964:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787828383.777983:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828383/real 1787828383] req@ffff9181c5635c00 x1874663043239680/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828408 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828383.777998:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043239680/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828408 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828383.778008:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828383.778012:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787828383.778031:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043239680:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787828384.353046:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828385.376883:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828386.400953:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828387.424915:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828388.449058:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828388.896862:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828388.896877:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828388.896882:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828388.896897:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10455 00000100:00080000:3.0:1787828388.896911:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828388.896949:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828388.897020:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634a80 x1874663043239808/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828388.897060:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828388.897067:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828388.897071:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828388.897079:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10455 00000100:00080000:3.0:1787828388.897086:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828388.897094:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828388.897110:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828388.897150:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043239936/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828388.897168:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828388.897173:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:0.0:1787828388.897176:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043239808:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787828388.897178:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828388.897184:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10455 00000100:00080000:3.0:1787828388.897189:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828388.897195:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828388.897203:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828388.897217:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637b80 x1874663043240064/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787828388.897281:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787828388.897364:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828388/real 1787828388] req@ffff9181c5634a80 x1874663043239808/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828413 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828388.897388:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634a80 x1874663043239808/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828413 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828388.897401:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828388.897406:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787828388.897423:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043239808:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787828388.897467:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043239936:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787828388.897513:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787828388.897532:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828388/real 1787828388] req@ffff9181c5634000 x1874663043239936/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828413 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828388.897547:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043239936/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828413 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828388.897564:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828388.897568:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787828388.897572:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043239936:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787828388.897600:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043240064:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787828388.897632:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787828388.897643:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828388/real 1787828388] req@ffff9181c5637b80 x1874663043240064/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828413 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828388.897657:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637b80 x1874663043240064/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828413 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828388.897666:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828388.897669:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787828388.897672:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043240064:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828389.472893:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828390.497015:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828391.520954:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828392.544978:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828393.569039:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828394.016866:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828394.016878:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828394.016882:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828394.016893:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10460 00000100:00080000:0.0:1787828394.016905:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828394.016913:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787828394.016961:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5f80 x1874663043240192/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828394.017018:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828394.017026:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828394.017028:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828394.017034:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10460 00000100:00080000:0.0:1787828394.017039:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828394.017044:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828394.017066:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828394.017092:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4700 x1874663043240320/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828394.017110:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828394.017133:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828394.017137:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828394.017149:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10460 00000100:00080000:0.0:1787828394.017158:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1787828394.017167:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043240192:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787828394.017169:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828394.017193:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828394.017229:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5c00 x1874663043240448/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828394.017313:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828394.017378:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828394/real 1787828394] req@ffff9181c55a5f80 x1874663043240192/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828419 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828394.017404:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5f80 x1874663043240192/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828419 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828394.017422:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828394.017428:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828394.017459:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043240192:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828394.017472:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043240320:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828394.017519:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828394.017576:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828394/real 1787828394] req@ffff9181c55a4700 x1874663043240320/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828419 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828394.017593:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4700 x1874663043240320/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828419 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828394.017604:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828394.017609:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828394.017614:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043240320:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828394.017651:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043240448:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828394.017677:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828394.017690:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828394/real 1787828394] req@ffff9181c55a5c00 x1874663043240448/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828419 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828394.017715:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5c00 x1874663043240448/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828419 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828394.017725:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828394.017729:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828394.017732:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043240448:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828394.592947:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828395.616904:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828396.640892:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828397.665001:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828398.688911:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828399.136834:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828399.136849:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828399.136855:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828399.136869:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10465 00000100:00080000:0.0:1787828399.136884:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828399.136897:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787828399.136952:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4380 x1874663043240576/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828399.137004:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828399.137011:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828399.137013:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828399.137017:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10465 00000100:00080000:0.0:1787828399.137022:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828399.137027:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828399.137037:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828399.137049:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5180 x1874663043240704/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828399.137060:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043240576:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787828399.137061:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828399.137067:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828399.137071:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828399.137075:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10465 00000100:00080000:0.0:1787828399.137077:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828399.137079:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828399.137082:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828399.137091:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5500 x1874663043240832/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828399.137143:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828399.137182:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828399/real 1787828399] req@ffff9181c55a4380 x1874663043240576/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828424 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828399.137197:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4380 x1874663043240576/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828424 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828399.137207:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828399.137211:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828399.137225:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043240576:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828399.137256:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043240704:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828399.137282:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828399.137293:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828399/real 1787828399] req@ffff9181c55a5180 x1874663043240704/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828424 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828399.137302:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5180 x1874663043240704/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828424 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828399.137313:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828399.137316:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828399.137318:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043240704:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828399.137323:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043240832:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828399.137340:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828399.137351:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828399/real 1787828399] req@ffff9181c55a5500 x1874663043240832/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828424 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828399.137365:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5500 x1874663043240832/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828424 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828399.137374:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828399.137378:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828399.137381:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043240832:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828399.712934:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828400.736916:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828401.761018:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828402.784892:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828403.808953:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828404.256852:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828404.256862:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 10000000:01000000:0.0:1787828404.256863:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'params' failed (-108). 00000100:00080000:3.0:1787828404.256866:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 10000000:00020000:0.0:1787828404.256868:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:0.0:1787828404.256875:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:0.0:1787828404.256877:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff9181c87b19b8 from 1 00000100:00080000:3.0:1787828404.256882:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10470 10000000:01000000:0.0:1787828404.256883:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00000100:00080000:3.0:1787828404.256896:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828404.256904:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828404.256933:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637b80 x1874663043241088/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:0.0:1787828404.256950:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0513200/0xbef42c0221d6f767 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828404.256962:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e0513200/0xbef42c0221d6f767 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828404.256972:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e0513200/0xbef42c0221d6f767 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787828404.256973:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828404.256983:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828404.256985:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1787828404.256993:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d01a5300 going to sleep for 0 seconds 00000100:00080000:3.0:1787828404.256993:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10470 00000100:00080000:3.0:1787828404.257001:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1787828404.257004:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043240960:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00080000:3.0:1787828404.257010:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787828404.257017:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043241088:192.168.203.147@tcp:250:kworker.0 00010000:00010000:0.0:1787828404.257018:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e0513200/0xbef42c0221d6f767 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000008:00100000:3.0:1787828404.257024:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:0.0:1787828404.257035:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e0513200/0xbef42c0221d6f767 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828404.257041:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0513200/0xbef42c0221d6f767 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828404.257051:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e0513200/0xbef42c0221d6f767 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787828404.257052:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043241216/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:0.0:1787828404.257070:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e0513200/0xbef42c0221d6f767 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828404.257089:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0513200 already unused, calling callback (ffffffffc1273c20) 00000100:00080000:3.0:1787828404.257089:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 10000000:00010000:0.0:1787828404.257095:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e0513200/0xbef42c0221d6f767 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787828404.257105:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00010000:00010000:0.0:1787828404.257106:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e0513200/0xbef42c0221d6f767 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787828404.257109:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 10000000:00010000:0.0:1787828404.257116:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e0513200/0xbef42c0221d6f767 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787828404.257120:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10470 10000000:01000000:0.0:1787828404.257125:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00000100:00100000:2.0:1787828404.257127:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043241216:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1787828404.257130:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00000100:00080000:3.0:1787828404.257131:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00010000:00010000:0.0:1787828404.257138:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e0513200/0xbef42c0221d6f767 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787828404.257142:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:0.0:1787828404.257148:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e0513200/0xbef42c0221d6f767 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000008:00100000:3.0:1787828404.257153:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 10000000:01000000:0.0:1787828404.257169:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:0.0:1787828404.257174:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:CONNECTING 00000100:00100000:3.0:1787828404.257175:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634a80 x1874663043241344/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828404.257222:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043241344:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828404.832848:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828405.857023:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828406.880979:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1787828407.329253:0:2379:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:1.0:1787828407.329356:0:2379:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787828407.329380:0:2379:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1787828407.329481:0:2379:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1787828407.329504:0:2379:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:2.0:1787828407.329571:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828404/real 1787828407] req@ffff9181c5634a80 x1874663043241344/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828429 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828407.329610:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634a80 x1874663043241344/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828429 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828407.329628:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828407.329635:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828407.329641:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043241344:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828407.329656:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828404/real 1787828407] req@ffff9181c5634000 x1874663043241216/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828429 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828407.329681:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043241216/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828429 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828407.329711:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828407.329719:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828407.329727:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043241216:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828407.329790:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828404/real 1787828407] req@ffff9181c5637b80 x1874663043241088/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828429 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828407.329833:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637b80 x1874663043241088/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828429 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828407.329849:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828407.329922:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828407.329952:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043241088:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787828407.330162:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828407.330175:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828407.330181:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828407.330195:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10475 00000100:00080000:3.0:1787828407.330208:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828407.330222:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828407.330317:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043241472/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828407.330385:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828407.330393:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828407.330397:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828407.330416:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10475 00000100:00080000:3.0:1787828407.330425:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828407.330433:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828407.330479:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828407.330509:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043241600/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828407.330531:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828407.330538:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828407.330541:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828407.330549:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10475 00000100:00080000:3.0:1787828407.330557:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828407.330566:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828407.330576:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828407.330599:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043241728/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828407.330879:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043241472:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787828407.330994:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828407.331034:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828407/real 1787828407] req@ffff9181c5635c00 x1874663043241472/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828432 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828407.331052:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043241472/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828432 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828407.331063:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828407.331069:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828407.331108:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043241472:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828407.331123:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043241600:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828407.331177:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828407.331205:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828407/real 1787828407] req@ffff9181c5636d80 x1874663043241600/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828432 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828407.331228:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043241600/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828432 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828407.331239:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828407.331244:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828407.331248:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043241600:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828407.331256:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043241728:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828407.331287:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828407.331309:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828407/real 1787828407] req@ffff9181c5637800 x1874663043241728/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828432 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828407.331333:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043241728/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828432 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828407.331349:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828407.331357:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828407.331367:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043241728:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828407.905097:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828408.929006:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828409.952903:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828410.976848:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828412.000959:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828412.449002:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828412.449016:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828412.449022:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828412.449035:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10479 00000100:00080000:0.0:1787828412.449049:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828412.449059:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787828412.449109:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7800 x1874663043241856/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828412.449159:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828412.449163:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828412.449165:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828412.449170:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10479 00000100:00080000:0.0:1787828412.449174:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828412.449178:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828412.449187:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828412.449240:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4e00 x1874663043241984/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828412.449255:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828412.449259:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828412.449262:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828412.449267:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10479 00000100:00080000:0.0:1787828412.449274:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828412.449280:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828412.449287:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828412.449303:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5500 x1874663043242112/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828412.449312:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043241856:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787828412.449408:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828412.449484:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828412/real 1787828412] req@ffff9181c55a7800 x1874663043241856/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828437 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828412.449517:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7800 x1874663043241856/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828437 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828412.449531:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828412.449536:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828412.449558:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043241856:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828412.449569:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043241984:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828412.449602:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828412.449627:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828412/real 1787828412] req@ffff9181c55a4e00 x1874663043241984/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828437 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828412.449644:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4e00 x1874663043241984/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828437 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828412.449660:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828412.449667:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828412.449674:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043241984:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828412.449728:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043242112:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828412.449812:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828412.449835:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828412/real 1787828412] req@ffff9181c55a5500 x1874663043242112/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828437 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828412.449852:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5500 x1874663043242112/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828437 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828412.449871:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828412.449875:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828412.449879:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043242112:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828413.024918:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828414.049009:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828415.072999:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828416.097007:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828417.120946:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828417.569015:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828417.569031:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828417.569038:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828417.569055:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10484 00000100:00080000:2.0:1787828417.569072:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828417.569083:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787828417.569160:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007df80 x1874663043242240/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828417.569243:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828417.569250:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828417.569252:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828417.569259:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10484 00000100:00080000:2.0:1787828417.569265:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828417.569272:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828417.569287:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828417.569310:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ed80 x1874663043242368/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828417.569327:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787828417.569332:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043242240:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787828417.569334:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828417.569343:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828417.569355:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10484 00000100:00080000:2.0:1787828417.569359:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828417.569364:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828417.569405:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828417.569434:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d500 x1874663043242496/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828417.569512:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828417.569583:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828417/real 1787828417] req@ffff9181e007df80 x1874663043242240/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828442 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828417.569612:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007df80 x1874663043242240/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828442 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828417.569628:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828417.569634:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828417.569660:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043242240:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828417.569704:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043242368:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828417.569819:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828417.569840:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828417/real 1787828417] req@ffff9181e007ed80 x1874663043242368/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828442 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828417.569857:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ed80 x1874663043242368/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828442 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828417.569869:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828417.569886:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828417.569891:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043242368:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828417.569900:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043242496:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828417.569926:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828417.569940:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828417/real 1787828417] req@ffff9181e007d500 x1874663043242496/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828442 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828417.569955:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d500 x1874663043242496/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828442 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828417.569974:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828417.569979:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828417.569983:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043242496:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828418.144903:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828419.169006:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828420.192984:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828421.216900:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828422.240906:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828422.688929:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828422.688938:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828422.688941:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828422.688951:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10489 00000100:00080000:0.0:1787828422.688962:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828422.688968:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787828422.689005:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5180 x1874663043242624/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828422.689036:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828422.689039:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828422.689041:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828422.689044:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10489 00000100:00080000:0.0:1787828422.689046:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828422.689050:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828422.689058:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828422.689067:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4380 x1874663043242752/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828422.689074:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828422.689077:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828422.689078:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828422.689098:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10489 00000100:00080000:0.0:1787828422.689101:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828422.689104:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828422.689107:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828422.689116:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5c00 x1874663043242880/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828422.689149:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043242624:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1787828422.689215:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828422.689252:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828422/real 1787828422] req@ffff9181c55a5180 x1874663043242624/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828447 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828422.689268:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5180 x1874663043242624/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828447 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828422.689278:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828422.689281:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828422.689297:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043242624:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828422.689304:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043242752:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828422.689324:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828422.689375:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828422/real 1787828422] req@ffff9181c55a4380 x1874663043242752/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828447 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828422.689389:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4380 x1874663043242752/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828447 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828422.689397:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828422.689400:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828422.689403:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043242752:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828422.689409:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043242880:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828422.689429:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828422.689439:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828422/real 1787828422] req@ffff9181c55a5c00 x1874663043242880/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828447 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828422.689449:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5c00 x1874663043242880/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828447 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828422.689456:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828422.689459:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828422.689461:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043242880:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828423.264946:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828424.289001:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:2.0:1787828424.289039:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-sptlrpc' failed (-108). 10000000:00020000:2.0:1787828424.289051:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00010000:00010000:3.0:1787828425.312924:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:2.0:1787828426.272928:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:2.0:1787828426.272940:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff9181c5620000 from 341 10000000:01000000:2.0:1787828426.272950:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:2.0:1787828426.273024:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1843200/0xbef42c0221d6f76e lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787828426.273046:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181d1843200/0xbef42c0221d6f76e lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787828426.273066:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181d1843200/0xbef42c0221d6f76e lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787828426.273100:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5c94c00 going to sleep for 0 seconds 00000100:00100000:2.0:1787828426.273140:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043243008:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:2.0:1787828426.273167:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181d1843200/0xbef42c0221d6f76e lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787828426.273185:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181d1843200/0xbef42c0221d6f76e lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787828426.273211:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1843200/0xbef42c0221d6f76e lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787828426.273227:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181d1843200/0xbef42c0221d6f76e lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787828426.273240:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181d1843200/0xbef42c0221d6f76e lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787828426.273252:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181d1843200 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:2.0:1787828426.273259:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181d1843200/0xbef42c0221d6f76e lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787828426.273273:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181d1843200/0xbef42c0221d6f76e lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1787828426.273285:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181d1843200/0xbef42c0221d6f76e lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787828426.273296:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:2.0:1787828426.273302:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787828426.273312:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181d1843200/0xbef42c0221d6f76e lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787828426.273357:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181d1843200/0xbef42c0221d6f76e lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787828426.273409:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:2.0:1787828426.273416:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:1.0:1787828426.273532:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828426.273547:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828426.273554:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828426.273569:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10494 00000100:00080000:1.0:1787828426.273586:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828426.273597:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787828426.273657:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2d80 x1874663043243136/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828426.273699:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828426.273705:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828426.273708:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828426.273713:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10494 00000100:00080000:1.0:1787828426.273718:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828426.273723:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828426.273741:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828426.273830:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0a80 x1874663043243264/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828426.273839:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043243136:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787828426.273870:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828426.273876:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828426.273879:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828426.273888:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10494 00000100:00080000:1.0:1787828426.273895:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828426.273904:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828426.273914:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828426.273937:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2a00 x1874663043243392/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828426.273968:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828426.274027:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828426/real 1787828426] req@ffff9181c35b2d80 x1874663043243136/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828451 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828426.274052:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2d80 x1874663043243136/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828451 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828426.274068:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828426.274074:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828426.274100:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043243136:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828426.274135:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043243264:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828426.274177:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828426.274196:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828426/real 1787828426] req@ffff9181c35b0a80 x1874663043243264/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828451 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828426.274213:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0a80 x1874663043243264/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828451 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828426.274224:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828426.274228:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828426.274233:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043243264:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828426.274240:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043243392:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828426.274259:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828426.274270:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828426/real 1787828426] req@ffff9181c35b2a00 x1874663043243392/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828451 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828426.274286:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2a00 x1874663043243392/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828451 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828426.274302:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828426.274310:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828426.274348:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043243392:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828426.337030:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828427.360988:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828428.385031:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828429.408974:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828430.433003:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828431.392877:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828431.392891:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828431.392896:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828431.392909:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10498 00000100:00080000:1.0:1787828431.392922:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828431.392932:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787828431.392979:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2680 x1874663043243520/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828431.393031:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828431.393037:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828431.393040:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828431.393046:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10498 00000100:00080000:1.0:1787828431.393052:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828431.393059:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828431.393071:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828431.393084:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043243520:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787828431.393094:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3480 x1874663043243648/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828431.393112:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828431.393117:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828431.393120:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828431.393126:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10498 00000100:00080000:1.0:1787828431.393131:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828431.393138:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828431.393146:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828431.393166:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0e00 x1874663043243776/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828431.393172:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828431.393220:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828431/real 1787828431] req@ffff9181c35b2680 x1874663043243520/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828456 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828431.393248:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2680 x1874663043243520/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828456 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828431.393263:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828431.393268:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828431.393301:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043243520:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828431.393334:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043243648:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828431.393365:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828431.393378:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828431/real 1787828431] req@ffff9181c35b3480 x1874663043243648/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828456 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828431.393389:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3480 x1874663043243648/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828456 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828431.393397:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828431.393400:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828431.393403:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043243648:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828431.393408:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043243776:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828431.393432:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828431.393448:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828431/real 1787828431] req@ffff9181c35b0e00 x1874663043243776/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828456 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828431.393465:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0e00 x1874663043243776/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828456 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828431.393485:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828431.393488:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828431.393491:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043243776:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828431.456919:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828432.480853:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828433.504971:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828434.528858:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828435.552883:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828436.512878:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828436.512891:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828436.512897:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828436.512910:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10503 00000100:00080000:1.0:1787828436.512924:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828436.512933:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787828436.512979:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0000 x1874663043243904/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828436.513014:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828436.513019:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828436.513022:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828436.513028:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10503 00000100:00080000:1.0:1787828436.513032:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828436.513046:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828436.513059:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828436.513076:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1180 x1874663043244032/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828436.513092:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828436.513096:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828436.513099:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828436.513108:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10503 00000100:00100000:3.0:1787828436.513112:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043243904:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787828436.513114:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828436.513134:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828436.513147:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828436.513162:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3100 x1874663043244160/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828436.513236:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828436.513326:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828436/real 1787828436] req@ffff9181c35b0000 x1874663043243904/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828461 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828436.513373:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0000 x1874663043243904/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828461 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828436.513395:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828436.513405:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828436.513438:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043243904:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828436.513490:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043244032:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828436.513555:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828436.513592:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828436/real 1787828436] req@ffff9181c35b1180 x1874663043244032/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828461 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828436.513630:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1180 x1874663043244032/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828461 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828436.513667:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828436.513675:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828436.513837:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043244032:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828436.513864:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043244160:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828436.513903:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828436.513920:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828436/real 1787828436] req@ffff9181c35b3100 x1874663043244160/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828461 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828436.513936:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3100 x1874663043244160/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828461 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828436.513947:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828436.513951:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828436.513955:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043244160:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828436.576876:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828437.600915:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828438.624993:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828439.648975:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828440.673051:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828441.632885:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828441.632897:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828441.632903:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828441.632915:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10508 00000100:00080000:1.0:1787828441.632942:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828441.632952:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787828441.633003:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1500 x1874663043244288/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828441.633044:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828441.633049:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828441.633051:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828441.633056:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10508 00000100:00080000:1.0:1787828441.633060:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828441.633065:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828441.633076:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828441.633091:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1c00 x1874663043244416/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828441.633106:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828441.633110:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828441.633113:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828441.633117:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10508 00000100:00080000:1.0:1787828441.633120:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828441.633125:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828441.633130:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828441.633145:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043244288:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787828441.633146:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0380 x1874663043244544/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828441.633266:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828441.633314:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828441/real 1787828441] req@ffff9181c35b1500 x1874663043244288/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828466 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828441.633337:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1500 x1874663043244288/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828466 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828441.633350:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828441.633356:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828441.633376:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043244288:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828441.633386:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043244416:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828441.633418:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828441.633438:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828441/real 1787828441] req@ffff9181c35b1c00 x1874663043244416/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828466 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828441.633460:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1c00 x1874663043244416/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828466 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828441.633471:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828441.633475:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828441.633478:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043244416:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828441.633509:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043244544:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828441.633529:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828441.633541:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828441/real 1787828441] req@ffff9181c35b0380 x1874663043244544/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828466 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828441.633553:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0380 x1874663043244544/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828466 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828441.633563:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828441.633566:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828441.633571:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043244544:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828441.697042:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828442.721006:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828443.744941:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828444.768881:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828445.792986:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828446.752888:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828446.752896:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828446.752899:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828446.752925:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10513 10000000:01000000:2.0:1787828446.752927:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-cliir' failed (-108). 10000000:00000400:2.0:1787828446.752934:0:159735:0:(mgc_request.c:1900:mgc_process_log()) MGC192.168.203.147@tcp: IR log lustre-cliir failed, not fatal: rc = -108 00000100:00080000:1.0:1787828446.752936:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 10000000:01000000:2.0:1787828446.752954:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 00000100:00080000:1.0:1787828446.752958:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 10000000:01000000:2.0:1787828446.752960:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff9181c5620000 from 52 10000000:01000000:2.0:1787828446.752982:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00000100:00100000:1.0:1787828446.753011:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0380 x1874663043244800/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:2.0:1787828446.753020:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1842a00/0xbef42c0221d6f775 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787828446.753030:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181d1842a00/0xbef42c0221d6f775 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787828446.753034:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828446.753038:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00010000:00010000:2.0:1787828446.753040:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181d1842a00/0xbef42c0221d6f775 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787828446.753041:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828446.753049:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10513 00000100:00080000:1.0:1787828446.753053:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828446.753058:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787828446.753062:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5c94c00 going to sleep for 0 seconds 00000008:00100000:1.0:1787828446.753067:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828446.753072:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043244672:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00100000:3.0:1787828446.753078:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043244800:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787828446.753080:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1c00 x1874663043244928/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:2.0:1787828446.753085:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181d1842a00/0xbef42c0221d6f775 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787828446.753126:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181d1842a00/0xbef42c0221d6f775 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787828446.753127:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828446.753138:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828446.753143:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00010000:00010000:2.0:1787828446.753145:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1842a00/0xbef42c0221d6f775 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787828446.753150:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10513 00000100:00080000:1.0:1787828446.753160:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00010000:00010000:2.0:1787828446.753162:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181d1842a00/0xbef42c0221d6f775 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000800:00000100:3.0:1787828446.753165:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1787828446.753186:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828446.753196:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:2.0:1787828446.753202:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181d1842a00/0xbef42c0221d6f775 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787828446.753229:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1500 x1874663043245056/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:2.0:1787828446.753236:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181d1842a00 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:2.0:1787828446.753241:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181d1842a00/0xbef42c0221d6f775 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787828446.753247:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828446/real 1787828446] req@ffff9181c35b0380 x1874663043244800/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828471 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:2.0:1787828446.753250:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181d1842a00/0xbef42c0221d6f775 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1787828446.753267:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181d1842a00/0xbef42c0221d6f775 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787828446.753272:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00000100:00100000:3.0:1787828446.753275:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0380 x1874663043244800/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828471 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:2.0:1787828446.753276:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787828446.753283:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181d1842a00/0xbef42c0221d6f775 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787828446.753286:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00010000:00010000:2.0:1787828446.753292:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181d1842a00/0xbef42c0221d6f775 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787828446.753293:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828446.753300:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043244800:192.168.203.147@tcp:250:kworker.0 10000000:01000000:2.0:1787828446.753317:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:2.0:1787828446.753320:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00100000:3.0:1787828446.753325:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043244928:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828446.753345:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828446.753357:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828446/real 1787828446] req@ffff9181c35b1c00 x1874663043244928/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828471 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828446.753358:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828446.753364:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828446.753368:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828446.753377:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10518 00000100:00100000:3.0:1787828446.753381:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1c00 x1874663043244928/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828471 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828446.753382:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828446.753390:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00080000:3.0:1787828446.753393:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828446.753398:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828446.753401:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043244928:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787828446.753405:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3100 x1874663043245184/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828446.753408:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043245056:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828446.753467:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828446.753490:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828446/real 1787828446] req@ffff9181c35b1500 x1874663043245056/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828471 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828446.753510:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1500 x1874663043245056/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828471 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828446.753518:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828446.753521:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828446.753524:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043245056:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828446.753537:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043245184:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1787828446.753553:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828446.753561:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828446/real 1787828446] req@ffff9181c35b3100 x1874663043245184/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828471 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828446.753573:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3100 x1874663043245184/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828471 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828446.753581:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828446.753583:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828446.753600:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043245184:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787828446.816935:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828447.840933:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828448.865015:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828449.888935:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828450.912928:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828451.872882:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828451.872897:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828451.872902:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828451.872934:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10518 00000100:00080000:1.0:1787828451.872948:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828451.872958:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787828451.873009:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1180 x1874663043245312/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828451.873044:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828451.873065:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828451.873067:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828451.873072:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10518 00000100:00080000:1.0:1787828451.873076:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828451.873080:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828451.873093:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828451.873107:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0000 x1874663043245440/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828451.873118:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828451.873123:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828451.873126:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828451.873131:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10518 00000100:00080000:1.0:1787828451.873134:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787828451.873159:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043245312:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787828451.873180:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828451.873190:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828451.873204:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0e00 x1874663043245568/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828451.873278:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828451.873326:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828451/real 1787828451] req@ffff9181c35b1180 x1874663043245312/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828476 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828451.873347:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1180 x1874663043245312/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828476 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828451.873366:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828451.873370:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828451.873393:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043245312:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828451.873402:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043245440:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828451.873425:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828451.873440:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828451/real 1787828451] req@ffff9181c35b0000 x1874663043245440/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828476 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828451.873457:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0000 x1874663043245440/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828476 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828451.873467:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828451.873472:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828451.873485:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043245440:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828451.873516:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043245568:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828451.873533:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828451.873542:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828451/real 1787828451] req@ffff9181c35b0e00 x1874663043245568/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828476 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828451.873553:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0e00 x1874663043245568/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828476 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828451.873561:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828451.873565:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828451.873567:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043245568:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828451.937016:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828452.960951:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828453.985012:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828455.009045:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828456.032990:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828456.992875:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828456.992888:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828456.992892:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828456.992923:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10523 00000100:00080000:1.0:1787828456.992937:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828456.992947:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787828456.993025:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3480 x1874663043245696/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828456.993058:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828456.993062:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828456.993064:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828456.993069:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10523 00000100:00080000:1.0:1787828456.993073:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828456.993077:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828456.993087:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828456.993120:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2680 x1874663043245824/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828456.993134:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828456.993137:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828456.993150:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828456.993155:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10523 00000100:00080000:1.0:1787828456.993159:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828456.993164:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828456.993170:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828456.993191:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2a00 x1874663043245952/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828456.993197:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043245696:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1787828456.993304:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828456.993360:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828456/real 1787828456] req@ffff9181c35b3480 x1874663043245696/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828481 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828456.993383:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3480 x1874663043245696/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828481 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828456.993404:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828456.993410:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828456.993433:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043245696:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828456.993445:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043245824:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828456.993470:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828456.993482:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828456/real 1787828456] req@ffff9181c35b2680 x1874663043245824/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828481 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828456.993492:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2680 x1874663043245824/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828481 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828456.993500:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828456.993503:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828456.993515:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043245824:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828456.993566:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043245952:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828456.993587:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828456.993599:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828456/real 1787828456] req@ffff9181c35b2a00 x1874663043245952/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828481 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828456.993611:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2a00 x1874663043245952/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828481 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828456.993619:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828456.993622:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828456.993625:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043245952:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828457.057026:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828458.080906:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828459.105023:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828460.128890:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828461.152927:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828462.112926:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828462.112937:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828462.112941:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828462.112951:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10528 00000100:00080000:0.0:1787828462.112962:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828462.112970:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787828462.113021:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4700 x1874663043246080/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828462.113060:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828462.113063:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828462.113078:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828462.113082:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10528 00000100:00080000:0.0:1787828462.113086:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828462.113090:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828462.113099:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828462.113113:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5f80 x1874663043246208/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828462.113132:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043246080:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787828462.113136:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828462.113141:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828462.113143:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828462.113148:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10528 00000100:00080000:0.0:1787828462.113152:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828462.113156:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828462.113161:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828462.113173:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6d80 x1874663043246336/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828462.113193:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828462.113224:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828462/real 1787828462] req@ffff9181c55a4700 x1874663043246080/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828487 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828462.113237:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4700 x1874663043246080/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828487 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828462.113245:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828462.113248:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828462.113261:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043246080:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828462.113279:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043246208:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828462.113298:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828462.113307:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828462/real 1787828462] req@ffff9181c55a5f80 x1874663043246208/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828487 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828462.113315:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5f80 x1874663043246208/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828487 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828462.113321:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828462.113323:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828462.113325:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043246208:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828462.113329:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043246336:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828462.113339:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828462.113344:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828462/real 1787828462] req@ffff9181c55a6d80 x1874663043246336/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828487 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828462.113351:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6d80 x1874663043246336/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828487 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828462.113356:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828462.113358:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828462.113360:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043246336:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787828462.176931:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828463.200884:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828464.224867:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828465.248922:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828466.272921:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828467.232883:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828467.232893:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828467.232897:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828467.232907:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10533 00000100:00080000:2.0:1787828467.232919:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828467.232926:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 10000000:01000000:0.0:1787828467.232949:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-client' failed (-108). 10000000:00020000:0.0:1787828467.232956:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:0.0:1787828467.232962:0:159735:0:(mgc_request.c:611:do_requeue()) updating log params 00000100:00100000:2.0:1787828467.232963:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d180 x1874663043246464/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 10000000:01000000:0.0:1787828467.232965:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff9181c5620000 from 7 10000000:01000000:0.0:1787828467.232972:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00000100:00080000:2.0:1787828467.232998:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828467.233001:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828467.233004:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828467.233007:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10533 00000100:00080000:2.0:1787828467.233010:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00010000:00010000:0.0:1787828467.233012:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0513200/0xbef42c0221d6f77c lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787828467.233017:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:0.0:1787828467.233047:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e0513200/0xbef42c0221d6f77c lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000008:00100000:2.0:1787828467.233050:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:0.0:1787828467.233066:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e0513200/0xbef42c0221d6f77c lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787828467.233070:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043246464:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828467.233073:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e300 x1874663043246720/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828467.233106:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1787828467.233107:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d01a5300 going to sleep for 0 seconds 00000100:00080000:2.0:1787828467.233114:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828467.233117:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1787828467.233121:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043246592:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00080000:2.0:1787828467.233126:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10533 00010000:00010000:0.0:1787828467.233140:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e0513200/0xbef42c0221d6f77c lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787828467.233143:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828467.233153:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:0.0:1787828467.233154:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e0513200/0xbef42c0221d6f77c lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000008:00100000:2.0:1787828467.233162:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1787828467.233162:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00010000:00010000:0.0:1787828467.233171:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0513200/0xbef42c0221d6f77c lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828467.233180:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e0513200/0xbef42c0221d6f77c lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787828467.233182:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c700 x1874663043246848/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:0.0:1787828467.233189:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e0513200/0xbef42c0221d6f77c lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828467.233213:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0513200 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:0.0:1787828467.233220:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e0513200/0xbef42c0221d6f77c lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787828467.233226:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828467/real 1787828467] req@ffff9181e007d180 x1874663043246464/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828492 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:0.0:1787828467.233230:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e0513200/0xbef42c0221d6f77c lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1787828467.233253:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e0513200/0xbef42c0221d6f77c lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787828467.233260:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:0.0:1787828467.233264:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00000100:00100000:3.0:1787828467.233269:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d180 x1874663043246464/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828492 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:0.0:1787828467.233283:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e0513200/0xbef42c0221d6f77c lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787828467.233291:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00010000:00010000:0.0:1787828467.233292:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e0513200/0xbef42c0221d6f77c lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787828467.233300:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828467.233314:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043246464:192.168.203.147@tcp:250:kworker.0 10000000:01000000:0.0:1787828467.233332:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:0.0:1787828467.233346:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00100000:3.0:1787828467.233359:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043246720:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828467.233388:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1787828467.233395:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828467.233401:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828467.233403:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787828467.233406:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828467/real 1787828467] req@ffff9181e007e300 x1874663043246720/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828492 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828467.233412:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10538 00000100:00080000:2.0:1787828467.233420:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828467.233427:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828467.233431:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e300 x1874663043246720/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828492 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828467.233441:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828467.233448:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828467.233449:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ea00 x1874663043246976/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828467.233453:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043246720:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828467.233470:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043246848:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828467.233494:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828467.233503:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828467/real 1787828467] req@ffff9181e007c700 x1874663043246848/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828492 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828467.233513:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c700 x1874663043246848/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828492 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828467.233519:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828467.233522:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828467.233524:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043246848:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828467.233536:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043246976:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1787828467.233571:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828467.233580:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828467/real 1787828467] req@ffff9181e007ea00 x1874663043246976/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828492 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828467.233592:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ea00 x1874663043246976/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828492 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828467.233599:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828467.233615:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828467.233629:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043246976:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1787828467.296893:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828468.320875:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828469.344900:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828470.368991:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828471.392941:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828472.352912:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828472.352925:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828472.352931:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828472.352943:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10538 00000100:00080000:2.0:1787828472.352957:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828472.352966:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787828472.353043:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f800 x1874663043247104/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828472.353091:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828472.353096:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828472.353099:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828472.353105:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10538 00000100:00080000:2.0:1787828472.353111:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828472.353117:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828472.353129:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828472.353155:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f100 x1874663043247232/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828472.353180:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828472.353191:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828472.353201:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787828472.353201:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043247104:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787828472.353230:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10538 00000100:00080000:2.0:1787828472.353241:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828472.353253:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828472.353262:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828472.353284:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f480 x1874663043247360/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828472.353376:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043247232:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828472.353420:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043247360:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787828472.416887:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828473.440916:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828474.465105:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1787828475.425172:0:2376:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:0.0:1787828475.425236:0:2376:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1787828475.425273:0:2376:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1787828475.425321:0:2376:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1787828475.425329:0:2376:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:3.0:1787828475.425384:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828472/real 1787828475] req@ffff9181e007f100 x1874663043247232/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828497 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828475.425416:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f100 x1874663043247232/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828497 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828475.425435:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828475.425443:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828475.425449:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043247232:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828475.425460:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828472/real 1787828475] req@ffff9181e007f480 x1874663043247360/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828497 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828475.425501:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f480 x1874663043247360/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828497 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828475.425514:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828475.425519:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828475.425524:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043247360:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828475.425533:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828472/real 1787828475] req@ffff9181e007f800 x1874663043247104/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828497 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828475.425551:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f800 x1874663043247104/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828497 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828475.425561:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828475.425566:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828475.425592:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043247104:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787828475.488944:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828476.512921:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828477.472848:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828477.472861:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828477.472867:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828477.472881:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10544 00000100:00080000:2.0:1787828477.472895:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828477.472904:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787828477.472948:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ce00 x1874663043247488/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828477.473016:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828477.473022:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828477.473025:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828477.473032:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10544 00000100:00080000:2.0:1787828477.473038:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828477.473044:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828477.473056:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828477.473075:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d880 x1874663043247616/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828477.473092:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828477.473096:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828477.473099:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787828477.473109:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043247488:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787828477.473112:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10544 00000100:00080000:2.0:1787828477.473125:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828477.473131:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828477.473137:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828477.473155:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e680 x1874663043247744/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828477.473230:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828477.473290:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828477/real 1787828477] req@ffff9181e007ce00 x1874663043247488/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828502 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828477.473324:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ce00 x1874663043247488/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828502 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828477.473344:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828477.473352:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828477.473382:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043247488:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828477.473423:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043247616:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828477.473490:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828477.473519:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828477/real 1787828477] req@ffff9181e007d880 x1874663043247616/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828502 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828477.473545:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d880 x1874663043247616/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828502 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828477.473561:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828477.473568:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828477.473575:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043247616:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828477.473586:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043247744:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828477.473623:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828477.473644:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828477/real 1787828477] req@ffff9181e007e680 x1874663043247744/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828502 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828477.473666:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e680 x1874663043247744/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828502 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828477.473682:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828477.473689:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828477.473695:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043247744:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828477.537031:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828478.561047:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828479.584986:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828480.609005:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828481.633028:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828482.592832:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828482.592841:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828482.592845:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828482.592853:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10549 00000100:00080000:2.0:1787828482.592862:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828482.592868:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787828482.592902:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007fb80 x1874663043247872/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828482.592933:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828482.592936:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828482.592937:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828482.592941:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10549 00000100:00080000:2.0:1787828482.592943:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828482.592946:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828482.592953:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828482.592962:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ca80 x1874663043248000/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828482.592970:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828482.592972:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828482.592973:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828482.592976:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10549 00000100:00080000:2.0:1787828482.592978:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828482.592981:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828482.592985:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828482.592988:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043247872:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828482.592994:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007dc00 x1874663043248128/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828482.593048:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828482.593079:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828482/real 1787828482] req@ffff9181e007fb80 x1874663043247872/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828507 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828482.593093:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007fb80 x1874663043247872/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828507 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828482.593101:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828482.593104:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828482.593118:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043247872:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828482.593126:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043248000:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828482.593149:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828482.593162:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828482/real 1787828482] req@ffff9181e007ca80 x1874663043248000/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828507 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828482.593175:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ca80 x1874663043248000/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828507 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828482.593184:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828482.593187:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828482.593191:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043248000:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828482.593214:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043248128:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828482.593231:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828482.593239:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828482/real 1787828482] req@ffff9181e007dc00 x1874663043248128/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828507 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828482.593247:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007dc00 x1874663043248128/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828507 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828482.593256:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828482.593259:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828482.593261:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043248128:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787828482.656916:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828483.680882:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828484.704949:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828485.728993:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828486.752854:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828487.712958:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 10000000:01000000:0.0:1787828487.712969:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'params' failed (-108). 00000100:00080000:1.0:1787828487.712969:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828487.712976:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 10000000:00020000:0.0:1787828487.712978:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:0.0:1787828487.712987:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 00000100:00080000:1.0:1787828487.712990:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10554 10000000:01000000:0.0:1787828487.712992:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff9181c87b19b8 from 1 10000000:01000000:0.0:1787828487.712999:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00000100:00080000:1.0:1787828487.713005:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828487.713016:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787828487.713048:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0a80 x1874663043248256/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:0.0:1787828487.713064:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512400/0xbef42c0221d6f783 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787828487.713078:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828487.713091:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828487.713097:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00010000:00010000:0.0:1787828487.713100:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e0512400/0xbef42c0221d6f783 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787828487.713106:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10554 00000100:00080000:1.0:1787828487.713116:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828487.713126:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:0.0:1787828487.713128:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e0512400/0xbef42c0221d6f783 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000008:00100000:1.0:1787828487.713138:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828487.713142:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043248256:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787828487.713162:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2d80 x1874663043248512/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787828487.713167:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d01a5300 going to sleep for 0 seconds 00000100:00100000:0.0:1787828487.713181:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043248384:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00080000:1.0:1787828487.713185:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828487.713192:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828487.713195:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00010000:00010000:0.0:1787828487.713199:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e0512400/0xbef42c0221d6f783 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787828487.713203:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10554 00000100:00080000:1.0:1787828487.713211:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00010000:00010000:0.0:1787828487.713218:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e0512400/0xbef42c0221d6f783 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000800:00000100:3.0:1787828487.713220:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1787828487.713222:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828487.713234:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:0.0:1787828487.713236:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512400/0xbef42c0221d6f783 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828487.713249:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e0512400/0xbef42c0221d6f783 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787828487.713257:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1f80 x1874663043248640/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:0.0:1787828487.713260:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e0512400/0xbef42c0221d6f783 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787828487.713266:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828487/real 1787828487] req@ffff9181c35b0a80 x1874663043248256/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828512 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:0.0:1787828487.713289:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0512400 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:0.0:1787828487.713297:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512400/0xbef42c0221d6f783 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787828487.713307:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0a80 x1874663043248256/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828512 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:0.0:1787828487.713310:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e0512400/0xbef42c0221d6f783 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1787828487.713341:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512400/0xbef42c0221d6f783 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787828487.713344:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828487.713356:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 10000000:01000000:0.0:1787828487.713362:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00000100:00100000:3.0:1787828487.713365:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043248256:192.168.203.147@tcp:250:kworker.0 00010000:00010000:0.0:1787828487.713371:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787828487.713382:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e0512400/0xbef42c0221d6f783 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828487.713390:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e0512400/0xbef42c0221d6f783 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787828487.713394:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043248512:192.168.203.147@tcp:38:kworker.0 10000000:01000000:0.0:1787828487.713414:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:0.0:1787828487.713419:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000800:00000100:3.0:1787828487.713443:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828487.713466:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828487/real 1787828487] req@ffff9181c35b2d80 x1874663043248512/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828512 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828487.713478:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828487.713484:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00100000:3.0:1787828487.713486:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2d80 x1874663043248512/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828512 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828487.713487:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828487.713493:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10559 00000100:00080000:1.0:1787828487.713499:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828487.713499:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828487.713504:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00080000:3.0:1787828487.713504:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828487.713509:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043248512:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828487.713517:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043248640:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787828487.713533:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3800 x1874663043248768/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828487.713544:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828487.713561:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828487/real 1787828487] req@ffff9181c35b1f80 x1874663043248640/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828512 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828487.713579:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1f80 x1874663043248640/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828512 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828487.713591:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828487.713596:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828487.713601:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043248640:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828487.713621:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043248768:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1787828487.713642:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828487.713652:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828487/real 1787828487] req@ffff9181c35b3800 x1874663043248768/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828512 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828487.713663:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3800 x1874663043248768/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828512 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828487.713680:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828487.713684:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828487.713696:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043248768:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787828487.777003:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828488.800865:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828489.824915:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828490.849037:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828491.872958:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828492.832933:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828492.832944:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828492.832947:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828492.832957:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10559 00000100:00080000:3.0:1787828492.832968:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828492.832977:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828492.833015:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043248896/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828492.833040:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828492.833044:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828492.833046:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828492.833051:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10559 00000100:00080000:3.0:1787828492.833055:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828492.833061:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828492.833072:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828492.833087:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043249024/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828492.833092:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043248896:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787828492.833100:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828492.833104:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828492.833105:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828492.833108:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10559 00000100:00080000:3.0:1787828492.833111:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828492.833114:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828492.833118:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828492.833126:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043249152/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828492.833162:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828492.833193:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828492/real 1787828492] req@ffff9181c5637100 x1874663043248896/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828517 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828492.833206:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043248896/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828517 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828492.833213:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828492.833216:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828492.833229:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043248896:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828492.833249:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043249024:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828492.833268:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828492.833279:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828492/real 1787828492] req@ffff9181c5637480 x1874663043249024/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828517 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828492.833287:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043249024/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828517 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828492.833293:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828492.833295:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828492.833297:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043249024:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828492.833301:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043249152:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828492.833312:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828492.833318:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828492/real 1787828492] req@ffff9181c5636a00 x1874663043249152/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828517 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828492.833325:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043249152/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828517 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828492.833335:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828492.833337:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828492.833339:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043249152:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828492.896904:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828493.920908:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828494.944963:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828495.968949:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828496.992983:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828497.952939:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828497.952950:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828497.952954:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828497.952965:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10564 00000100:00080000:2.0:1787828497.952975:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828497.952982:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787828497.953023:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c380 x1874663043249280/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828497.953052:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828497.953056:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828497.953057:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828497.953061:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10564 00000100:00080000:2.0:1787828497.953064:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828497.953068:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828497.953077:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828497.953089:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c000 x1874663043249408/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828497.953098:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043249280:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787828497.953104:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828497.953109:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828497.953111:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828497.953116:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10564 00000100:00080000:2.0:1787828497.953118:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828497.953122:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828497.953127:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828497.953140:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007dc00 x1874663043249536/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828497.953173:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828497.953212:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828497/real 1787828497] req@ffff9181e007c380 x1874663043249280/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828522 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828497.953233:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c380 x1874663043249280/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828522 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828497.953248:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828497.953254:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828497.953274:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043249280:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828497.953304:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043249408:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828497.953337:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828497.953356:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828497/real 1787828497] req@ffff9181e007c000 x1874663043249408/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828522 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828497.953375:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c000 x1874663043249408/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828522 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828497.953387:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828497.953391:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828497.953396:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043249408:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828497.953410:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043249536:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828497.953441:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828497.953456:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828497/real 1787828497] req@ffff9181e007dc00 x1874663043249536/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828522 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828497.953467:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007dc00 x1874663043249536/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828522 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828497.953474:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828497.953477:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828497.953480:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043249536:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828498.016988:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828499.040970:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828500.064975:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828501.089007:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828502.112886:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828503.072892:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828503.072902:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828503.072907:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828503.072918:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10569 00000100:00080000:3.0:1787828503.072929:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828503.072936:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828503.072973:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043249664/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828503.073003:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828503.073007:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828503.073008:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828503.073012:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10569 00000100:00080000:3.0:1787828503.073017:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828503.073022:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828503.073031:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828503.073043:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634e00 x1874663043249792/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828503.073051:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043249664:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787828503.073059:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828503.073063:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828503.073065:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828503.073069:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10569 00000100:00080000:3.0:1787828503.073072:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828503.073076:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828503.073081:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828503.073092:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043249920/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828503.073125:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828503.073178:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828503/real 1787828503] req@ffff9181c5634380 x1874663043249664/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828528 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828503.073194:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043249664/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828528 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828503.073204:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828503.073208:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828503.073222:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043249664:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828503.073244:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043249792:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828503.073268:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828503.073283:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828503/real 1787828503] req@ffff9181c5634e00 x1874663043249792/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828528 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828503.073296:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634e00 x1874663043249792/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828528 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828503.073305:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828503.073308:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828503.073312:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043249792:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828503.073317:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043249920:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828503.073349:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828503.073357:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828503/real 1787828503] req@ffff9181c5635500 x1874663043249920/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828528 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828503.073367:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043249920/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828528 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828503.073374:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828503.073377:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828503.073379:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043249920:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828503.136905:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828504.160909:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828505.184923:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828506.208913:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828507.232897:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828508.192909:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828508.192923:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828508.192928:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828508.192942:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10574 10000000:01000000:0.0:1787828508.192952:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-sptlrpc' failed (-108). 00000100:00080000:3.0:1787828508.192958:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 10000000:00020000:0.0:1787828508.192961:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00080000:3.0:1787828508.192974:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828508.193029:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635f80 x1874663043250048/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828508.193076:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828508.193082:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828508.193085:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828508.193092:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10574 00000100:00080000:3.0:1787828508.193098:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828508.193106:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828508.193119:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828508.193141:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043250048:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828508.193147:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043250176/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828508.193175:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828508.193181:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828508.193184:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828508.193190:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10574 00000100:00080000:3.0:1787828508.193199:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828508.193209:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828508.193219:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828508.193244:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634e00 x1874663043250304/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828508.193262:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828508.193318:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828508/real 1787828508] req@ffff9181c5635f80 x1874663043250048/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828533 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828508.193359:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635f80 x1874663043250048/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828533 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828508.193375:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828508.193381:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828508.193387:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043250048:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828508.193419:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043250176:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828508.193465:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828508.193510:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828508/real 1787828508] req@ffff9181c5635500 x1874663043250176/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828533 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828508.193539:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043250176/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828533 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828508.193558:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828508.193567:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828508.193574:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043250176:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828508.193589:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043250304:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828508.193636:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828508.193656:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828508/real 1787828508] req@ffff9181c5634e00 x1874663043250304/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828533 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828508.193675:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634e00 x1874663043250304/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828533 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828508.193687:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828508.193693:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828508.193698:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043250304:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828508.256968:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828509.280885:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:0.0:1787828509.856894:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:0.0:1787828509.856904:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff9181c5620000 from 341 10000000:01000000:0.0:1787828509.856911:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:0.0:1787828509.856977:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512400/0xbef42c0221d6f78a lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828509.856994:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e0512400/0xbef42c0221d6f78a lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828509.857011:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e0512400/0xbef42c0221d6f78a lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787828509.857046:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d01a5180 going to sleep for 0 seconds 00000100:00100000:0.0:1787828509.857066:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043250432:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:0.0:1787828509.857121:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e0512400/0xbef42c0221d6f78a lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828509.857133:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e0512400/0xbef42c0221d6f78a lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828509.857142:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512400/0xbef42c0221d6f78a lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828509.857149:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e0512400/0xbef42c0221d6f78a lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828509.857155:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e0512400/0xbef42c0221d6f78a lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828509.857161:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0512400 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:0.0:1787828509.857164:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512400/0xbef42c0221d6f78a lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828509.857202:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e0512400/0xbef42c0221d6f78a lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1787828509.857218:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512400/0xbef42c0221d6f78a lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787828509.857231:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:0.0:1787828509.857237:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787828509.857248:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e0512400/0xbef42c0221d6f78a lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828509.857265:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e0512400/0xbef42c0221d6f78a lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787828509.857330:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:0.0:1787828509.857334:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:3.0:1787828509.857399:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828509.857408:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828509.857412:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828509.857422:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10579 00000100:00080000:3.0:1787828509.857434:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828509.857441:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828509.857491:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043250560/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828509.857518:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828509.857521:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828509.857522:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828509.857526:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10579 00000100:00080000:3.0:1787828509.857528:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828509.857531:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828509.857541:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828509.857550:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043250688/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828509.857558:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828509.857562:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828509.857563:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828509.857577:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10579 00000100:00080000:3.0:1787828509.857580:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828509.857582:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828509.857587:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828509.857594:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043250816/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828509.857614:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043250560:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787828509.857727:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828509.857961:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828509/real 1787828509] req@ffff9181c5634380 x1874663043250560/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828534 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828509.857985:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043250560/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828534 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828509.858009:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828509.858015:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828509.858037:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043250560:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828509.858048:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043250688:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828509.858087:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828509.858131:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828509/real 1787828509] req@ffff9181c5636a00 x1874663043250688/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828534 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828509.858143:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043250688/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828534 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828509.858150:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828509.858153:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828509.858156:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043250688:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828509.858210:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043250816:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828509.858234:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828509.858247:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828509/real 1787828509] req@ffff9181c5637480 x1874663043250816/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828534 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828509.858261:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043250816/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828534 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828509.858280:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828509.858284:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828509.858288:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043250816:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828510.304975:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828511.328972:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828512.353026:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828513.377046:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828514.401025:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828515.360864:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828515.360876:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828515.360881:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828515.360910:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10581 00000100:00080000:3.0:1787828515.360925:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828515.360935:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828515.361000:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043250944/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828515.361055:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828515.361062:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828515.361066:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828515.361077:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10581 00000100:00080000:3.0:1787828515.361085:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828515.361094:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787828515.361105:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043250944:192.168.203.147@tcp:250:kworker.0 00000008:00100000:3.0:1787828515.361109:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828515.361145:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043251072/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828515.361164:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828515.361169:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828515.361172:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828515.361180:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10581 00000100:00080000:3.0:1787828515.361186:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828515.361193:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:1.0:1787828515.361202:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:3.0:1787828515.361207:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828515.361232:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043251200/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828515.361281:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828515/real 1787828515] req@ffff9181c5637100 x1874663043250944/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828540 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828515.361317:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043250944/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828540 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828515.361330:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828515.361336:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787828515.361356:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043250944:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787828515.361389:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043251072:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787828515.361429:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787828515.361450:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828515/real 1787828515] req@ffff9181c5637800 x1874663043251072/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828540 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828515.361467:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043251072/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828540 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828515.361477:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828515.361481:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787828515.361485:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043251072:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787828515.361491:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043251200:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787828515.361509:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787828515.361519:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828515/real 1787828515] req@ffff9181c5636d80 x1874663043251200/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828540 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828515.361531:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043251200/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828540 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828515.361539:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828515.361542:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787828515.361546:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043251200:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828515.424971:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828516.448957:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828517.473057:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828518.496917:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828519.520997:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828520.480806:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828520.480817:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828520.480821:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828520.480833:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10587 00000100:00080000:3.0:1787828520.480844:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828520.480852:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828520.480896:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043251328/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828520.480919:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828520.480922:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828520.480924:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828520.480927:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10587 00000100:00080000:3.0:1787828520.480930:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828520.480934:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828520.480943:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828520.480952:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043251328:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828520.480955:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637b80 x1874663043251456/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828520.480966:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828520.480969:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828520.480970:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828520.480973:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10587 00000100:00080000:3.0:1787828520.480975:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828520.480977:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828520.480981:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828520.480988:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043251584/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828520.481029:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787828520.481067:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828520/real 1787828520] req@ffff9181c5635c00 x1874663043251328/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828545 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828520.481098:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043251328/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828545 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828520.481112:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828520.481115:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787828520.481127:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043251328:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787828520.481151:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043251456:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787828520.481171:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787828520.481185:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828520/real 1787828520] req@ffff9181c5637b80 x1874663043251456/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828545 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828520.481203:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637b80 x1874663043251456/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828545 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828520.481213:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828520.481218:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787828520.481224:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043251456:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787828520.481232:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043251584:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787828520.481251:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787828520.481259:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828520/real 1787828520] req@ffff9181c5634000 x1874663043251584/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828545 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828520.481270:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043251584/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828545 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828520.481279:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828520.481282:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787828520.481286:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043251584:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828520.544902:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828521.569059:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828522.592935:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828523.616977:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828524.640902:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828525.600827:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828525.600837:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828525.600841:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828525.600851:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10592 00000100:00080000:3.0:1787828525.600862:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828525.600871:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828525.600915:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634a80 x1874663043251712/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828525.600942:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828525.600946:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828525.600948:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828525.600952:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10592 00000100:00080000:3.0:1787828525.600955:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828525.600959:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828525.600969:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828525.600983:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043251840/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828525.600993:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828525.600997:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:1.0:1787828525.601001:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043251712:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787828525.601001:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828525.601010:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10592 00000100:00080000:3.0:1787828525.601013:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828525.601016:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828525.601020:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828525.601031:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637b80 x1874663043251968/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828525.601072:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787828525.601109:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828525/real 1787828525] req@ffff9181c5634a80 x1874663043251712/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828550 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828525.601124:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634a80 x1874663043251712/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828550 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828525.601134:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828525.601137:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787828525.601152:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043251712:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787828525.601173:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043251840:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787828525.601197:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787828525.601208:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828525/real 1787828525] req@ffff9181c5634000 x1874663043251840/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828550 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828525.601225:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043251840/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828550 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828525.601232:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828525.601235:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787828525.601238:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043251840:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787828525.601245:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043251968:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787828525.601262:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787828525.601269:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828525/real 1787828525] req@ffff9181c5637b80 x1874663043251968/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828550 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828525.601279:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637b80 x1874663043251968/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828550 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828525.601285:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828525.601287:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787828525.601290:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043251968:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828525.664970:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828526.688956:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828527.712965:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828528.737022:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828529.760995:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:0.0:1787828530.208979:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-cliir' failed (-108). 10000000:00000400:0.0:1787828530.208992:0:159735:0:(mgc_request.c:1900:mgc_process_log()) MGC192.168.203.147@tcp: IR log lustre-cliir failed, not fatal: rc = -108 10000000:01000000:0.0:1787828530.209004:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:0.0:1787828530.209007:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff9181c5620000 from 52 10000000:01000000:0.0:1787828530.209011:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:0.0:1787828530.209097:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512400/0xbef42c0221d6f791 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828530.209116:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e0512400/0xbef42c0221d6f791 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828530.209134:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e0512400/0xbef42c0221d6f791 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787828530.209186:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d01a5180 going to sleep for 0 seconds 00000100:00100000:0.0:1787828530.209206:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043252096:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:0.0:1787828530.209225:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e0512400/0xbef42c0221d6f791 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828530.209235:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e0512400/0xbef42c0221d6f791 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828530.209243:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512400/0xbef42c0221d6f791 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828530.209252:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e0512400/0xbef42c0221d6f791 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828530.209262:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e0512400/0xbef42c0221d6f791 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828530.209269:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0512400 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:0.0:1787828530.209274:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512400/0xbef42c0221d6f791 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828530.209283:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e0512400/0xbef42c0221d6f791 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1787828530.209290:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512400/0xbef42c0221d6f791 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787828530.209297:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:0.0:1787828530.209301:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787828530.209309:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e0512400/0xbef42c0221d6f791 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828530.209317:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e0512400/0xbef42c0221d6f791 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787828530.209373:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:0.0:1787828530.209376:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:3.0:1787828530.209501:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828530.209526:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828530.209530:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828530.209539:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10597 00000100:00080000:3.0:1787828530.209552:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828530.209559:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828530.209589:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043252224/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828530.209664:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828530.209669:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828530.209671:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828530.209676:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10597 00000100:00080000:3.0:1787828530.209680:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828530.209686:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828530.209715:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828530.209740:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043252352/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828530.209804:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043252224:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787828530.209820:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828530.209839:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828530.209844:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828530.209852:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10597 00000100:00080000:3.0:1787828530.209858:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828530.209864:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828530.209871:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828530.209905:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043252480/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828530.209923:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787828530.209978:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828530/real 1787828530] req@ffff9181c5635c00 x1874663043252224/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828555 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828530.209997:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043252224/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828555 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828530.210010:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828530.210014:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787828530.210049:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043252224:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787828530.210056:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043252352:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787828530.210085:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787828530.210100:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828530/real 1787828530] req@ffff9181c5636d80 x1874663043252352/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828555 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828530.210146:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043252352/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828555 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828530.210155:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828530.210159:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787828530.210162:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043252352:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787828530.210185:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043252480:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787828530.210203:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787828530.210213:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828530/real 1787828530] req@ffff9181c5637800 x1874663043252480/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828555 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828530.210226:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043252480/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828555 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828530.210234:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828530.210237:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787828530.210240:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043252480:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828530.785022:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828531.808962:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828532.832895:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828533.856917:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828534.880886:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828535.328876:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828535.328884:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828535.328887:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828535.328896:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10601 00000100:00080000:0.0:1787828535.328905:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828535.328912:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787828535.328945:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7480 x1874663043252608/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828535.328968:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828535.328971:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828535.328972:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828535.328975:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10601 00000100:00080000:0.0:1787828535.328977:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828535.328980:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828535.328987:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828535.328995:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5880 x1874663043252736/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828535.329008:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828535.329010:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828535.329012:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828535.329015:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10601 00000100:00080000:0.0:1787828535.329017:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828535.329019:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828535.329022:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828535.329028:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043252608:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787828535.329032:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7100 x1874663043252864/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828535.329126:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043252736:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787828535.329169:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043252864:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787828535.904867:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828536.928997:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828537.952874:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1787828538.401109:0:2377:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:2.0:1787828538.401172:0:2377:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1787828538.401192:0:2377:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:2.0:1787828538.401246:0:2377:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:2.0:1787828538.401256:0:2377:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:1.0:1787828538.401506:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828535/real 1787828538] req@ffff9181c55a7100 x1874663043252864/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828560 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828538.401576:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7100 x1874663043252864/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828560 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828538.401598:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828538.401606:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787828538.401614:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043252864:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787828538.401628:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828535/real 1787828538] req@ffff9181c55a7480 x1874663043252608/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828560 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828538.401649:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7480 x1874663043252608/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828560 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828538.401680:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828538.401685:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787828538.401717:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043252608:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787828538.401733:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828535/real 1787828538] req@ffff9181c55a5880 x1874663043252736/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828560 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828538.401863:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5880 x1874663043252736/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828560 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828538.401900:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828538.401908:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787828538.401914:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043252736:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828538.976910:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828540.001057:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828540.448891:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828540.448906:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828540.448913:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828540.448929:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10607 00000100:00080000:0.0:1787828540.448945:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828540.448956:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787828540.449014:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7b80 x1874663043252992/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828540.449060:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828540.449065:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828540.449067:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828540.449072:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10607 00000100:00080000:0.0:1787828540.449076:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828540.449080:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828540.449093:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828540.449125:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6680 x1874663043253120/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828540.449140:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828540.449144:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828540.449147:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828540.449152:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10607 00000100:00080000:0.0:1787828540.449156:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828540.449162:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828540.449169:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828540.449186:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043252992:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787828540.449189:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5880 x1874663043253248/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828540.449347:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787828540.449417:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828540/real 1787828540] req@ffff9181c55a7b80 x1874663043252992/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828565 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828540.449453:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7b80 x1874663043252992/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828565 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828540.449474:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828540.449482:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787828540.449551:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043252992:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787828540.449568:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043253120:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787828540.449624:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787828540.449650:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828540/real 1787828540] req@ffff9181c55a6680 x1874663043253120/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828565 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828540.449688:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6680 x1874663043253120/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828565 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828540.449714:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828540.449721:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787828540.449728:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043253120:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787828540.449833:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043253248:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787828540.449875:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787828540.449899:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828540/real 1787828540] req@ffff9181c55a5880 x1874663043253248/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828565 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828540.449923:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5880 x1874663043253248/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828565 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828540.449939:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828540.449945:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787828540.449951:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043253248:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828541.025003:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828542.048882:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828543.073064:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828544.097011:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828545.120886:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828545.568878:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828545.568890:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828545.568896:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828545.568908:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10612 00000100:00080000:0.0:1787828545.568922:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828545.568931:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787828545.568996:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7480 x1874663043253376/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828545.569036:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828545.569043:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828545.569046:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828545.569053:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10612 00000100:00080000:0.0:1787828545.569060:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828545.569069:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828545.569082:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828545.569100:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043253376:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787828545.569112:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7100 x1874663043253504/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828545.569131:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828545.569135:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828545.569137:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828545.569142:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10612 00000100:00080000:0.0:1787828545.569147:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828545.569153:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828545.569161:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828545.569175:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6d80 x1874663043253632/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828545.569200:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787828545.569254:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828545/real 1787828545] req@ffff9181c55a7480 x1874663043253376/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828570 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828545.569276:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7480 x1874663043253376/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828570 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828545.569288:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828545.569293:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787828545.569326:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043253376:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787828545.569356:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043253504:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787828545.569384:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787828545.569398:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828545/real 1787828545] req@ffff9181c55a7100 x1874663043253504/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828570 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828545.569428:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7100 x1874663043253504/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828570 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828545.569438:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828545.569441:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787828545.569445:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043253504:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787828545.569451:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043253632:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787828545.569469:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787828545.569479:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828545/real 1787828545] req@ffff9181c55a6d80 x1874663043253632/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828570 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828545.569520:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6d80 x1874663043253632/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828570 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828545.569554:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828545.569557:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787828545.569561:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043253632:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828546.144967:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828547.168959:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828548.192896:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828549.216968:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828550.240911:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:0.0:1787828550.688996:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-client' failed (-108). 00000100:00080000:1.0:1787828550.688996:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828550.689019:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 10000000:00020000:0.0:1787828550.689023:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00080000:1.0:1787828550.689028:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 10000000:01000000:0.0:1787828550.689053:0:159735:0:(mgc_request.c:611:do_requeue()) updating log params 00000100:00080000:1.0:1787828550.689058:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10617 10000000:01000000:0.0:1787828550.689061:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff9181c5620000 from 7 10000000:01000000:0.0:1787828550.689069:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00000100:00080000:1.0:1787828550.689074:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828550.689092:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:0.0:1787828550.689136:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512600/0xbef42c0221d6f798 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787828550.689140:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0700 x1874663043253888/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:0.0:1787828550.689178:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e0512600/0xbef42c0221d6f798 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828550.689203:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e0512600/0xbef42c0221d6f798 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787828550.689220:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1787828550.689223:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d01a5180 going to sleep for 0 seconds 00000100:00080000:1.0:1787828550.689227:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828550.689230:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828550.689237:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10617 00000100:00100000:0.0:1787828550.689238:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043253760:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00080000:1.0:1787828550.689246:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828550.689256:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:0.0:1787828550.689258:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e0512600/0xbef42c0221d6f798 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828550.689267:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e0512600/0xbef42c0221d6f798 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000008:00100000:1.0:1787828550.689269:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:0.0:1787828550.689287:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512600/0xbef42c0221d6f798 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828550.689293:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e0512600/0xbef42c0221d6f798 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787828550.689295:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3b80 x1874663043254016/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828550.689349:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:0.0:1787828550.689351:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e0512600/0xbef42c0221d6f798 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787828550.689357:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043253888:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787828550.689369:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828550.689397:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00010000:00010000:0.0:1787828550.689399:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0512600 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:0.0:1787828550.689417:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512600/0xbef42c0221d6f798 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787828550.689423:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10617 00000100:00080000:1.0:1787828550.689439:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00010000:00010000:0.0:1787828550.689443:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e0512600/0xbef42c0221d6f798 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787828550.689487:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 10000000:00010000:0.0:1787828550.689491:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512600/0xbef42c0221d6f798 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000008:00100000:1.0:1787828550.689498:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 10000000:01000000:0.0:1787828550.689499:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:0.0:1787828550.689519:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787828550.689532:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e0512600/0xbef42c0221d6f798 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787828550.689537:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3800 x1874663043254144/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:0.0:1787828550.689593:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e0512600/0xbef42c0221d6f798 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000800:00000100:2.0:1787828550.689614:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 10000000:01000000:0.0:1787828550.689660:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:0.0:1787828550.689666:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:CONNECTING 00000100:00100000:2.0:1787828550.689727:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828550/real 1787828550] req@ffff9181c35b0700 x1874663043253888/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828575 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828550.689815:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0700 x1874663043253888/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828575 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828550.689830:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828550.689868:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828550.689890:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043253888:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787828550.689939:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1787828550.689940:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043254016:192.168.203.147@tcp:38:kworker.0 00000100:00080000:1.0:1787828550.689949:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828550.689954:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828550.689962:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10622 00000100:00080000:1.0:1787828550.689969:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828550.689976:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:2.0:1787828550.690003:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787828550.690010:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1f80 x1874663043254272/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828550.690038:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828550/real 1787828550] req@ffff9181c35b3b80 x1874663043254016/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828575 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828550.690081:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3b80 x1874663043254016/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828575 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828550.690091:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828550.690095:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828550.690099:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043254016:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828550.690106:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043254144:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828550.690142:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828550.690162:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828550/real 1787828550] req@ffff9181c35b3800 x1874663043254144/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828575 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828550.690202:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3800 x1874663043254144/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828575 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828550.690218:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828550.690236:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828550.690243:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043254144:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828550.690274:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043254272:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787828550.690325:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828550.690353:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828550/real 1787828550] req@ffff9181c35b1f80 x1874663043254272/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828575 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828550.690394:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1f80 x1874663043254272/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828575 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828550.690411:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828550.690416:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828550.690434:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043254272:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1787828551.265035:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828552.288871:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828553.312880:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828554.336898:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828555.360989:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828555.808900:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828555.808915:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828555.808921:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828555.808937:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10622 00000100:00080000:0.0:1787828555.808954:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828555.808963:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787828555.809008:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5f80 x1874663043254400/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828555.809047:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828555.809052:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828555.809054:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828555.809059:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10622 00000100:00080000:0.0:1787828555.809063:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828555.809068:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828555.809078:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828555.809091:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4700 x1874663043254528/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828555.809103:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828555.809107:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828555.809109:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828555.809114:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10622 00000100:00080000:0.0:1787828555.809117:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828555.809121:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828555.809126:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828555.809136:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5c00 x1874663043254656/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828555.809152:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043254400:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787828555.809240:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828555.809296:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828555/real 1787828555] req@ffff9181c55a5f80 x1874663043254400/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828580 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828555.809316:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5f80 x1874663043254400/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828580 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828555.809328:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828555.809333:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828555.809362:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043254400:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828555.809372:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043254528:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828555.809403:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828555.809458:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828555/real 1787828555] req@ffff9181c55a4700 x1874663043254528/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828580 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828555.809474:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4700 x1874663043254528/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828580 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828555.809484:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828555.809488:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828555.809492:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043254528:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828555.809532:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043254656:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828555.809556:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828555.809568:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828555/real 1787828555] req@ffff9181c55a5c00 x1874663043254656/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828580 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828555.809587:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5c00 x1874663043254656/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828580 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828555.809595:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828555.809599:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828555.809602:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043254656:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828556.384907:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828557.408919:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828558.432965:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828559.456908:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828560.481011:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828560.928838:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828560.928846:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828560.928849:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828560.928858:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10627 00000100:00080000:1.0:1787828560.928867:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828560.928873:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787828560.928904:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2d80 x1874663043254784/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828560.928926:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828560.928929:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828560.928930:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828560.928934:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10627 00000100:00080000:1.0:1787828560.928936:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828560.928939:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828560.928946:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828560.928955:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0a80 x1874663043254912/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828560.928964:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828560.928966:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828560.928969:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1787828560.928972:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043254784:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787828560.928974:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10627 00000100:00080000:1.0:1787828560.928980:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828560.928985:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828560.928988:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828560.928999:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2a00 x1874663043255040/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828560.929042:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828560.929078:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828560/real 1787828560] req@ffff9181c35b2d80 x1874663043254784/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828585 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828560.929092:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2d80 x1874663043254784/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828585 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828560.929100:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828560.929103:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828560.929114:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043254784:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828560.929147:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043254912:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828560.929172:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828560.929188:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828560/real 1787828560] req@ffff9181c35b0a80 x1874663043254912/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828585 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828560.929202:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0a80 x1874663043254912/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828585 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828560.929212:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828560.929215:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828560.929219:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043254912:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828560.929226:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043255040:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828560.929244:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828560.929252:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828560/real 1787828560] req@ffff9181c35b2a00 x1874663043255040/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828585 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828560.929260:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2a00 x1874663043255040/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828585 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828560.929266:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828560.929268:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828560.929270:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043255040:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828561.504927:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828562.528953:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828563.552907:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828564.576964:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828565.600945:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828566.048876:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828566.048890:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828566.048896:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828566.048911:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10632 00000100:00080000:1.0:1787828566.048926:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828566.048939:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787828566.048991:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2680 x1874663043255168/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828566.049031:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828566.049036:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828566.049039:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828566.049044:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10632 00000100:00080000:1.0:1787828566.049049:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828566.049055:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828566.049066:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828566.049084:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3480 x1874663043255296/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828566.049098:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1787828566.049101:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043255168:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787828566.049106:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828566.049112:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828566.049120:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10632 00000100:00080000:1.0:1787828566.049124:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828566.049128:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828566.049134:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828566.049150:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0e00 x1874663043255424/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828566.049188:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828566.049248:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828566/real 1787828566] req@ffff9181c35b2680 x1874663043255168/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828591 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828566.049283:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2680 x1874663043255168/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828591 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828566.049303:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828566.049308:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828566.049329:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043255168:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828566.049391:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043255296:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828566.049422:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828566.049436:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828566/real 1787828566] req@ffff9181c35b3480 x1874663043255296/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828591 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828566.049449:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3480 x1874663043255296/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828591 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828566.049457:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828566.049460:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828566.049463:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043255296:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828566.049470:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043255424:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828566.049485:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828566.049493:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828566/real 1787828566] req@ffff9181c35b0e00 x1874663043255424/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828591 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828566.049504:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0e00 x1874663043255424/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828591 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828566.049512:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828566.049515:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828566.049518:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043255424:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828566.624856:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828567.648897:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828568.673036:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828569.696889:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828570.720869:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828571.168903:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828571.168912:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828571.168916:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828571.168928:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10637 10000000:01000000:0.0:1787828571.168932:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'params' failed (-108). 10000000:00020000:0.0:1787828571.168939:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00080000:1.0:1787828571.168941:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 10000000:01000000:0.0:1787828571.168945:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:0.0:1787828571.168949:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff9181c87b19b8 from 1 00000100:00080000:1.0:1787828571.168954:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 10000000:01000000:0.0:1787828571.168956:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00000100:00100000:1.0:1787828571.169014:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0000 x1874663043255680/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:0.0:1787828571.169029:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f79f lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828571.169044:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f79f lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787828571.169050:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828571.169057:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00010000:00010000:0.0:1787828571.169059:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f79f lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787828571.169060:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828571.169074:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10637 00000100:00080000:1.0:1787828571.169081:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828571.169088:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787828571.169092:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d01a5780 going to sleep for 0 seconds 00000008:00100000:1.0:1787828571.169099:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828571.169108:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043255552:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00100000:2.0:1787828571.169123:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043255680:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787828571.169140:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1180 x1874663043255808/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:0.0:1787828571.169142:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f79f lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828571.169174:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f79f lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787828571.169178:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828571.169186:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00010000:00010000:0.0:1787828571.169187:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f79f lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787828571.169190:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828571.169202:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10637 00010000:00010000:0.0:1787828571.169205:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f79f lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787828571.169226:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00010000:00010000:0.0:1787828571.169239:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f79f lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1787828571.169240:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:2.0:1787828571.169243:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:1.0:1787828571.169254:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:0.0:1787828571.169256:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0513400 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:0.0:1787828571.169265:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f79f lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787828571.169278:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3100 x1874663043255936/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:0.0:1787828571.169286:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f79f lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787828571.169298:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828571/real 1787828571] req@ffff9181c35b0000 x1874663043255680/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828596 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 10000000:00010000:0.0:1787828571.169327:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f79f lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787828571.169344:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:0.0:1787828571.169375:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00000100:00100000:2.0:1787828571.169382:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0000 x1874663043255680/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828596 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:0.0:1787828571.169385:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f79f lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787828571.169408:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e0513400/0xbef42c0221d6f79f lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787828571.169417:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828571.169426:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 10000000:01000000:0.0:1787828571.169435:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00100000:2.0:1787828571.169435:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043255680:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787828571.169441:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00100000:2.0:1787828571.169475:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043255808:192.168.203.147@tcp:38:kworker.0 00000100:00080000:1.0:1787828571.169501:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:2.0:1787828571.169511:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1787828571.169512:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828571.169517:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828571.169527:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10642 00000100:00100000:2.0:1787828571.169533:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828571/real 1787828571] req@ffff9181c35b1180 x1874663043255808/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828596 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828571.169537:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828571.169551:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787828571.169561:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1180 x1874663043255808/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828596 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828571.169573:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:1.0:1787828571.169576:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1500 x1874663043256064/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828571.169580:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828571.169590:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043255808:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828571.169609:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043255936:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828571.169644:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828571.169661:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828571/real 1787828571] req@ffff9181c35b3100 x1874663043255936/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828596 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828571.169681:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3100 x1874663043255936/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828596 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828571.169691:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828571.169693:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828571.169697:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043255936:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828571.169712:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043256064:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787828571.169732:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828571.169741:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828571/real 1787828571] req@ffff9181c35b1500 x1874663043256064/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828596 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828571.169752:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1500 x1874663043256064/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828596 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828571.169798:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828571.169801:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828571.169820:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043256064:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787828571.744836:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828572.768933:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828573.792911:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828574.816924:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828575.841035:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828576.288865:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828576.288876:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828576.288881:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828576.288890:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10642 00000100:00080000:1.0:1787828576.288902:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828576.288910:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787828576.288946:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1c00 x1874663043256192/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828576.288971:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828576.288973:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828576.288975:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828576.288978:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10642 00000100:00080000:1.0:1787828576.288981:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828576.288984:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828576.288992:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828576.289000:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0380 x1874663043256320/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828576.289008:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828576.289011:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828576.289012:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828576.289015:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10642 00000100:00080000:1.0:1787828576.289016:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828576.289020:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828576.289024:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828576.289027:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043256192:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787828576.289035:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1500 x1874663043256448/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828576.289091:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828576.289122:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828576/real 1787828576] req@ffff9181c35b1c00 x1874663043256192/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828601 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828576.289135:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1c00 x1874663043256192/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828601 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828576.289148:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828576.289151:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828576.289162:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043256192:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828576.289168:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043256320:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828576.289186:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828576.289197:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828576/real 1787828576] req@ffff9181c35b0380 x1874663043256320/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828601 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828576.289206:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0380 x1874663043256320/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828601 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828576.289212:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828576.289214:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828576.289217:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043256320:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828576.289235:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043256448:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828576.289246:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828576.289252:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828576/real 1787828576] req@ffff9181c35b1500 x1874663043256448/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828601 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828576.289259:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1500 x1874663043256448/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828601 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828576.289265:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828576.289267:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828576.289269:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043256448:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828576.864957:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828577.889087:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828578.913022:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828579.936947:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828580.960932:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828581.408870:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828581.408879:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828581.408883:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828581.408892:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10648 00000100:00080000:2.0:1787828581.408902:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828581.408909:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787828581.408942:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ca80 x1874663043256576/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828581.408972:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828581.408975:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828581.408977:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828581.408980:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10648 00000100:00080000:2.0:1787828581.408983:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828581.408986:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828581.408996:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828581.409007:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007fb80 x1874663043256704/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828581.409014:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043256576:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787828581.409020:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828581.409024:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828581.409026:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828581.409030:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10648 00000100:00080000:2.0:1787828581.409033:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828581.409036:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828581.409042:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828581.409052:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e680 x1874663043256832/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828581.409084:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787828581.409117:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828581/real 1787828581] req@ffff9181e007ca80 x1874663043256576/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828606 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828581.409133:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ca80 x1874663043256576/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828606 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828581.409143:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828581.409146:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787828581.409160:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043256576:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787828581.409183:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043256704:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787828581.409206:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787828581.409221:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828581/real 1787828581] req@ffff9181e007fb80 x1874663043256704/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828606 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828581.409231:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007fb80 x1874663043256704/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828606 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828581.409255:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828581.409259:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787828581.409262:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043256704:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787828581.409267:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043256832:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787828581.409283:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787828581.409291:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828581/real 1787828581] req@ffff9181e007e680 x1874663043256832/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828606 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828581.409301:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e680 x1874663043256832/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828606 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828581.409307:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828581.409310:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787828581.409313:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043256832:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828581.984925:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828583.008937:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828584.032947:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828585.056849:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828586.080895:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828586.528821:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828586.528829:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828586.528832:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828586.528840:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10653 00000100:00080000:2.0:1787828586.528848:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828586.528855:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787828586.528885:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d880 x1874663043256960/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828586.528901:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828586.528904:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828586.528905:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828586.528908:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10653 00000100:00080000:2.0:1787828586.528911:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1787828586.528914:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043256960:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787828586.528917:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828586.528927:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828586.528938:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ce00 x1874663043257088/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828586.528953:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828586.528955:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828586.528957:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828586.528961:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10653 00000100:00080000:2.0:1787828586.528965:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828586.528967:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828586.528971:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828586.528978:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f800 x1874663043257216/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828586.528988:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787828586.529019:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828586/real 1787828586] req@ffff9181e007d880 x1874663043256960/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828611 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828586.529033:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d880 x1874663043256960/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828611 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828586.529042:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828586.529044:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787828586.529058:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043256960:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787828586.529077:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043257088:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787828586.529113:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787828586.529132:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828586/real 1787828586] req@ffff9181e007ce00 x1874663043257088/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828611 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828586.529149:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ce00 x1874663043257088/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828611 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828586.529163:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828586.529170:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787828586.529175:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043257088:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787828586.529184:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043257216:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787828586.529257:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787828586.529265:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828586/real 1787828586] req@ffff9181e007f800 x1874663043257216/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828611 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828586.529273:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f800 x1874663043257216/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828611 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828586.529279:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828586.529281:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787828586.529283:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043257216:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828587.104862:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828588.128871:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828589.152862:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828590.176870:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828591.200901:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828591.648840:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828591.648848:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828591.648851:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828591.648861:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10658 00000100:00080000:0.0:1787828591.648869:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828591.648875:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 10000000:01000000:2.0:1787828591.648896:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-sptlrpc' failed (-108). 10000000:00020000:2.0:1787828591.648901:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00100000:0.0:1787828591.648906:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4380 x1874663043257344/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828591.648928:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828591.648931:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828591.648933:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828591.648937:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10658 00000100:00080000:0.0:1787828591.648939:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828591.648942:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828591.648949:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828591.648957:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5180 x1874663043257472/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828591.648964:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828591.648966:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828591.648967:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828591.648969:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10658 00000100:00080000:0.0:1787828591.648971:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828591.648973:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828591.648976:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828591.648981:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5500 x1874663043257600/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828591.649007:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043257344:192.168.203.147@tcp:250:kworker.0 00000800:00000100:1.0:1787828591.649061:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787828591.649088:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828591/real 1787828591] req@ffff9181c55a4380 x1874663043257344/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828616 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828591.649102:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4380 x1874663043257344/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828616 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828591.649110:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828591.649113:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787828591.649116:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043257344:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787828591.649120:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043257472:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787828591.649134:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787828591.649142:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828591/real 1787828591] req@ffff9181c55a5180 x1874663043257472/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828616 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828591.649150:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5180 x1874663043257472/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828616 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828591.649155:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828591.649157:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787828591.649159:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043257472:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787828591.649179:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043257600:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787828591.649195:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787828591.649203:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828591/real 1787828591] req@ffff9181c55a5500 x1874663043257600/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828616 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828591.649211:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5500 x1874663043257600/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828616 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828591.649216:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828591.649218:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787828591.649221:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043257600:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828592.224935:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:2.0:1787828592.736923:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:2.0:1787828592.736934:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff9181c5620000 from 341 10000000:01000000:2.0:1787828592.736941:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:2.0:1787828592.737008:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1843200/0xbef42c0221d6f7a6 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787828592.737031:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181d1843200/0xbef42c0221d6f7a6 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787828592.737046:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181d1843200/0xbef42c0221d6f7a6 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787828592.737075:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c3cba9c0 going to sleep for 0 seconds 00000100:00100000:2.0:1787828592.737089:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043257728:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:2.0:1787828592.737108:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181d1843200/0xbef42c0221d6f7a6 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787828592.737117:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181d1843200/0xbef42c0221d6f7a6 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787828592.737130:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1843200/0xbef42c0221d6f7a6 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787828592.737139:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181d1843200/0xbef42c0221d6f7a6 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787828592.737148:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181d1843200/0xbef42c0221d6f7a6 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787828592.737173:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181d1843200 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:2.0:1787828592.737177:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181d1843200/0xbef42c0221d6f7a6 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787828592.737185:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181d1843200/0xbef42c0221d6f7a6 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1787828592.737190:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181d1843200/0xbef42c0221d6f7a6 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787828592.737195:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:2.0:1787828592.737197:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787828592.737203:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181d1843200/0xbef42c0221d6f7a6 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787828592.737209:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181d1843200/0xbef42c0221d6f7a6 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787828592.737231:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:2.0:1787828592.737234:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:0.0:1787828592.737293:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828592.737302:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828592.737305:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828592.737314:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10663 00000100:00080000:0.0:1787828592.737325:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828592.737332:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787828592.737370:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4e00 x1874663043257856/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828592.737398:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828592.737403:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828592.737405:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828592.737409:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10663 00000100:00080000:0.0:1787828592.737413:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828592.737418:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828592.737427:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828592.737441:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7800 x1874663043257984/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828592.737442:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043257856:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787828592.737451:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828592.737454:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828592.737456:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828592.737460:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10663 00000100:00080000:0.0:1787828592.737463:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828592.737466:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828592.737471:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828592.737480:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5500 x1874663043258112/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787828592.737501:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787828592.737536:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828592/real 1787828592] req@ffff9181c55a4e00 x1874663043257856/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828617 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828592.737551:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4e00 x1874663043257856/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828617 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828592.737558:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828592.737560:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787828592.737573:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043257856:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787828592.737597:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043257984:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787828592.737615:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787828592.737625:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828592/real 1787828592] req@ffff9181c55a7800 x1874663043257984/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828617 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828592.737633:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7800 x1874663043257984/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828617 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828592.737638:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828592.737640:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787828592.737643:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043257984:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787828592.737646:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043258112:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787828592.737656:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787828592.737661:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828592/real 1787828592] req@ffff9181c55a5500 x1874663043258112/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828617 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828592.737669:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5500 x1874663043258112/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828617 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828592.737674:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828592.737675:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787828592.737677:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043258112:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828593.248948:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828594.272885:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828595.296896:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828596.320888:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828597.344998:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828597.792888:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828597.792899:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828597.792904:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828597.792915:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10664 00000100:00080000:1.0:1787828597.792926:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828597.792934:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787828597.792979:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3100 x1874663043258240/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828597.793009:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828597.793012:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828597.793013:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828597.793016:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10664 00000100:00080000:1.0:1787828597.793019:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828597.793022:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828597.793031:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828597.793042:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1180 x1874663043258368/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828597.793054:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1787828597.793056:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043258240:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787828597.793059:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828597.793065:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828597.793072:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10664 00000100:00080000:1.0:1787828597.793074:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828597.793077:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828597.793081:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828597.793096:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0000 x1874663043258496/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828597.793148:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828597.793183:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828597/real 1787828597] req@ffff9181c35b3100 x1874663043258240/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828622 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828597.793204:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3100 x1874663043258240/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828622 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828597.793216:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828597.793221:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828597.793239:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043258240:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828597.793266:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043258368:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828597.793291:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828597.793300:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828597/real 1787828597] req@ffff9181c35b1180 x1874663043258368/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828622 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828597.793309:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1180 x1874663043258368/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828622 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828597.793315:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828597.793317:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828597.793320:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043258368:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828597.793324:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043258496:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828597.793336:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828597.793342:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828597/real 1787828597] req@ffff9181c35b0000 x1874663043258496/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828622 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828597.793355:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0000 x1874663043258496/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828622 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828597.793360:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828597.793362:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828597.793364:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043258496:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828598.368902:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828599.392922:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828600.416895:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828601.440922:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828602.464908:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828602.912870:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828602.912883:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828602.912888:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828602.912899:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10669 00000100:00080000:1.0:1787828602.912912:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828602.912920:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787828602.912964:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0e00 x1874663043258624/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828602.912992:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828602.912995:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828602.912997:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828602.913000:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10669 00000100:00080000:1.0:1787828602.913003:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828602.913007:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828602.913016:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828602.913030:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3480 x1874663043258752/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828602.913037:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043258624:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787828602.913044:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828602.913050:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828602.913053:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828602.913057:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10669 00000100:00080000:1.0:1787828602.913059:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828602.913062:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828602.913066:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828602.913074:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2680 x1874663043258880/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828602.913164:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043258752:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828602.913190:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043258880:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828603.488950:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828604.512968:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828605.536902:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1787828605.985152:0:2378:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:2.0:1787828605.985208:0:2378:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1787828605.985226:0:2378:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:2.0:1787828605.985291:0:2378:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:2.0:1787828605.985302:0:2378:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:1.0:1787828605.985345:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828602/real 1787828605] req@ffff9181c35b3480 x1874663043258752/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828627 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828605.985386:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3480 x1874663043258752/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828627 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828605.985408:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828605.985414:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787828605.985424:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043258752:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787828605.985434:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828602/real 1787828605] req@ffff9181c35b2680 x1874663043258880/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828627 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828605.985445:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2680 x1874663043258880/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828627 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828605.985452:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828605.985455:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787828605.985458:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043258880:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787828605.985465:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828602/real 1787828605] req@ffff9181c35b0e00 x1874663043258624/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828627 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787828605.985479:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0e00 x1874663043258624/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828627 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828605.985488:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787828605.985492:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787828605.985529:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043258624:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787828606.560895:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828607.584878:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828608.032869:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828608.032883:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828608.032887:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828608.032901:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10674 00000100:00080000:1.0:1787828608.032915:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828608.032925:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787828608.032970:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0e00 x1874663043259008/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828608.033008:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828608.033013:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828608.033014:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828608.033020:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10674 00000100:00080000:1.0:1787828608.033024:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828608.033030:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828608.033043:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828608.033077:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2680 x1874663043259136/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828608.033095:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043259008:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787828608.033101:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828608.033111:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828608.033114:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828608.033120:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10674 00000100:00080000:1.0:1787828608.033125:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828608.033133:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828608.033139:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828608.033158:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3480 x1874663043259264/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828608.033201:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828608.033247:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828608/real 1787828608] req@ffff9181c35b0e00 x1874663043259008/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828633 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828608.033269:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0e00 x1874663043259008/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828633 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828608.033283:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828608.033289:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828608.033317:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043259008:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828608.033356:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043259136:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828608.033390:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828608.033413:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828608/real 1787828608] req@ffff9181c35b2680 x1874663043259136/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828633 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828608.033427:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2680 x1874663043259136/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828633 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828608.033436:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828608.033440:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828608.033443:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043259136:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828608.033451:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043259264:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828608.033484:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828608.033497:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828608/real 1787828608] req@ffff9181c35b3480 x1874663043259264/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828633 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828608.033510:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3480 x1874663043259264/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828633 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828608.033518:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828608.033521:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828608.033524:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043259264:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828608.608867:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828609.632874:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828610.656872:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828611.680856:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828612.704872:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828613.152960:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 10000000:01000000:1.0:1787828613.152969:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-cliir' failed (-108). 00000100:00080000:2.0:1787828613.152972:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 10000000:00000400:1.0:1787828613.152980:0:159735:0:(mgc_request.c:1900:mgc_process_log()) MGC192.168.203.147@tcp: IR log lustre-cliir failed, not fatal: rc = -108 00000100:00080000:2.0:1787828613.152980:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 10000000:01000000:1.0:1787828613.152991:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 00000100:00080000:2.0:1787828613.152996:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10679 10000000:01000000:1.0:1787828613.152998:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff9181c5620000 from 52 10000000:01000000:1.0:1787828613.153005:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00000100:00080000:2.0:1787828613.153027:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828613.153037:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787828613.153077:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f480 x1874663043259520/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787828613.153091:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6f7ad lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828613.153102:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6f7ad lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828613.153114:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6f7ad lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787828613.153115:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828613.153131:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828613.153137:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828613.153143:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10679 00000100:00080000:2.0:1787828613.153150:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1787828613.153152:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215d180 going to sleep for 0 seconds 00000100:00080000:2.0:1787828613.153157:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787828613.153166:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043259392:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000008:00100000:2.0:1787828613.153167:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828613.153176:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043259520:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787828613.153191:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6f7ad lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787828613.153192:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f100 x1874663043259648/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787828613.153233:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6f7ad lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828613.153243:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6f7ad lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787828613.153248:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:1.0:1787828613.153250:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6f7ad lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787828613.153266:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000800:00000100:3.0:1787828613.153266:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00010000:00010000:1.0:1787828613.153269:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6f7ad lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787828613.153276:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00010000:00010000:1.0:1787828613.153279:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b5400 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787828613.153284:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6f7ad lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787828613.153292:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10679 00010000:00010000:1.0:1787828613.153295:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6f7ad lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787828613.153299:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 10000000:00010000:1.0:1787828613.153303:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6f7ad lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787828613.153310:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 10000000:01000000:1.0:1787828613.153311:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1787828613.153320:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00000008:00100000:2.0:1787828613.153322:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:1.0:1787828613.153328:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6f7ad lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787828613.153330:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828613/real 1787828613] req@ffff9181e007f480 x1874663043259520/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828638 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:1.0:1787828613.153338:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6f7ad lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787828613.153348:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ea00 x1874663043259776/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828613.153385:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f480 x1874663043259520/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828638 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828613.153401:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 10000000:01000000:1.0:1787828613.153406:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:3.0:1787828613.153410:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00080000:1.0:1787828613.153411:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00100000:3.0:1787828613.153417:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043259520:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828613.153445:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043259648:192.168.203.147@tcp:38:kworker.0 00000100:00080000:2.0:1787828613.153450:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828613.153454:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828613.153456:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828613.153462:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10684 00000100:00080000:2.0:1787828613.153470:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828613.153477:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:3.0:1787828613.153485:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828613.153501:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c700 x1874663043259904/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828613.153511:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828613/real 1787828613] req@ffff9181e007f100 x1874663043259648/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828638 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828613.153534:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f100 x1874663043259648/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828638 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828613.153544:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828613.153548:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828613.153552:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043259648:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828613.153559:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043259776:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828613.153581:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828613.153591:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828613/real 1787828613] req@ffff9181e007ea00 x1874663043259776/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828638 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828613.153603:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ea00 x1874663043259776/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828638 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828613.153612:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828613.153616:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828613.153619:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043259776:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828613.153632:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043259904:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1787828613.153646:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828613.153654:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828613/real 1787828613] req@ffff9181e007c700 x1874663043259904/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828638 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828613.153666:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c700 x1874663043259904/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828638 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828613.153674:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828613.153678:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828613.153690:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043259904:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787828613.728933:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828614.752843:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828615.776917:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828616.800877:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828617.824920:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828618.272881:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828618.272892:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828618.272896:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828618.272908:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10684 00000100:00080000:2.0:1787828618.272920:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828618.272927:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787828618.272973:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e300 x1874663043260032/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828618.273015:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828618.273018:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828618.273020:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828618.273023:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10684 00000100:00080000:2.0:1787828618.273027:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828618.273031:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828618.273041:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828618.273055:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043260032:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828618.273056:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d180 x1874663043260160/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828618.273070:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828618.273072:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828618.273073:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828618.273076:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10684 00000100:00080000:2.0:1787828618.273084:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828618.273088:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828618.273091:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828618.273101:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d500 x1874663043260288/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828618.273148:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828618.273195:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828618/real 1787828618] req@ffff9181e007e300 x1874663043260032/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828643 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828618.273215:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e300 x1874663043260032/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828643 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828618.273224:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828618.273228:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828618.273241:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043260032:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828618.273274:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043260160:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828618.273303:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828618.273315:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828618/real 1787828618] req@ffff9181e007d180 x1874663043260160/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828643 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828618.273324:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d180 x1874663043260160/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828643 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828618.273330:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828618.273333:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828618.273335:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043260160:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828618.273340:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043260288:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828618.273354:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828618.273360:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828618/real 1787828618] req@ffff9181e007d500 x1874663043260288/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828643 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828618.273368:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d500 x1874663043260288/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828643 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828618.273378:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828618.273388:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828618.273391:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043260288:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828618.848845:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828619.872845:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828620.896881:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828621.920864:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828622.944852:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828623.392862:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828623.392873:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828623.392877:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828623.392888:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10690 00000100:00080000:2.0:1787828623.392900:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828623.392907:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787828623.392968:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ed80 x1874663043260416/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828623.393004:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828623.393008:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828623.393009:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828623.393013:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10690 00000100:00080000:2.0:1787828623.393016:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828623.393021:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828623.393030:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828623.393043:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007df80 x1874663043260544/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828623.393054:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043260416:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787828623.393059:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828623.393064:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828623.393066:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828623.393072:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10690 00000100:00080000:2.0:1787828623.393076:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828623.393081:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828623.393087:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828623.393104:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d500 x1874663043260672/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828623.393143:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828623.393186:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828623/real 1787828623] req@ffff9181e007ed80 x1874663043260416/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828648 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828623.393202:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ed80 x1874663043260416/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828648 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828623.393210:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828623.393214:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828623.393231:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043260416:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828623.393254:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043260544:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828623.393275:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828623.393287:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828623/real 1787828623] req@ffff9181e007df80 x1874663043260544/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828648 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828623.393296:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007df80 x1874663043260544/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828648 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828623.393304:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828623.393306:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828623.393310:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043260544:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828623.393314:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043260672:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828623.393325:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828623.393331:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828623/real 1787828623] req@ffff9181e007d500 x1874663043260672/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828648 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828623.393338:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d500 x1874663043260672/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828648 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828623.393343:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828623.393345:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828623.393347:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043260672:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828623.968902:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828624.992954:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828626.016897:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828627.040882:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828628.064875:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828628.512814:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828628.512824:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828628.512828:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828628.512840:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10695 00000100:00080000:2.0:1787828628.512852:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828628.512859:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787828628.512902:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d180 x1874663043260800/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787828628.512935:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828628.512939:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828628.512941:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828628.512944:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10695 00000100:00080000:2.0:1787828628.512947:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828628.512950:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828628.512960:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828628.512970:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e300 x1874663043260928/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828628.512979:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043260800:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787828628.512981:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828628.512987:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787828628.512989:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828628.512993:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10695 00000100:00080000:2.0:1787828628.512995:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828628.512998:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787828628.513001:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828628.513009:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c700 x1874663043261056/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828628.513057:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828628.513092:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828628/real 1787828628] req@ffff9181e007d180 x1874663043260800/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828653 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828628.513107:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d180 x1874663043260800/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828653 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828628.513117:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828628.513120:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828628.513134:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043260800:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828628.513167:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043260928:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828628.513184:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828628.513194:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828628/real 1787828628] req@ffff9181e007e300 x1874663043260928/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828653 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828628.513203:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e300 x1874663043260928/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828653 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828628.513209:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828628.513211:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828628.513213:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043260928:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828628.513217:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043261056:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828628.513228:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828628.513233:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828628/real 1787828628] req@ffff9181e007c700 x1874663043261056/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828653 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828628.513241:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c700 x1874663043261056/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828653 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828628.513246:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828628.513248:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828628.513249:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043261056:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828629.088890:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828630.112877:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828631.136942:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828632.160986:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828633.184918:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787828633.632846:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828633.632860:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828633.632865:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787828633.632893:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10700 10000000:01000000:1.0:1787828633.632905:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-client' failed (-108). 00000100:00080000:2.0:1787828633.632907:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 10000000:00020000:1.0:1787828633.632911:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:1.0:1787828633.632918:0:159735:0:(mgc_request.c:611:do_requeue()) updating log params 00000100:00080000:2.0:1787828633.632922:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 10000000:01000000:1.0:1787828633.632923:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff9181c5620000 from 7 10000000:01000000:1.0:1787828633.632928:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00000100:00100000:2.0:1787828633.632967:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ea00 x1874663043261312/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787828633.632982:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5200/0xbef42c0221d6f7b4 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828633.632996:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b5200/0xbef42c0221d6f7b4 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828633.633009:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b5200/0xbef42c0221d6f7b4 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787828633.633012:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828633.633018:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787828633.633021:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1787828633.633026:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215d180 going to sleep for 0 seconds 00000100:00080000:2.0:1787828633.633028:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10700 00000100:00080000:2.0:1787828633.633035:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1787828633.633036:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043261184:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00080000:2.0:1787828633.633045:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:1.0:1787828633.633053:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5200/0xbef42c0221d6f7b4 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000008:00100000:2.0:1787828633.633059:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:1.0:1787828633.633061:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b5200/0xbef42c0221d6f7b4 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828633.633070:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5200/0xbef42c0221d6f7b4 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828633.633076:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b5200/0xbef42c0221d6f7b4 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787828633.633082:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f100 x1874663043261440/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787828633.633084:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b5200/0xbef42c0221d6f7b4 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828633.633103:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b5200 already unused, calling callback (ffffffffc1273c20) 00000100:00100000:3.0:1787828633.633103:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043261312:192.168.203.147@tcp:250:kworker.0 10000000:00010000:1.0:1787828633.633111:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5200/0xbef42c0221d6f7b4 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787828633.633113:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828633.633130:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00010000:00010000:1.0:1787828633.633132:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b5200/0xbef42c0221d6f7b4 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787828633.633134:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 10000000:00010000:1.0:1787828633.633144:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5200/0xbef42c0221d6f7b4 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787828633.633148:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10700 10000000:01000000:1.0:1787828633.633152:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:1.0:1787828633.633157:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00000100:00080000:2.0:1787828633.633158:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00010000:00010000:1.0:1787828633.633163:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b5200/0xbef42c0221d6f7b4 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787828633.633168:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:1.0:1787828633.633171:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b5200/0xbef42c0221d6f7b4 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000008:00100000:2.0:1787828633.633179:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1787828633.633191:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 10000000:01000000:1.0:1787828633.633192:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787828633.633196:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:CONNECTING 00000100:00100000:2.0:1787828633.633201:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f480 x1874663043261568/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828633.633241:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828633/real 1787828633] req@ffff9181e007ea00 x1874663043261312/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828658 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828633.633256:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ea00 x1874663043261312/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828658 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828633.633269:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828633.633287:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828633.633296:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043261312:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828633.633320:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043261440:192.168.203.147@tcp:38:kworker.0 00000100:00080000:2.0:1787828633.633329:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787828633.633335:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828633.633338:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000800:00000100:3.0:1787828633.633344:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1787828633.633349:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10705 00000100:00080000:2.0:1787828633.633358:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787828633.633360:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828633/real 1787828633] req@ffff9181e007f100 x1874663043261440/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828658 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828633.633372:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828633.633387:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f100 x1874663043261440/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828658 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828633.633396:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f800 x1874663043261696/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828633.633400:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828633.633412:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828633.633423:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043261440:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828633.633438:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043261568:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828633.633457:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828633.633465:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828633/real 1787828633] req@ffff9181e007f480 x1874663043261568/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828658 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828633.633473:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f480 x1874663043261568/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828658 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828633.633482:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828633.633486:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828633.633489:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043261568:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828633.633505:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043261696:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1787828633.633526:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828633.633536:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828633/real 1787828633] req@ffff9181e007f800 x1874663043261696/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828658 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828633.633545:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f800 x1874663043261696/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828658 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828633.633550:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828633.633552:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828633.633562:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043261696:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787828634.208845:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828635.232903:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828636.256943:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828637.280888:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828638.304873:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828638.752933:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828638.752943:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828638.752947:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828638.752957:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10705 00000100:00080000:1.0:1787828638.752968:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828638.752976:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787828638.753022:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2a00 x1874663043261824/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828638.753069:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828638.753075:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828638.753078:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828638.753083:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10705 00000100:00080000:1.0:1787828638.753088:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828638.753094:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828638.753106:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828638.753122:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0a80 x1874663043261952/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787828638.753128:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043261824:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787828638.753161:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828638.753167:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828638.753170:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828638.753177:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10705 00000100:00080000:1.0:1787828638.753181:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828638.753188:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828638.753195:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828638.753213:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2d80 x1874663043262080/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828638.753251:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828638.753309:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828638/real 1787828638] req@ffff9181c35b2a00 x1874663043261824/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828663 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828638.753325:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2a00 x1874663043261824/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828663 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828638.753335:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828638.753338:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828638.753352:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043261824:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828638.753394:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043261952:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828638.753419:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828638.753430:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828638/real 1787828638] req@ffff9181c35b0a80 x1874663043261952/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828663 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828638.753441:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0a80 x1874663043261952/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828663 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828638.753448:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828638.753451:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828638.753454:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043261952:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828638.753458:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043262080:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828638.753471:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828638.753479:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828638/real 1787828638] req@ffff9181c35b2d80 x1874663043262080/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828663 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828638.753489:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2d80 x1874663043262080/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828663 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828638.753495:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828638.753498:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828638.753500:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043262080:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828639.328978:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828640.352970:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828641.376889:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828642.400931:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828643.424921:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828643.872911:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828643.872921:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828643.872924:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828643.872933:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10710 00000100:00080000:1.0:1787828643.872942:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828643.872948:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787828643.872982:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1f80 x1874663043262208/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828643.873007:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828643.873010:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828643.873012:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828643.873015:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10710 00000100:00080000:1.0:1787828643.873017:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828643.873021:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828643.873030:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828643.873039:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3800 x1874663043262336/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828643.873048:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828643.873051:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828643.873053:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828643.873057:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10710 00000100:00080000:1.0:1787828643.873060:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787828643.873065:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043262208:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787828643.873066:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828643.873077:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828643.873085:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3b80 x1874663043262464/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828643.873155:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828643.873196:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828643/real 1787828643] req@ffff9181c35b1f80 x1874663043262208/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828668 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828643.873217:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1f80 x1874663043262208/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828668 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828643.873226:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828643.873230:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828643.873243:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043262208:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828643.873279:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043262336:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828643.873302:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828643.873313:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828643/real 1787828643] req@ffff9181c35b3800 x1874663043262336/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828668 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828643.873323:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3800 x1874663043262336/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828668 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828643.873333:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828643.873347:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828643.873351:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043262336:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828643.873358:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043262464:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828643.873380:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828643.873392:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828643/real 1787828643] req@ffff9181c35b3b80 x1874663043262464/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828668 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828643.873401:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3b80 x1874663043262464/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828668 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828643.873411:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828643.873414:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828643.873416:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043262464:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828644.448933:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828645.472960:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828646.497050:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828647.520913:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828648.544913:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828648.992876:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828648.992905:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828648.992910:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828648.992920:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10715 00000100:00080000:1.0:1787828648.992933:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828648.992940:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787828648.992978:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0700 x1874663043262592/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828648.993026:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828648.993031:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828648.993033:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828648.993038:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10715 00000100:00080000:1.0:1787828648.993042:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828648.993045:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828648.993053:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828648.993066:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3b80 x1874663043262720/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828648.993092:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828648.993095:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828648.993097:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787828648.993103:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043262592:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787828648.993104:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10715 00000100:00080000:1.0:1787828648.993109:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828648.993115:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828648.993119:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828648.993129:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3800 x1874663043262848/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828648.993171:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828648.993202:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828648/real 1787828648] req@ffff9181c35b0700 x1874663043262592/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828673 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828648.993215:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0700 x1874663043262592/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828673 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828648.993224:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828648.993243:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828648.993256:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043262592:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828648.993275:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043262720:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828648.993312:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828648.993325:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828648/real 1787828648] req@ffff9181c35b3b80 x1874663043262720/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828673 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828648.993334:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3b80 x1874663043262720/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828673 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828648.993345:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828648.993348:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828648.993350:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043262720:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828648.993354:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043262848:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787828648.993366:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828648.993372:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828648/real 1787828648] req@ffff9181c35b3800 x1874663043262848/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828673 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828648.993380:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3800 x1874663043262848/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828673 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828648.993385:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828648.993387:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828648.993389:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043262848:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828649.568963:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828650.592916:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828651.616867:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828652.640915:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828653.664952:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828654.112898:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828654.112907:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828654.112909:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828654.112918:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10720 00000100:00080000:0.0:1787828654.112929:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 10000000:01000000:1.0:1787828654.112934:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'params' failed (-108). 00000100:00080000:0.0:1787828654.112936:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 10000000:00020000:1.0:1787828654.112954:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:1.0:1787828654.112961:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:1.0:1787828654.112964:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff9181c87b19b8 from 1 10000000:01000000:1.0:1787828654.112968:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00000100:00100000:0.0:1787828654.112969:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5180 x1874663043262976/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828654.112987:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828654.112991:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828654.112993:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828654.112996:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10720 00000100:00080000:0.0:1787828654.112999:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828654.113002:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:1.0:1787828654.113007:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5600/0xbef42c0221d6f7bb lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000008:00100000:0.0:1787828654.113009:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828654.113013:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043262976:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787828654.113026:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4380 x1874663043263232/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787828654.113028:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b5600/0xbef42c0221d6f7bb lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787828654.113047:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828654.113051:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00010000:00010000:1.0:1787828654.113052:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b5600/0xbef42c0221d6f7bb lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787828654.113053:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828654.113060:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10720 00000100:00080000:0.0:1787828654.113066:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828654.113069:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828654.113073:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828654.113076:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215d180 going to sleep for 0 seconds 00000800:00000100:3.0:1787828654.113083:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787828654.113085:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5c00 x1874663043263360/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828654.113086:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043263104:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787828654.113107:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5600/0xbef42c0221d6f7bb lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828654.113114:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b5600/0xbef42c0221d6f7bb lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828654.113120:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5600/0xbef42c0221d6f7bb lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828654.113128:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b5600/0xbef42c0221d6f7bb lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828654.113134:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b5600/0xbef42c0221d6f7bb lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787828654.113135:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828654/real 1787828654] req@ffff9181c55a5180 x1874663043262976/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828679 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:1.0:1787828654.113156:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b5600 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787828654.113162:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5600/0xbef42c0221d6f7bb lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787828654.113169:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5180 x1874663043262976/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828679 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:1.0:1787828654.113175:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b5600/0xbef42c0221d6f7bb lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787828654.113187:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 10000000:00010000:1.0:1787828654.113189:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5600/0xbef42c0221d6f7bb lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787828654.113196:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 10000000:01000000:1.0:1787828654.113198:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1787828654.113204:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00000100:00100000:3.0:1787828654.113204:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043262976:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787828654.113213:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b5600/0xbef42c0221d6f7bb lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828654.113220:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b5600/0xbef42c0221d6f7bb lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787828654.113232:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043263232:192.168.203.147@tcp:38:kworker.0 10000000:01000000:1.0:1787828654.113236:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787828654.113240:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000800:00000100:3.0:1787828654.113263:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828654.113280:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828654/real 1787828654] req@ffff9181c55a4380 x1874663043263232/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828679 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828654.113293:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787828654.113298:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4380 x1874663043263232/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828679 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787828654.113299:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828654.113306:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828654.113315:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10725 00000100:00080000:3.0:1787828654.113315:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787828654.113323:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828654.113325:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00080000:0.0:1787828654.113330:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828654.113335:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043263232:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828654.113346:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043263360:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787828654.113349:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4700 x1874663043263488/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787828654.113379:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828654.113393:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828654/real 1787828654] req@ffff9181c55a5c00 x1874663043263360/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828679 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828654.113406:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5c00 x1874663043263360/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828679 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828654.113416:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828654.113420:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828654.113424:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043263360:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828654.113439:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043263488:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1787828654.113461:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787828654.113472:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828654/real 1787828654] req@ffff9181c55a4700 x1874663043263488/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828679 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828654.113485:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4700 x1874663043263488/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828679 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828654.113494:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828654.113497:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828654.113512:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043263488:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1787828654.688921:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828655.712855:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828656.736864:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828657.760917:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828658.784885:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828659.232889:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828659.232900:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828659.232904:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828659.232914:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10725 00000100:00080000:3.0:1787828659.232929:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828659.232939:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828659.232997:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043263616/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828659.233028:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828659.233031:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828659.233033:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828659.233038:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10725 00000100:00080000:3.0:1787828659.233043:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828659.233048:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828659.233058:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828659.233072:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043263744/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828659.233083:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043263616:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787828659.233086:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828659.233099:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828659.233101:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828659.233106:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10725 00000100:00080000:3.0:1787828659.233109:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828659.233113:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828659.233119:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828659.233132:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043263872/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828659.233159:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828659.233205:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828659/real 1787828659] req@ffff9181c5637100 x1874663043263616/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828684 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828659.233221:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043263616/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828684 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828659.233231:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828659.233236:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828659.233265:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043263616:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828659.233296:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043263744:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828659.233334:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828659.233349:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828659/real 1787828659] req@ffff9181c5637480 x1874663043263744/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828684 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828659.233361:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043263744/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828684 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828659.233369:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828659.233372:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828659.233375:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043263744:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828659.233381:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043263872:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828659.233407:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828659.233416:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828659/real 1787828659] req@ffff9181c5636a00 x1874663043263872/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828684 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828659.233426:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043263872/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828684 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828659.233433:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828659.233435:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828659.233438:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043263872:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828659.808887:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828660.832859:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828661.856893:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828662.880866:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828663.904911:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828664.352833:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828664.352841:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828664.352844:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828664.352853:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10730 00000100:00080000:3.0:1787828664.352864:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828664.352874:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828664.352914:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043264000/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828664.352945:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828664.352949:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828664.352951:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828664.352955:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10730 00000100:00080000:3.0:1787828664.352958:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828664.352962:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828664.352970:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828664.352983:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634e00 x1874663043264128/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828664.352995:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828664.353000:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828664.353001:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1787828664.353005:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043264000:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787828664.353006:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10730 00000100:00080000:3.0:1787828664.353014:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828664.353020:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828664.353026:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828664.353038:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043264256/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828664.353069:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828664.353101:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828664/real 1787828664] req@ffff9181c5634380 x1874663043264000/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828689 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828664.353113:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043264000/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828689 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828664.353121:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828664.353124:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828664.353136:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043264000:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828664.353154:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043264128:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828664.353169:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828664.353177:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828664/real 1787828664] req@ffff9181c5634e00 x1874663043264128/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828689 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828664.353187:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634e00 x1874663043264128/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828689 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828664.353194:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828664.353197:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828664.353199:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043264128:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828664.353205:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043264256:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828664.353218:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828664.353224:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828664/real 1787828664] req@ffff9181c5635500 x1874663043264256/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828689 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828664.353231:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043264256/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828689 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828664.353236:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828664.353238:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828664.353240:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043264256:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828664.928875:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828665.952882:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828666.976876:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828668.000888:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828669.024874:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828669.472869:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828669.472876:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828669.472880:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828669.472888:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10736 00000100:00080000:1.0:1787828669.472898:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828669.472904:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787828669.472936:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1f80 x1874663043264384/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828669.472960:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828669.472963:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828669.472964:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828669.472967:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10736 00000100:00080000:1.0:1787828669.472970:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828669.472972:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828669.472979:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828669.472988:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2d80 x1874663043264512/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828669.472996:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828669.472998:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828669.473000:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828669.473002:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10736 00000100:00080000:1.0:1787828669.473005:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1787828669.473007:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043264384:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787828669.473010:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828669.473017:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828669.473025:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0a80 x1874663043264640/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828669.473107:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043264512:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828669.473152:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043264640:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828670.048957:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828671.072935:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828672.096894:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1787828672.545033:0:2379:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:1.0:1787828672.545080:0:2379:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787828672.545097:0:2379:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1787828672.545160:0:2379:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1787828672.545172:0:2379:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:2.0:1787828672.545219:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828669/real 1787828672] req@ffff9181c35b0a80 x1874663043264640/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828694 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828672.545254:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0a80 x1874663043264640/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828694 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828672.545278:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828672.545283:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828672.545287:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043264640:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828672.545296:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828669/real 1787828672] req@ffff9181c35b1f80 x1874663043264384/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828694 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828672.545309:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1f80 x1874663043264384/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828694 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828672.545316:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828672.545319:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828672.545348:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043264384:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828672.545355:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828669/real 1787828672] req@ffff9181c35b2d80 x1874663043264512/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828694 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828672.545367:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2d80 x1874663043264512/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828694 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828672.545375:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828672.545379:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828672.545382:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043264512:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1787828673.120902:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787828674.144858:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-sptlrpc' failed (-108). 10000000:00020000:1.0:1787828674.144866:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00010000:00010000:3.0:1787828674.144868:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828674.592913:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828674.592923:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828674.592926:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828674.592935:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10741 00000100:00080000:1.0:1787828674.592946:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828674.592952:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787828674.592984:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2a00 x1874663043264768/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828674.593010:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828674.593014:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828674.593017:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828674.593022:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10741 00000100:00080000:1.0:1787828674.593027:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828674.593033:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828674.593044:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828674.593081:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043264768:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787828674.593088:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3480 x1874663043264896/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828674.593102:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828674.593106:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828674.593108:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828674.593113:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10741 00000100:00080000:1.0:1787828674.593118:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828674.593123:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828674.593129:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828674.593143:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2680 x1874663043265024/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828674.593170:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828674.593213:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828674/real 1787828674] req@ffff9181c35b2a00 x1874663043264768/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828699 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828674.593232:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2a00 x1874663043264768/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828699 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828674.593242:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828674.593247:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828674.593251:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043264768:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828674.593277:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043264896:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828674.593304:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828674.593318:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828674/real 1787828674] req@ffff9181c35b3480 x1874663043264896/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828699 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828674.593331:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3480 x1874663043264896/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828699 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828674.593340:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828674.593343:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828674.593346:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043264896:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828674.593353:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043265024:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828674.593373:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828674.593385:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828674/real 1787828674] req@ffff9181c35b2680 x1874663043265024/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828699 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828674.593404:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2680 x1874663043265024/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828699 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828674.593412:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828674.593415:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828674.593418:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043265024:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787828675.168856:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787828675.872838:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:1.0:1787828675.872845:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff9181c5620000 from 341 10000000:01000000:1.0:1787828675.872850:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:1.0:1787828675.872892:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5600/0xbef42c0221d6f7c2 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828675.872901:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b5600/0xbef42c0221d6f7c2 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828675.872910:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b5600/0xbef42c0221d6f7c2 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787828675.872925:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215d180 going to sleep for 0 seconds 00000100:00100000:1.0:1787828675.872951:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043265152:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787828675.872966:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5600/0xbef42c0221d6f7c2 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828675.872972:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b5600/0xbef42c0221d6f7c2 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828675.872978:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5600/0xbef42c0221d6f7c2 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828675.872987:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b5600/0xbef42c0221d6f7c2 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828675.872994:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b5600/0xbef42c0221d6f7c2 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828675.872999:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b5600 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787828675.873001:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5600/0xbef42c0221d6f7c2 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828675.873007:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b5600/0xbef42c0221d6f7c2 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787828675.873013:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5600/0xbef42c0221d6f7c2 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787828675.873017:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:1.0:1787828675.873019:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787828675.873024:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b5600/0xbef42c0221d6f7c2 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828675.873029:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b5600/0xbef42c0221d6f7c2 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787828675.873046:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787828675.873048:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:3.0:1787828675.873144:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828675.873149:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828675.873158:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828675.873163:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10746 00000100:00080000:3.0:1787828675.873170:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828675.873175:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828675.873193:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635f80 x1874663043265280/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828675.873213:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828675.873216:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828675.873218:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828675.873223:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10746 00000100:00080000:3.0:1787828675.873226:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828675.873231:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828675.873239:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828675.873252:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043265408/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828675.873262:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043265280:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787828675.873263:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828675.873267:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828675.873269:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828675.873273:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10746 00000100:00080000:3.0:1787828675.873276:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828675.873280:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828675.873285:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828675.873294:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634e00 x1874663043265536/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828675.873320:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828675.873352:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828675/real 1787828675] req@ffff9181c5635f80 x1874663043265280/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828700 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828675.873363:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635f80 x1874663043265280/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828700 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828675.873370:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828675.873373:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828675.873383:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043265280:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828675.873399:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043265408:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828675.873417:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828675.873426:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828675/real 1787828675] req@ffff9181c5635500 x1874663043265408/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828700 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828675.873433:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043265408/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828700 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828675.873444:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828675.873446:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828675.873449:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043265408:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828675.873452:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043265536:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828675.873462:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828675.873468:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828675/real 1787828675] req@ffff9181c5634e00 x1874663043265536/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828700 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828675.873475:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634e00 x1874663043265536/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828700 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828675.873481:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828675.873483:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828675.873485:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043265536:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828676.192877:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828677.216878:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828678.240899:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828679.264930:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828680.288916:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828681.248873:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828681.248881:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828681.248885:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828681.248893:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10747 00000100:00080000:1.0:1787828681.248902:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828681.248908:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787828681.248940:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0e00 x1874663043265664/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828681.248961:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828681.248964:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828681.248965:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828681.248968:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10747 00000100:00080000:1.0:1787828681.248970:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828681.248973:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828681.248987:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828681.248997:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0000 x1874663043265792/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828681.249002:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043265664:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787828681.249009:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828681.249013:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828681.249014:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828681.249017:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10747 00000100:00080000:1.0:1787828681.249020:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828681.249022:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828681.249026:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828681.249033:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1180 x1874663043265920/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828681.249063:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828681.249094:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828681/real 1787828681] req@ffff9181c35b0e00 x1874663043265664/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828706 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828681.249107:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0e00 x1874663043265664/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828706 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828681.249115:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828681.249118:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828681.249129:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043265664:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828681.249149:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043265792:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828681.249166:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828681.249175:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828681/real 1787828681] req@ffff9181c35b0000 x1874663043265792/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828706 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828681.249183:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0000 x1874663043265792/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828706 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828681.249189:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828681.249191:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828681.249193:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043265792:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828681.249196:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043265920:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828681.249207:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828681.249212:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828681/real 1787828681] req@ffff9181c35b1180 x1874663043265920/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828706 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828681.249219:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1180 x1874663043265920/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828706 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828681.249225:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828681.249227:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828681.249228:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043265920:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828681.312870:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828682.336860:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828683.360938:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828684.384864:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828685.408932:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828686.368893:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828686.368907:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828686.368913:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828686.368926:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10752 00000100:00080000:1.0:1787828686.368940:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828686.368950:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787828686.369003:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3100 x1874663043266048/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828686.369051:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828686.369056:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828686.369058:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828686.369063:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10752 00000100:00080000:1.0:1787828686.369068:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828686.369075:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828686.369088:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828686.369106:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1500 x1874663043266176/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828686.369119:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828686.369126:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828686.369129:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1787828686.369129:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043266048:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787828686.369141:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10752 00000100:00080000:1.0:1787828686.369150:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828686.369155:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828686.369161:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828686.369176:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0380 x1874663043266304/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828686.369242:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828686.369301:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828686/real 1787828686] req@ffff9181c35b3100 x1874663043266048/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828711 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828686.369320:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3100 x1874663043266048/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828711 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828686.369334:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828686.369340:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828686.369363:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043266048:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828686.369392:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043266176:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828686.369418:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828686.369430:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828686/real 1787828686] req@ffff9181c35b1500 x1874663043266176/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828711 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828686.369440:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1500 x1874663043266176/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828711 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828686.369448:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828686.369450:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828686.369453:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043266176:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828686.369458:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043266304:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828686.369473:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828686.369480:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828686/real 1787828686] req@ffff9181c35b0380 x1874663043266304/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828711 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828686.369489:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0380 x1874663043266304/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828711 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828686.369501:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828686.369504:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828686.369527:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043266304:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828686.432875:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828687.456915:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828688.480940:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828689.504970:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828690.528917:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828691.488853:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828691.488865:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828691.488869:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828691.488881:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10758 00000100:00080000:1.0:1787828691.488891:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828691.488898:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787828691.488950:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1c00 x1874663043266432/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828691.488987:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828691.488993:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828691.488995:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828691.489002:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10758 00000100:00080000:1.0:1787828691.489006:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828691.489012:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828691.489040:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828691.489049:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043266432:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787828691.489055:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0380 x1874663043266560/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828691.489066:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828691.489069:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828691.489071:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828691.489075:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10758 00000100:00080000:1.0:1787828691.489079:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828691.489082:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828691.489087:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828691.489128:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1500 x1874663043266688/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828691.489129:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828691.489178:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828691/real 1787828691] req@ffff9181c35b1c00 x1874663043266432/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828716 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828691.489200:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1c00 x1874663043266432/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828716 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828691.489209:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828691.489213:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828691.489227:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043266432:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828691.489256:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043266560:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828691.489288:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828691.489306:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828691/real 1787828691] req@ffff9181c35b0380 x1874663043266560/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828716 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828691.489318:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0380 x1874663043266560/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828716 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828691.489325:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828691.489328:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828691.489331:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043266560:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828691.489355:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043266688:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828691.489374:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828691.489383:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828691/real 1787828691] req@ffff9181c35b1500 x1874663043266688/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828716 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828691.489394:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1500 x1874663043266688/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828716 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828691.489401:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828691.489403:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828691.489406:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043266688:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828691.552944:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828692.576884:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828693.600958:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828694.624996:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828695.648947:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787828696.096919:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-cliir' failed (-108). 10000000:00000400:1.0:1787828696.096927:0:159735:0:(mgc_request.c:1900:mgc_process_log()) MGC192.168.203.147@tcp: IR log lustre-cliir failed, not fatal: rc = -108 10000000:01000000:1.0:1787828696.096935:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:1.0:1787828696.096937:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff9181c5620000 from 52 10000000:01000000:1.0:1787828696.096940:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:1.0:1787828696.096984:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f7c9 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828696.096993:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f7c9 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828696.097018:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f7c9 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787828696.097036:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215d180 going to sleep for 0 seconds 00000100:00100000:1.0:1787828696.097046:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043266816:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787828696.097056:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f7c9 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828696.097062:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f7c9 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828696.097067:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f7c9 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828696.097073:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f7c9 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828696.097078:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f7c9 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828696.097083:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4c00 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787828696.097086:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f7c9 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828696.097091:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f7c9 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787828696.097097:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f7c9 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787828696.097102:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1787828696.097104:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787828696.097109:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f7c9 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828696.097113:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b4c00/0xbef42c0221d6f7c9 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787828696.097129:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787828696.097131:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:3.0:1787828696.097225:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828696.097233:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828696.097237:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828696.097246:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10763 00000100:00080000:3.0:1787828696.097257:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828696.097264:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828696.097325:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043266944/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828696.097360:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828696.097364:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828696.097366:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828696.097371:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10763 00000100:00080000:3.0:1787828696.097374:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828696.097380:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828696.097391:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828696.097405:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043267072/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828696.097418:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828696.097422:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828696.097424:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828696.097432:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10763 00000100:00100000:2.0:1787828696.097436:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043266944:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787828696.097437:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828696.097473:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828696.097504:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828696.097514:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043267200/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828696.097612:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828696.097669:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828696/real 1787828696] req@ffff9181c5634380 x1874663043266944/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828721 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828696.097699:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043266944/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828721 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828696.097730:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828696.097738:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828696.097871:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043266944:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828696.097919:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043267072:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828696.097951:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828696.097967:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828696/real 1787828696] req@ffff9181c5636a00 x1874663043267072/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828721 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828696.097982:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043267072/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828721 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828696.097991:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828696.097994:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828696.097998:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043267072:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828696.098003:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043267200:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828696.098034:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828696.098041:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828696/real 1787828696] req@ffff9181c5637480 x1874663043267200/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828721 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828696.098050:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043267200/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828721 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828696.098055:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828696.098057:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828696.098059:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043267200:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828696.672863:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828697.696864:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828698.720897:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828699.744865:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828700.768856:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828701.216893:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828701.216906:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828701.216910:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828701.216921:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10767 00000100:00080000:3.0:1787828701.216933:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828701.216943:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828701.216995:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043267328/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828701.217022:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828701.217026:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828701.217029:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828701.217034:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10767 00000100:00080000:3.0:1787828701.217038:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828701.217043:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828701.217064:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828701.217074:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043267328:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828701.217081:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043267456/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828701.217096:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828701.217100:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828701.217102:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828701.217107:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10767 00000100:00080000:3.0:1787828701.217134:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828701.217140:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:2.0:1787828701.217146:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:3.0:1787828701.217147:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828701.217166:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043267584/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828701.217192:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828701/real 1787828701] req@ffff9181c5637100 x1874663043267328/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828726 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828701.217207:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043267328/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828726 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828701.217220:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828701.217223:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828701.217235:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043267328:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828701.217264:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043267456:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828701.217292:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828701.217303:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828701/real 1787828701] req@ffff9181c5637800 x1874663043267456/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828726 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828701.217312:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043267456/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828726 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828701.217318:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828701.217321:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828701.217323:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043267456:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828701.217327:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043267584:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828701.217349:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828701.217355:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828701/real 1787828701] req@ffff9181c5636d80 x1874663043267584/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828726 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828701.217362:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043267584/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828726 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828701.217367:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828701.217369:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828701.217371:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043267584:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828701.792887:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828702.816880:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828703.840840:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828704.864859:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828705.888872:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828706.336823:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828706.336830:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828706.336834:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828706.336841:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10772 00000100:00080000:0.0:1787828706.336851:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828706.336857:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787828706.336888:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5f80 x1874663043267712/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828706.336908:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828706.336911:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828706.336913:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828706.336917:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10772 00000100:00080000:0.0:1787828706.336920:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828706.336924:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828706.336933:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828706.336946:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6d80 x1874663043267840/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828706.336967:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828706.336971:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828706.336974:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828706.336979:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10772 00000100:00080000:0.0:1787828706.336983:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828706.336986:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828706.336991:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828706.336999:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043267712:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787828706.337002:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7100 x1874663043267968/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828706.337070:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828706.337101:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828706/real 1787828706] req@ffff9181c55a5f80 x1874663043267712/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828731 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828706.337114:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5f80 x1874663043267712/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828731 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828706.337122:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828706.337125:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828706.337136:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043267712:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828706.337142:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043267840:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828706.337158:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828706.337170:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828706/real 1787828706] req@ffff9181c55a6d80 x1874663043267840/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828731 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828706.337179:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6d80 x1874663043267840/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828731 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828706.337184:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828706.337187:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828706.337189:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043267840:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828706.337206:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043267968:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828706.337217:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828706.337223:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828706/real 1787828706] req@ffff9181c55a7100 x1874663043267968/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828731 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828706.337231:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7100 x1874663043267968/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828731 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828706.337236:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828706.337238:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828706.337240:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043267968:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828706.912883:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828707.936878:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828708.960957:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828709.984934:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828711.008961:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828711.456856:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828711.456865:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828711.456869:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828711.456879:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10778 00000100:00080000:0.0:1787828711.456890:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828711.456898:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787828711.456939:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7480 x1874663043268096/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828711.456970:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828711.456974:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828711.456976:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828711.456980:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10778 00000100:00080000:0.0:1787828711.456984:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828711.457000:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828711.457011:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828711.457019:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043268096:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787828711.457026:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5880 x1874663043268224/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828711.457040:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828711.457043:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828711.457045:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828711.457050:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10778 00000100:00080000:0.0:1787828711.457054:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828711.457058:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828711.457062:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828711.457073:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6680 x1874663043268352/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828711.457080:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828711.457115:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828711/real 1787828711] req@ffff9181c55a7480 x1874663043268096/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828736 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828711.457139:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7480 x1874663043268096/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828736 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828711.457147:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828711.457150:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828711.457161:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043268096:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828711.457179:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043268224:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828711.457194:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828711.457203:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828711/real 1787828711] req@ffff9181c55a5880 x1874663043268224/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828736 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828711.457212:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5880 x1874663043268224/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828736 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828711.457232:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828711.457235:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828711.457239:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043268224:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828711.457244:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043268352:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828711.457256:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828711.457262:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828711/real 1787828711] req@ffff9181c55a6680 x1874663043268352/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828736 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828711.457269:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6680 x1874663043268352/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828736 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828711.457274:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828711.457276:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828711.457278:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043268352:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828712.032915:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828713.056964:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828714.080918:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828715.104942:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828716.128951:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828716.576881:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828716.576889:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828716.576892:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 10000000:01000000:1.0:1787828716.576898:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-client' failed (-108). 00000100:00080000:3.0:1787828716.576901:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10783 10000000:00020000:1.0:1787828716.576906:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:1.0:1787828716.576910:0:159735:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:1.0:1787828716.576912:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff9181c5620000 from 7 00000100:00080000:3.0:1787828716.576913:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 10000000:01000000:1.0:1787828716.576915:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00000100:00080000:3.0:1787828716.576921:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828716.576959:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043268608/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787828716.576964:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f7d0 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828716.576984:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f7d0 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828716.576994:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f7d0 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787828716.577011:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215d180 going to sleep for 0 seconds 00000100:00080000:3.0:1787828716.577011:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828716.577016:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828716.577018:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1787828716.577020:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043268480:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00080000:3.0:1787828716.577025:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10783 00000100:00080000:3.0:1787828716.577031:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00010000:00010000:1.0:1787828716.577033:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f7d0 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787828716.577042:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:1.0:1787828716.577043:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f7d0 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828716.577052:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f7d0 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000008:00100000:3.0:1787828716.577054:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:1.0:1787828716.577057:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f7d0 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828716.577065:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f7d0 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828716.577070:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b5000 already unused, calling callback (ffffffffc1273c20) 00000100:00100000:3.0:1787828716.577076:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637b80 x1874663043268736/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 10000000:00010000:1.0:1787828716.577080:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f7d0 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787828716.577080:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043268608:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787828716.577096:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828716.577106:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828716.577109:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00010000:00010000:1.0:1787828716.577110:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f7d0 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787828716.577115:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10783 10000000:00010000:1.0:1787828716.577121:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f7d0 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787828716.577124:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 10000000:01000000:1.0:1787828716.577127:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00000100:00080000:3.0:1787828716.577130:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:1.0:1787828716.577132:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00000008:00100000:3.0:1787828716.577136:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:1.0:1787828716.577139:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f7d0 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828716.577145:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f7d0 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787828716.577149:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043268864/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 10000000:01000000:1.0:1787828716.577165:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787828716.577169:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:CONNECTING 00000800:00000100:2.0:1787828716.577184:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828716.577225:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828716/real 1787828716] req@ffff9181c5635c00 x1874663043268608/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828741 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828716.577240:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043268608/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828741 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828716.577249:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828716.577265:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828716.577282:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043268608:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787828716.577306:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828716.577312:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828716.577314:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828716.577319:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10788 00000100:00080000:3.0:1787828716.577325:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1787828716.577328:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043268736:192.168.203.147@tcp:38:kworker.0 00000100:00080000:3.0:1787828716.577331:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828716.577350:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634a80 x1874663043268992/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828716.577373:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828716.577391:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828716/real 1787828716] req@ffff9181c5637b80 x1874663043268736/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828741 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828716.577405:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637b80 x1874663043268736/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828741 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828716.577414:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828716.577419:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828716.577422:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043268736:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828716.577430:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043268864:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828716.577452:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828716.577464:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828716/real 1787828716] req@ffff9181c5634000 x1874663043268864/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828741 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828716.577478:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043268864/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828741 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828716.577486:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828716.577496:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828716.577501:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043268864:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828716.577517:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043268992:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787828716.577539:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828716.577551:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828716/real 1787828716] req@ffff9181c5634a80 x1874663043268992/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828741 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828716.577564:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634a80 x1874663043268992/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828741 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828716.577573:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828716.577576:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828716.577592:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043268992:192.168.203.147@tcp:250:kworker.0 00010000:00010000:0.0:1787828717.152908:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828718.176890:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828719.200886:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828720.224979:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828721.248839:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828721.696816:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828721.696826:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828721.696830:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828721.696841:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10788 00000100:00080000:3.0:1787828721.696852:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828721.696860:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828721.696904:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634a80 x1874663043269120/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828721.696926:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828721.696929:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828721.696930:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828721.696933:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10788 00000100:00080000:3.0:1787828721.696936:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828721.696939:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828721.696946:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828721.696960:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043269248/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828721.696968:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828721.696972:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:2.0:1787828721.696973:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043269120:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787828721.696974:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828721.696980:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10788 00000100:00080000:3.0:1787828721.696985:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828721.696987:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828721.696991:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828721.697004:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637b80 x1874663043269376/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828721.697037:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828721.697076:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828721/real 1787828721] req@ffff9181c5634a80 x1874663043269120/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828746 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828721.697090:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634a80 x1874663043269120/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828746 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828721.697098:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828721.697101:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828721.697112:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043269120:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828721.697133:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043269248:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828721.697157:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828721.697171:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828721/real 1787828721] req@ffff9181c5634000 x1874663043269248/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828746 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828721.697179:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043269248/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828746 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828721.697190:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828721.697192:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828721.697194:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043269248:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828721.697198:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043269376:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828721.697209:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828721.697215:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828721/real 1787828721] req@ffff9181c5637b80 x1874663043269376/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828746 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828721.697222:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637b80 x1874663043269376/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828746 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828721.697227:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828721.697229:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828721.697231:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043269376:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828722.272914:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828723.296885:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828724.320952:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828725.344919:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828726.368932:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828726.816874:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828726.816883:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828726.816886:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828726.816894:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10793 00000100:00080000:0.0:1787828726.816903:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828726.816909:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787828726.816942:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7b80 x1874663043269504/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828726.816967:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828726.816970:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828726.816971:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828726.816974:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10793 00000100:00080000:0.0:1787828726.816976:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828726.816978:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828726.816985:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828726.816995:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6680 x1874663043269632/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828726.817003:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828726.817007:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828726.817009:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828726.817012:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10793 00000100:00080000:0.0:1787828726.817015:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1787828726.817015:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043269504:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787828726.817022:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828726.817027:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828726.817035:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5880 x1874663043269760/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828726.817076:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828726.817118:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828726/real 1787828726] req@ffff9181c55a7b80 x1874663043269504/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828751 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828726.817133:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7b80 x1874663043269504/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828751 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828726.817144:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828726.817148:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828726.817167:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043269504:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828726.817194:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043269632:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828726.817218:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828726.817254:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828726/real 1787828726] req@ffff9181c55a6680 x1874663043269632/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828751 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828726.817287:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6680 x1874663043269632/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828751 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828726.817297:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828726.817301:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828726.817305:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043269632:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828726.817313:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043269760:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828726.817344:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828726.817366:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828726/real 1787828726] req@ffff9181c55a5880 x1874663043269760/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828751 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828726.817380:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5880 x1874663043269760/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828751 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828726.817389:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828726.817393:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828726.817397:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043269760:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1787828727.392850:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828728.416884:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828729.440817:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828730.464861:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828731.488870:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828731.936885:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828731.936893:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828731.936896:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828731.936905:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10798 00000100:00080000:3.0:1787828731.936915:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828731.936921:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828731.936953:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043269888/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828731.936974:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828731.936977:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828731.936978:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828731.936981:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10798 00000100:00080000:3.0:1787828731.936983:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828731.936986:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828731.936992:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828731.937001:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043270016/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828731.937013:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828731.937017:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:2.0:1787828731.937018:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043269888:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787828731.937019:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828731.937026:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10798 00000100:00080000:3.0:1787828731.937029:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828731.937031:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828731.937034:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828731.937042:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043270144/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828731.937079:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828731.937114:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828731/real 1787828731] req@ffff9181c5635c00 x1874663043269888/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828756 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828731.937128:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043269888/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828756 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828731.937135:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828731.937138:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828731.937151:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043269888:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828731.937168:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043270016:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828731.937186:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828731.937210:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828731/real 1787828731] req@ffff9181c5636d80 x1874663043270016/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828756 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828731.937219:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043270016/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828756 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828731.937225:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828731.937227:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828731.937229:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043270016:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828731.937233:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043270144:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828731.937245:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828731.937251:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828731/real 1787828731] req@ffff9181c5637800 x1874663043270144/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828756 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828731.937258:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043270144/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828756 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828731.937268:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828731.937270:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828731.937272:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043270144:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828732.512924:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828733.536873:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828734.560852:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828735.584887:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828736.608883:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828737.056837:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828737.056855:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828737.056859:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 10000000:01000000:1.0:1787828737.056869:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'params' failed (-108). 00000100:00080000:3.0:1787828737.056871:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10803 10000000:00020000:1.0:1787828737.056876:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:1.0:1787828737.056881:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:1.0:1787828737.056884:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff9181c87b19b8 from 1 00000100:00080000:3.0:1787828737.056886:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 10000000:01000000:1.0:1787828737.056887:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00000100:00080000:3.0:1787828737.056896:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828737.056925:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043270400/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787828737.056935:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f7d7 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828737.056944:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f7d7 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787828737.056953:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:1.0:1787828737.056954:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f7d7 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787828737.056964:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828737.056968:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1787828737.056976:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215d180 going to sleep for 0 seconds 00000100:00080000:3.0:1787828737.056976:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10803 00000100:00080000:3.0:1787828737.056985:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1787828737.056989:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043270272:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00100000:2.0:1787828737.056990:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043270400:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787828737.056994:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:1.0:1787828737.057005:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f7d7 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000008:00100000:3.0:1787828737.057008:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:1.0:1787828737.057017:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f7d7 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828737.057023:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f7d7 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787828737.057029:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043270528/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787828737.057042:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f7d7 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828737.057059:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f7d7 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787828737.057066:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:1.0:1787828737.057067:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b5000 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787828737.057072:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f7d7 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787828737.057073:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828737.057078:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00010000:00010000:1.0:1787828737.057082:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f7d7 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787828737.057086:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10803 10000000:00010000:1.0:1787828737.057092:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f7d7 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787828737.057094:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1787828737.057095:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043270528:192.168.203.147@tcp:38:kworker.0 10000000:01000000:1.0:1787828737.057106:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00000100:00080000:3.0:1787828737.057106:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:1.0:1787828737.057113:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00000008:00100000:3.0:1787828737.057117:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:1.0:1787828737.057120:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f7d7 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828737.057127:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f7d7 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787828737.057137:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043270656/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 10000000:01000000:1.0:1787828737.057148:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787828737.057152:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:CONNECTING 00000100:00100000:2.0:1787828737.057201:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043270656:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1787828737.632862:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828738.656857:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828739.681028:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1787828740.129396:0:2376:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:1.0:1787828740.129489:0:2376:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787828740.129521:0:2376:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1787828740.129596:0:2376:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1787828740.129616:0:2376:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:2.0:1787828740.129852:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828737/real 1787828740] req@ffff9181c5636a00 x1874663043270656/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828762 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828740.129902:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043270656/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828762 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828740.129936:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828740.129946:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828740.129955:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043270656:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828740.129971:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828737/real 1787828740] req@ffff9181c5637480 x1874663043270528/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828762 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828740.129997:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043270528/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828762 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828740.130012:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828740.130019:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828740.130026:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043270528:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828740.130038:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828737/real 1787828740] req@ffff9181c5637100 x1874663043270400/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828762 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828740.130057:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043270400/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828762 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828740.130069:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828740.130114:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828740.130131:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043270400:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787828740.130610:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828740.130634:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828740.130640:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828740.130653:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10808 00000100:00080000:0.0:1787828740.130698:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828740.130713:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787828740.130841:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7480 x1874663043270784/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828740.130876:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828740.130883:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828740.130887:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828740.130894:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10808 00000100:00080000:0.0:1787828740.130902:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828740.130910:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828740.130928:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828740.130954:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7100 x1874663043270912/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828740.130973:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828740.130980:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828740.130983:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828740.130990:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10808 00000100:00080000:0.0:1787828740.130998:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828740.131005:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828740.131017:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828740.131038:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6d80 x1874663043271040/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828740.131474:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043270784:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787828740.131578:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828740.131625:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828740/real 1787828740] req@ffff9181c55a7480 x1874663043270784/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828765 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828740.131652:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7480 x1874663043270784/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828765 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828740.131684:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828740.131691:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828740.131714:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043270784:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828740.131727:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043270912:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828740.131833:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828740.131855:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828740/real 1787828740] req@ffff9181c55a7100 x1874663043270912/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828765 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828740.131885:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7100 x1874663043270912/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828765 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828740.131898:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828740.131903:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828740.131908:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043270912:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828740.131919:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043271040:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828740.131971:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828740.131990:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828740/real 1787828740] req@ffff9181c55a6d80 x1874663043271040/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828765 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828740.132010:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6d80 x1874663043271040/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828765 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828740.132022:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828740.132027:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828740.132033:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043271040:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828740.704899:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828741.728888:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828742.753008:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828743.776924:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828744.800883:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828745.248929:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828745.248941:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828745.248946:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828745.248957:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10811 00000100:00080000:0.0:1787828745.248970:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828745.248978:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787828745.249020:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5f80 x1874663043271168/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828745.249053:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828745.249058:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828745.249060:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828745.249064:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10811 00000100:00080000:0.0:1787828745.249067:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828745.249071:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828745.249082:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828745.249119:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4700 x1874663043271296/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828745.249147:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828745.249153:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828745.249157:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828745.249164:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10811 00000100:00080000:0.0:1787828745.249169:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828745.249176:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787828745.249195:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043271168:192.168.203.147@tcp:250:kworker.0 00000008:00100000:0.0:1787828745.249199:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828745.249232:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5c00 x1874663043271424/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828745.249334:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828745.249422:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828745/real 1787828745] req@ffff9181c55a5f80 x1874663043271168/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828770 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828745.249450:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5f80 x1874663043271168/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828770 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828745.249474:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828745.249480:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828745.249503:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043271168:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828745.249514:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043271296:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828745.249550:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828745.249572:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828745/real 1787828745] req@ffff9181c55a4700 x1874663043271296/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828770 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828745.249589:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4700 x1874663043271296/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828770 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828745.249600:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828745.249605:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828745.249609:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043271296:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828745.249643:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043271424:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828745.249663:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828745.249675:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828745/real 1787828745] req@ffff9181c55a5c00 x1874663043271424/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828770 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828745.249689:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5c00 x1874663043271424/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828770 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828745.249698:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828745.249701:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828745.249705:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043271424:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787828745.824965:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828746.849066:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828747.872892:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828748.897021:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828749.921031:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828750.368889:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828750.368900:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828750.368904:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828750.368915:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10816 00000100:00080000:3.0:1787828750.368925:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828750.368949:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828750.368988:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043271552/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828750.369018:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828750.369022:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828750.369024:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828750.369029:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10816 00000100:00080000:3.0:1787828750.369033:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828750.369038:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828750.369049:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828750.369076:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634e00 x1874663043271680/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828750.369097:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043271552:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787828750.369121:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828750.369127:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828750.369130:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828750.369138:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10816 00000100:00080000:3.0:1787828750.369143:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828750.369149:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828750.369157:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828750.369188:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043271808/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828750.369204:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828750.369249:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828750/real 1787828750] req@ffff9181c5634380 x1874663043271552/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828775 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828750.369264:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043271552/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828775 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828750.369274:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828750.369289:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828750.369302:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043271552:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828750.369330:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043271680:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828750.369359:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828750.369381:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828750/real 1787828750] req@ffff9181c5634e00 x1874663043271680/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828775 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828750.369390:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634e00 x1874663043271680/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828775 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828750.369400:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828750.369403:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828750.369405:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043271680:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828750.369410:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043271808:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828750.369424:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828750.369431:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828750/real 1787828750] req@ffff9181c5635500 x1874663043271808/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828775 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828750.369439:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043271808/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828775 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828750.369444:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828750.369446:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828750.369448:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043271808:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828750.944969:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828751.968900:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828752.993081:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828754.016955:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828755.041001:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828755.488879:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828755.488891:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828755.488895:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828755.488906:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10822 00000100:00080000:3.0:1787828755.488919:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828755.488927:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828755.488970:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635f80 x1874663043271936/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828755.489005:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828755.489010:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828755.489012:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828755.489016:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10822 00000100:00080000:3.0:1787828755.489020:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828755.489024:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828755.489057:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828755.489077:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043272064/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828755.489089:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828755.489092:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828755.489095:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828755.489099:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10822 00000100:00080000:3.0:1787828755.489102:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828755.489107:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828755.489111:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828755.489122:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634e00 x1874663043272192/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828755.489152:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043271936:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787828755.489269:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828755.489327:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828755/real 1787828755] req@ffff9181c5635f80 x1874663043271936/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828780 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828755.489369:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635f80 x1874663043271936/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828780 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828755.489385:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828755.489391:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828755.489416:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043271936:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828755.489426:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043272064:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828755.489457:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828755.489473:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828755/real 1787828755] req@ffff9181c5635500 x1874663043272064/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828780 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828755.489498:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043272064/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828780 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828755.489509:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828755.489514:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828755.489519:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043272064:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828755.489560:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043272192:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828755.489581:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828755.489593:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828755/real 1787828755] req@ffff9181c5634e00 x1874663043272192/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828780 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828755.489607:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634e00 x1874663043272192/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828780 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828755.489616:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828755.489619:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828755.489623:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043272192:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828756.064911:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787828757.088970:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-sptlrpc' failed (-108). 10000000:00020000:1.0:1787828757.088980:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00010000:00010000:3.0:1787828757.089001:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828758.112901:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787828758.560821:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:1.0:1787828758.560827:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff9181c5620000 from 341 10000000:01000000:1.0:1787828758.560832:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:1.0:1787828758.560889:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f7de lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828758.560899:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f7de lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828758.560909:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f7de lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787828758.560941:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215d180 going to sleep for 0 seconds 00000100:00100000:1.0:1787828758.560951:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043272320:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787828758.560964:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f7de lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828758.560970:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f7de lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828758.560976:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f7de lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828758.560982:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f7de lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828758.560988:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f7de lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828758.560995:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b5000 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787828758.560998:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f7de lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828758.561022:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f7de lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787828758.561030:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f7de lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787828758.561034:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:1.0:1787828758.561037:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787828758.561060:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f7de lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828758.561075:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f7de lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787828758.561094:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787828758.561097:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:0.0:1787828758.561166:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828758.561174:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828758.561178:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828758.561187:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10827 00000100:00080000:0.0:1787828758.561198:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828758.561205:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787828758.561239:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4380 x1874663043272448/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828758.561278:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828758.561281:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828758.561282:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828758.561286:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10827 00000100:00080000:0.0:1787828758.561288:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828758.561291:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828758.561300:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828758.561309:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5180 x1874663043272576/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828758.561319:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828758.561324:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:2.0:1787828758.561324:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043272448:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787828758.561326:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828758.561334:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10827 00000100:00080000:0.0:1787828758.561337:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828758.561339:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828758.561342:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828758.561351:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5500 x1874663043272704/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828758.561400:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828758.561435:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828758/real 1787828758] req@ffff9181c55a4380 x1874663043272448/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828783 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828758.561449:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4380 x1874663043272448/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828783 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828758.561463:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828758.561466:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828758.561479:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043272448:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828758.561500:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043272576:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828758.561523:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828758.561533:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828758/real 1787828758] req@ffff9181c55a5180 x1874663043272576/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828783 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828758.561541:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5180 x1874663043272576/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828783 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828758.561547:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828758.561550:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828758.561564:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043272576:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828758.561567:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043272704:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828758.561578:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828758.561584:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828758/real 1787828758] req@ffff9181c55a5500 x1874663043272704/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828783 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828758.561592:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5500 x1874663043272704/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828783 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828758.561597:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828758.561599:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828758.561601:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043272704:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828759.136856:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828760.160944:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828761.185028:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828762.208950:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828763.232950:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828763.680918:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828763.680933:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828763.680939:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828763.680953:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10830 00000100:00080000:3.0:1787828763.680990:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828763.681004:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828763.681056:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043272832/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828763.681100:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828763.681106:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828763.681110:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828763.681117:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10830 00000100:00080000:3.0:1787828763.681123:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828763.681129:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828763.681143:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828763.681167:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043272960/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828763.681191:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828763.681196:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828763.681200:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828763.681216:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10830 00000100:00100000:2.0:1787828763.681217:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043272832:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787828763.681227:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828763.681241:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828763.681249:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828763.681275:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043273088/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828763.681352:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828763.681427:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828763/real 1787828763] req@ffff9181c5634380 x1874663043272832/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828788 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828763.681457:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043272832/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828788 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828763.681473:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828763.681479:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828763.681518:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043272832:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828763.681565:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043272960:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828763.681600:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828763.681617:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828763/real 1787828763] req@ffff9181c5637100 x1874663043272960/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828788 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828763.681631:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043272960/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828788 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828763.681642:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828763.681646:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828763.681650:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043272960:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828763.681657:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043273088:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828763.681677:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828763.681688:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828763/real 1787828763] req@ffff9181c5637480 x1874663043273088/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828788 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828763.681702:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043273088/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828788 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828763.681711:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828763.681714:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828763.681718:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043273088:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1787828764.256933:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828765.281023:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828766.304821:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828767.328860:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828768.352846:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828768.800828:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828768.800837:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828768.800841:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828768.800850:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10835 00000100:00080000:3.0:1787828768.800860:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828768.800866:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828768.800901:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043273216/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828768.800923:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828768.800926:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828768.800927:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828768.800930:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10835 00000100:00080000:3.0:1787828768.800942:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828768.800944:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828768.800952:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828768.800961:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043273344/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828768.800969:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828768.800972:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828768.800973:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828768.800976:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10835 00000100:00080000:3.0:1787828768.800979:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1787828768.800982:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043273216:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787828768.800984:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828768.800993:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828768.800999:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043273472/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828768.801052:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828768.801095:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828768/real 1787828768] req@ffff9181c5636a00 x1874663043273216/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828793 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828768.801108:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043273216/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828793 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828768.801117:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828768.801119:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828768.801132:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043273216:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828768.801137:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043273344:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828768.801152:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828768.801161:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828768/real 1787828768] req@ffff9181c5637800 x1874663043273344/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828793 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828768.801170:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043273344/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828793 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828768.801178:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828768.801181:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828768.801184:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043273344:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828768.801202:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043273472:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828768.801213:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828768.801219:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828768/real 1787828768] req@ffff9181c5636d80 x1874663043273472/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828793 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828768.801226:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043273472/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828793 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828768.801231:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828768.801233:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828768.801235:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043273472:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828769.376881:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828770.400835:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828771.424906:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828772.448891:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828773.472896:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828773.920859:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828773.920870:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828773.920875:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828773.920890:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10840 00000100:00080000:3.0:1787828773.920916:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828773.920924:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828773.920962:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043273600/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828773.920991:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828773.920995:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828773.920997:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828773.921002:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10840 00000100:00080000:3.0:1787828773.921006:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828773.921011:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828773.921020:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828773.921033:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637b80 x1874663043273728/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828773.921044:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043273600:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787828773.921048:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828773.921054:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828773.921056:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828773.921064:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10840 00000100:00080000:3.0:1787828773.921068:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828773.921073:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828773.921078:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828773.921090:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043273856/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828773.921119:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828773.921158:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828773/real 1787828773] req@ffff9181c5635c00 x1874663043273600/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828798 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828773.921178:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043273600/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828798 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828773.921189:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828773.921194:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828773.921210:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043273600:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828773.921234:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043273728:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828773.921263:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828773.921276:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828773/real 1787828773] req@ffff9181c5637b80 x1874663043273728/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828798 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828773.921290:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637b80 x1874663043273728/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828798 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828773.921299:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828773.921303:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828773.921306:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043273728:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828773.921313:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043273856:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828773.921335:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828773.921357:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828773/real 1787828773] req@ffff9181c5634000 x1874663043273856/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828798 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828773.921371:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043273856/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828798 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828773.921380:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828773.921384:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828773.921388:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043273856:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1787828774.496937:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828775.520988:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828776.544870:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828777.568857:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828778.592837:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828779.040838:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828779.040846:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828779.040849:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828779.040858:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10845 00000100:00080000:3.0:1787828779.040880:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828779.040887:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 10000000:01000000:1.0:1787828779.040901:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-cliir' failed (-108). 10000000:00000400:1.0:1787828779.040906:0:159735:0:(mgc_request.c:1900:mgc_process_log()) MGC192.168.203.147@tcp: IR log lustre-cliir failed, not fatal: rc = -108 10000000:01000000:1.0:1787828779.040913:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:1.0:1787828779.040915:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff9181c5620000 from 52 00000100:00100000:3.0:1787828779.040919:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634a80 x1874663043273984/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 10000000:01000000:1.0:1787828779.040920:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00000100:00080000:3.0:1787828779.040945:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828779.040949:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00010000:00010000:1.0:1787828779.040951:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f7e5 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787828779.040951:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828779.040959:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10845 00000100:00080000:3.0:1787828779.040964:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00010000:00010000:1.0:1787828779.040967:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f7e5 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787828779.040969:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828779.040980:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:1.0:1787828779.040982:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f7e5 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787828779.040998:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043274240/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828779.040999:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215d180 going to sleep for 0 seconds 00000100:00100000:2.0:1787828779.041005:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043273984:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787828779.041017:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043274112:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00080000:3.0:1787828779.041028:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828779.041031:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00010000:00010000:1.0:1787828779.041032:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f7e5 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787828779.041033:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828779.041041:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10845 00010000:00010000:1.0:1787828779.041045:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f7e5 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828779.041057:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f7e5 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787828779.041062:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828779.041068:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:1.0:1787828779.041070:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f7e5 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000008:00100000:3.0:1787828779.041074:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000800:00000100:2.0:1787828779.041076:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00010000:00010000:1.0:1787828779.041087:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f7e5 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787828779.041088:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637b80 x1874663043274368/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787828779.041112:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b5000 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787828779.041115:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f7e5 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787828779.041121:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828779/real 1787828779] req@ffff9181c5634a80 x1874663043273984/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828804 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:1.0:1787828779.041139:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f7e5 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787828779.041145:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f7e5 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787828779.041146:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634a80 x1874663043273984/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828804 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 10000000:01000000:1.0:1787828779.041160:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00000100:00080000:2.0:1787828779.041168:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00010000:00010000:1.0:1787828779.041169:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00000100:00080000:2.0:1787828779.041175:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828779.041179:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043273984:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787828779.041180:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f7e5 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828779.041191:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f7e5 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787828779.041198:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043274240:192.168.203.147@tcp:38:kworker.0 10000000:01000000:1.0:1787828779.041215:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787828779.041219:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000800:00000100:2.0:1787828779.041226:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828779.041244:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828779/real 1787828779] req@ffff9181c5634000 x1874663043274240/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828804 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828779.041256:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1787828779.041261:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043274240/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828804 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828779.041262:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828779.041266:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828779.041273:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10850 00000100:00080000:3.0:1787828779.041279:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828779.041281:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828779.041285:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00080000:2.0:1787828779.041288:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828779.041292:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043274240:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828779.041298:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043274368:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828779.041301:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043274496/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828779.041322:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828779.041331:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828779/real 1787828779] req@ffff9181c5637b80 x1874663043274368/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828804 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828779.041339:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637b80 x1874663043274368/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828804 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828779.041349:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828779.041351:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828779.041353:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043274368:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828779.041362:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043274496:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787828779.041372:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828779.041377:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828779/real 1787828779] req@ffff9181c5635c00 x1874663043274496/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828804 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828779.041384:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043274496/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828804 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828779.041389:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828779.041391:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828779.041401:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043274496:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787828779.616843:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828780.640843:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828781.664878:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828782.688850:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828783.712873:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828784.160811:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828784.160821:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828784.160825:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828784.160846:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10850 00000100:00080000:3.0:1787828784.160857:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828784.160865:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828784.160901:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043274624/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828784.160929:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828784.160933:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828784.160935:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828784.160940:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10850 00000100:00080000:3.0:1787828784.160944:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828784.160949:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828784.160959:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828784.160968:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043274624:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828784.160972:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043274752/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828784.160983:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828784.160985:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828784.160987:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828784.160990:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10850 00000100:00080000:3.0:1787828784.160993:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828784.160995:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828784.160999:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828784.161007:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043274880/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828784.161030:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828784.161060:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828784/real 1787828784] req@ffff9181c5636d80 x1874663043274624/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828809 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828784.161074:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043274624/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828809 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828784.161081:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828784.161084:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828784.161095:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043274624:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828784.161115:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043274752:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828784.161133:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828784.161142:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828784/real 1787828784] req@ffff9181c5637800 x1874663043274752/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828809 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828784.161150:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043274752/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828809 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828784.161156:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828784.161158:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828784.161161:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043274752:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828784.161164:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043274880:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828784.161175:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828784.161181:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828784/real 1787828784] req@ffff9181c5636a00 x1874663043274880/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828809 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828784.161188:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043274880/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828809 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828784.161193:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828784.161195:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828784.161197:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043274880:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1787828784.736878:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828785.760970:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828786.784900:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828787.808867:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828788.832913:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828789.280840:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828789.280857:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828789.280862:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828789.280872:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10855 00000100:00080000:3.0:1787828789.280882:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828789.280890:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828789.280928:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043275008/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828789.280960:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828789.280965:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828789.280967:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828789.280973:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10855 00000100:00080000:3.0:1787828789.280978:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828789.280984:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828789.280996:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828789.281017:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043275008:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828789.281018:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043275136/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828789.281038:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828789.281042:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828789.281044:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828789.281047:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10855 00000100:00080000:3.0:1787828789.281052:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828789.281055:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828789.281060:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828789.281073:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043275264/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828789.281097:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828789.281162:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828789/real 1787828789] req@ffff9181c5637480 x1874663043275008/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828814 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828789.281181:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043275008/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828814 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828789.281203:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828789.281208:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828789.281222:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043275008:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828789.281254:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043275136:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828789.281296:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828789.281318:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828789/real 1787828789] req@ffff9181c5637100 x1874663043275136/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828814 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828789.281332:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043275136/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828814 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828789.281342:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828789.281345:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828789.281348:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043275136:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828789.281355:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043275264:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828789.281376:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828789.281387:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828789/real 1787828789] req@ffff9181c5634380 x1874663043275264/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828814 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828789.281416:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043275264/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828814 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828789.281431:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828789.281434:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828789.281437:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043275264:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828789.856892:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828790.880881:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828791.904877:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828792.928914:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828793.952886:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828794.400870:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828794.400879:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828794.400882:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828794.400893:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10861 00000100:00080000:0.0:1787828794.400905:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828794.400914:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787828794.400952:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7800 x1874663043275392/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828794.400975:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828794.400979:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828794.400981:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828794.400986:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10861 00000100:00080000:0.0:1787828794.400991:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828794.400995:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828794.401004:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828794.401017:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4e00 x1874663043275520/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828794.401021:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043275392:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787828794.401029:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828794.401032:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828794.401033:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828794.401037:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10861 00000100:00080000:0.0:1787828794.401041:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828794.401044:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828794.401048:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828794.401058:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5500 x1874663043275648/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828794.401080:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828794.401115:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828794/real 1787828794] req@ffff9181c55a7800 x1874663043275392/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828819 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828794.401129:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7800 x1874663043275392/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828819 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828794.401137:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828794.401139:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828794.401152:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043275392:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828794.401170:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043275520:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828794.401192:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828794.401203:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828794/real 1787828794] req@ffff9181c55a4e00 x1874663043275520/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828819 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828794.401213:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4e00 x1874663043275520/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828819 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828794.401222:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828794.401225:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828794.401228:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043275520:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828794.401236:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043275648:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828794.401257:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828794.401269:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828794/real 1787828794] req@ffff9181c55a5500 x1874663043275648/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828819 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828794.401280:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5500 x1874663043275648/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828819 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828794.401285:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828794.401288:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828794.401290:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043275648:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828794.976907:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828796.000909:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828797.024922:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828798.048898:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828799.072934:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828799.520829:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828799.520838:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828799.520841:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828799.520849:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10866 10000000:01000000:1.0:1787828799.520854:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-client' failed (-108). 00000100:00080000:3.0:1787828799.520858:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 10000000:00020000:1.0:1787828799.520860:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00080000:3.0:1787828799.520867:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 10000000:01000000:1.0:1787828799.520868:0:159735:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:1.0:1787828799.520872:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff9181c5620000 from 7 10000000:01000000:1.0:1787828799.520876:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00000100:00100000:3.0:1787828799.520906:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634e00 x1874663043275904/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787828799.520919:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f7ec lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787828799.520925:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828799.520933:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828799.520936:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00010000:00010000:1.0:1787828799.520939:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f7ec lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787828799.520941:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10866 00000100:00080000:3.0:1787828799.520946:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828799.520953:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:1.0:1787828799.520958:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f7ec lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000008:00100000:3.0:1787828799.520961:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828799.520962:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043275904:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828799.520977:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043276032/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828799.520987:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215d180 going to sleep for 0 seconds 00000100:00080000:3.0:1787828799.520988:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828799.520993:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828799.520995:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1787828799.521000:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043275776:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00080000:3.0:1787828799.521001:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10866 00000100:00080000:3.0:1787828799.521012:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828799.521017:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:1.0:1787828799.521019:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f7ec lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000008:00100000:3.0:1787828799.521021:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828799.521033:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635f80 x1874663043276160/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787828799.521034:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f7ec lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828799.521054:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f7ec lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787828799.521058:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043276032:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828799.521071:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f7ec lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828799.521081:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f7ec lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828799.521089:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4800 already unused, calling callback (ffffffffc1273c20) 00000100:00100000:2.0:1787828799.521092:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043276160:192.168.203.147@tcp:38:kworker.0 10000000:00010000:1.0:1787828799.521095:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f7ec lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828799.521106:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f7ec lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787828799.521116:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f7ec lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787828799.521122:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:1.0:1787828799.521126:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787828799.521135:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f7ec lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828799.521143:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f7ec lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787828799.521164:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787828799.521168:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:CONNECTING 00010000:00010000:1.0:1787828800.096937:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828801.120905:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828802.144913:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1787828802.593098:0:2377:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:0.0:1787828802.593150:0:2377:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1787828802.593172:0:2377:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1787828802.593229:0:2377:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1787828802.593243:0:2377:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:2.0:1787828802.593273:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828799/real 1787828802] req@ffff9181c5635500 x1874663043276032/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828824 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828802.593321:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043276032/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828824 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828802.593353:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828802.593359:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828802.593364:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043276032:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828802.593376:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828799/real 1787828802] req@ffff9181c5635f80 x1874663043276160/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828824 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828802.593391:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635f80 x1874663043276160/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828824 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828802.593400:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828802.593404:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828802.593408:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043276160:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828802.593416:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828799/real 1787828802] req@ffff9181c5634e00 x1874663043275904/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828824 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828802.593430:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634e00 x1874663043275904/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828824 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828802.593439:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828802.593463:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828802.593496:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043275904:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787828802.593543:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828802.593569:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828802.593574:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828802.593585:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10871 00000100:00080000:3.0:1787828802.593593:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828802.593602:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828802.593637:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634e00 x1874663043276288/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828802.593675:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828802.593679:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828802.593682:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828802.593687:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10871 00000100:00080000:3.0:1787828802.593692:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828802.593696:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828802.593723:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828802.593740:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635f80 x1874663043276416/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828802.593782:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828802.593786:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828802.593788:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828802.593793:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10871 00000100:00080000:3.0:1787828802.593797:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828802.593802:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828802.593808:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828802.593910:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043276544/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828802.593940:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043276288:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787828802.594006:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828802.594036:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828802/real 1787828802] req@ffff9181c5634e00 x1874663043276288/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828827 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828802.594066:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634e00 x1874663043276288/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828827 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828802.594075:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828802.594079:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828802.594093:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043276288:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828802.594100:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043276416:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828802.594125:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828802.594139:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828802/real 1787828802] req@ffff9181c5635f80 x1874663043276416/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828827 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828802.594157:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635f80 x1874663043276416/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828827 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828802.594165:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828802.594168:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828802.594171:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043276416:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828802.594179:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043276544:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828802.594200:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828802.594212:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828802/real 1787828802] req@ffff9181c5635500 x1874663043276544/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828827 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828802.594224:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043276544/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828827 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828802.594233:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828802.594236:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828802.594238:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043276544:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828803.168890:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828804.193015:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828805.216897:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828806.240919:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828807.264851:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828807.712840:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828807.712851:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828807.712855:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828807.712865:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10874 00000100:00080000:3.0:1787828807.712877:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828807.712885:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828807.712925:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043276672/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828807.712952:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828807.712955:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828807.712956:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828807.712960:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10874 00000100:00080000:3.0:1787828807.712963:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828807.712966:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828807.712974:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828807.712984:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043276800/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828807.712991:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828807.712994:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828807.712995:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828807.712998:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10874 00000100:00080000:3.0:1787828807.713000:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828807.713005:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787828807.713006:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043276672:192.168.203.147@tcp:250:kworker.0 00000008:00100000:3.0:1787828807.713009:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828807.713019:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043276928/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828807.713072:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828807.713112:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828807/real 1787828807] req@ffff9181c5634380 x1874663043276672/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828832 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828807.713126:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043276672/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828832 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828807.713134:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828807.713137:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828807.713149:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043276672:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828807.713155:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043276800:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828807.713176:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828807.713189:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828807/real 1787828807] req@ffff9181c5637100 x1874663043276800/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828832 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828807.713198:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043276800/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828832 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828807.713204:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828807.713206:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828807.713209:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043276800:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828807.713226:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043276928:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828807.713236:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828807.713243:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828807/real 1787828807] req@ffff9181c5637480 x1874663043276928/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828832 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828807.713255:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043276928/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828832 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828807.713260:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828807.713262:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828807.713264:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043276928:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828808.288870:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828809.313062:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828810.336922:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828811.361018:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828812.384922:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828812.832871:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828812.832886:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828812.832892:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828812.832919:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10879 00000100:00080000:3.0:1787828812.832936:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828812.832948:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828812.833037:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043277056/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828812.833095:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828812.833100:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828812.833103:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828812.833111:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10879 00000100:00080000:3.0:1787828812.833117:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828812.833122:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828812.833160:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828812.833185:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043277184/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828812.833204:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828812.833209:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828812.833213:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1787828812.833223:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043277056:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787828812.833224:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10879 00000100:00080000:3.0:1787828812.833236:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828812.833244:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828812.833251:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828812.833266:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043277312/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828812.833309:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828812.833347:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828812/real 1787828812] req@ffff9181c5636a00 x1874663043277056/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828837 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828812.833361:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043277056/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828837 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828812.833378:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828812.833381:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828812.833396:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043277056:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828812.833417:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043277184:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828812.833435:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828812.833450:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828812/real 1787828812] req@ffff9181c5637800 x1874663043277184/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828837 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828812.833476:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043277184/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828837 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828812.833487:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828812.833490:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828812.833493:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043277184:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828812.833497:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043277312:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828812.833511:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828812.833518:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828812/real 1787828812] req@ffff9181c5636d80 x1874663043277312/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828837 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828812.833525:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043277312/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828837 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828812.833530:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828812.833532:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828812.833534:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043277312:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828813.408960:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828814.432948:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828815.457101:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828816.480938:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828817.504916:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828817.952898:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828817.952909:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828817.952913:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828817.952924:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10884 00000100:00080000:1.0:1787828817.952934:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828817.952942:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787828817.952992:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3100 x1874663043277440/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828817.953015:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828817.953018:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828817.953019:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828817.953022:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10884 00000100:00080000:1.0:1787828817.953025:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828817.953029:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828817.953046:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828817.953056:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1180 x1874663043277568/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828817.953064:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828817.953067:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828817.953070:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1787828817.953071:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043277440:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787828817.953076:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10884 00000100:00080000:1.0:1787828817.953080:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828817.953083:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828817.953086:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828817.953095:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0000 x1874663043277696/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828817.953134:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828817.953166:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828817/real 1787828817] req@ffff9181c35b3100 x1874663043277440/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828842 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828817.953179:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3100 x1874663043277440/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828842 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828817.953187:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828817.953190:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828817.953202:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043277440:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828817.953234:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043277568:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828817.953251:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828817.953259:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828817/real 1787828817] req@ffff9181c35b1180 x1874663043277568/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828842 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828817.953267:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1180 x1874663043277568/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828842 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828817.953273:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828817.953275:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828817.953277:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043277568:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828817.953281:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043277696:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828817.953291:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828817.953296:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828817/real 1787828817] req@ffff9181c35b0000 x1874663043277696/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828842 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828817.953303:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0000 x1874663043277696/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828842 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828817.953308:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828817.953310:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828817.953312:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043277696:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828818.528882:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828819.552909:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787828819.552923:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'params' failed (-108). 10000000:00020000:1.0:1787828819.552932:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:1.0:1787828819.552949:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:1.0:1787828819.552951:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff9181c87b19b8 from 1 10000000:01000000:1.0:1787828819.552955:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00010000:00010000:1.0:1787828819.553002:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f7f3 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828819.553028:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f7f3 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828819.553044:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f7f3 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787828819.553064:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215d180 going to sleep for 0 seconds 00000100:00100000:1.0:1787828819.553086:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043277824:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787828819.553099:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f7f3 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828819.553105:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f7f3 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828819.553111:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f7f3 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828819.553117:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f7f3 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828819.553123:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f7f3 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828819.553129:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4800 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787828819.553132:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f7f3 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828819.553152:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f7f3 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787828819.553159:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f7f3 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787828819.553164:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1787828819.553166:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787828819.553171:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f7f3 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828819.553177:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f7f3 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787828819.553195:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787828819.553197:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:3.0:1787828819.553252:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828819.553258:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828819.553261:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828819.553267:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10889 00000100:00080000:3.0:1787828819.553275:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828819.553281:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828819.553303:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043277952/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828819.553324:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828819.553327:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828819.553329:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828819.553333:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10889 00000100:00080000:3.0:1787828819.553336:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828819.553339:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828819.553346:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828819.553355:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637b80 x1874663043278080/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828819.553363:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828819.553366:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828819.553375:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828819.553379:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10889 00000100:00080000:3.0:1787828819.553383:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828819.553386:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828819.553390:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828819.553399:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043278208/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828819.553404:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043277952:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787828819.553483:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828819.553526:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828819/real 1787828819] req@ffff9181c5635c00 x1874663043277952/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828844 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828819.553541:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043277952/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828844 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828819.553550:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828819.553554:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828819.553573:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043277952:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828819.553599:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043278080:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828819.553635:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828819.553647:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828819/real 1787828819] req@ffff9181c5637b80 x1874663043278080/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828844 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828819.553658:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637b80 x1874663043278080/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828844 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828819.553666:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828819.553668:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828819.553671:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043278080:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828819.553688:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043278208:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828819.553702:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828819.553710:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828819/real 1787828819] req@ffff9181c5634000 x1874663043278208/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828844 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828819.553719:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043278208/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828844 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828819.553737:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828819.553741:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828819.553774:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043278208:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828820.576877:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828821.600904:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828822.624878:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828823.648874:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828824.608867:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828824.608878:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828824.608882:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828824.608892:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10891 00000100:00080000:3.0:1787828824.608903:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828824.608911:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828824.608947:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634a80 x1874663043278336/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828824.608979:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828824.608983:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828824.608985:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828824.608990:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10891 00000100:00080000:3.0:1787828824.608994:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828824.608998:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828824.609008:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828824.609025:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043278464/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828824.609038:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1787828824.609042:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043278336:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787828824.609043:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828824.609048:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828824.609057:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10891 00000100:00080000:3.0:1787828824.609061:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828824.609066:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828824.609072:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828824.609086:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637b80 x1874663043278592/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828824.609120:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828824.609160:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828824/real 1787828824] req@ffff9181c5634a80 x1874663043278336/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828849 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828824.609180:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634a80 x1874663043278336/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828849 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828824.609190:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828824.609195:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828824.609210:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043278336:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828824.609235:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043278464:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828824.609274:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828824.609291:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828824/real 1787828824] req@ffff9181c5634000 x1874663043278464/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828849 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828824.609305:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043278464/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828849 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828824.609313:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828824.609317:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828824.609320:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043278464:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828824.609328:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043278592:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828824.609350:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828824.609363:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828824/real 1787828824] req@ffff9181c5637b80 x1874663043278592/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828849 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828824.609376:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637b80 x1874663043278592/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828849 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828824.609385:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828824.609389:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828824.609392:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043278592:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828824.672891:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828825.696890:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828826.720894:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828827.744895:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828828.768914:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828829.728799:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828829.728808:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828829.728811:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828829.728819:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10896 00000100:00080000:3.0:1787828829.728828:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828829.728834:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828829.728871:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043278720/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828829.728910:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828829.728914:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828829.728916:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828829.728921:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10896 00000100:00080000:3.0:1787828829.728924:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828829.728929:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828829.728936:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828829.728953:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043278720:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828829.728953:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043278848/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828829.728973:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828829.728974:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828829.728975:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828829.728979:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10896 00000100:00080000:3.0:1787828829.728981:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828829.728984:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828829.728987:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828829.728996:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043278976/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828829.729028:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828829.729060:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828829/real 1787828829] req@ffff9181c5635c00 x1874663043278720/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828854 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828829.729073:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043278720/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828854 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828829.729082:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828829.729086:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828829.729101:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043278720:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828829.729135:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043278848:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828829.729156:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828829.729166:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828829/real 1787828829] req@ffff9181c5636d80 x1874663043278848/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828854 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828829.729185:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043278848/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828854 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828829.729190:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828829.729193:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828829.729195:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043278848:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828829.729199:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043278976:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828829.729210:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828829.729216:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828829/real 1787828829] req@ffff9181c5637800 x1874663043278976/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828854 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828829.729224:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043278976/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828854 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828829.729236:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828829.729238:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828829.729240:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043278976:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828829.792927:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828830.816889:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828831.840991:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828832.864863:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828833.888864:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828834.848839:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828834.848848:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828834.848851:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828834.848860:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10901 00000100:00080000:0.0:1787828834.848869:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828834.848875:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787828834.848907:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5180 x1874663043279104/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828834.848929:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828834.848937:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828834.848938:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828834.848941:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10901 00000100:00080000:0.0:1787828834.848943:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828834.848947:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828834.848954:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828834.848965:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4380 x1874663043279232/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828834.848974:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828834.848977:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:2.0:1787828834.848977:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043279104:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787828834.848981:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828834.848988:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10901 00000100:00080000:0.0:1787828834.848990:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828834.848998:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828834.849002:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828834.849012:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5c00 x1874663043279360/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828834.849040:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828834.849071:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828834/real 1787828834] req@ffff9181c55a5180 x1874663043279104/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828859 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828834.849083:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5180 x1874663043279104/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828859 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828834.849096:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828834.849099:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828834.849110:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043279104:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828834.849129:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043279232:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828834.849146:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828834.849156:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828834/real 1787828834] req@ffff9181c55a4380 x1874663043279232/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828859 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828834.849165:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4380 x1874663043279232/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828859 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828834.849170:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828834.849173:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828834.849175:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043279232:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828834.849178:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043279360:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828834.849189:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828834.849195:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828834/real 1787828834] req@ffff9181c55a5c00 x1874663043279360/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828859 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828834.849202:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5c00 x1874663043279360/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828859 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828834.849208:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828834.849209:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828834.849211:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043279360:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828834.912859:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828835.936840:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828836.960854:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828837.984851:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828839.008927:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828839.968877:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828839.968888:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828839.968893:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828839.968908:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10906 00000100:00080000:3.0:1787828839.968919:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828839.968927:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 10000000:01000000:1.0:1787828839.968959:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-sptlrpc' failed (-108). 10000000:00020000:1.0:1787828839.968966:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00100000:3.0:1787828839.968969:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043279488/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828839.968997:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828839.969001:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828839.969003:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828839.969013:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10906 00000100:00080000:3.0:1787828839.969017:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828839.969023:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828839.969032:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828839.969043:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043279616/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828839.969051:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828839.969054:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828839.969056:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828839.969062:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10906 00000100:00080000:3.0:1787828839.969066:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1787828839.969068:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043279488:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787828839.969071:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828839.969075:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828839.969089:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043279744/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828839.969151:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828839.969183:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828839/real 1787828839] req@ffff9181c5636a00 x1874663043279488/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828864 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828839.969197:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043279488/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828864 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828839.969206:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828839.969210:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828839.969213:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043279488:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828839.969235:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043279616:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828839.969274:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828839.969288:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828839/real 1787828839] req@ffff9181c5637480 x1874663043279616/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828864 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828839.969297:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043279616/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828864 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828839.969304:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828839.969306:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828839.969308:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043279616:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828839.969313:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043279744:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828839.969329:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828839.969337:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828839/real 1787828839] req@ffff9181c5637100 x1874663043279744/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828864 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828839.969345:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043279744/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828864 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828839.969350:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828839.969352:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828839.969354:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043279744:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828840.032884:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828841.056969:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787828841.568839:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:1.0:1787828841.568845:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff9181c5620000 from 341 10000000:01000000:1.0:1787828841.568849:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:1.0:1787828841.568895:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f7fa lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828841.568908:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f7fa lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828841.568921:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f7fa lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787828841.568945:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215d180 going to sleep for 0 seconds 00000100:00100000:1.0:1787828841.568955:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043279872:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787828841.568967:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f7fa lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828841.568974:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f7fa lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828841.568979:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f7fa lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828841.568983:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f7fa lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828841.568989:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f7fa lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828841.568993:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4800 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787828841.568996:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f7fa lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828841.569002:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f7fa lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787828841.569007:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f7fa lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787828841.569011:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:1.0:1787828841.569014:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787828841.569018:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f7fa lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828841.569023:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f7fa lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787828841.569044:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787828841.569048:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:3.0:1787828841.569114:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828841.569119:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828841.569123:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828841.569130:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10911 00000100:00080000:3.0:1787828841.569139:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828841.569145:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828841.569163:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043280000/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828841.569182:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828841.569185:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828841.569186:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828841.569190:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10911 00000100:00080000:3.0:1787828841.569192:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828841.569194:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828841.569202:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828841.569209:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043280128/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828841.569216:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828841.569218:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828841.569219:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828841.569221:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10911 00000100:00080000:3.0:1787828841.569223:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828841.569225:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828841.569228:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828841.569233:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635f80 x1874663043280256/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828841.569280:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043280000:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787828841.569353:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828841.569396:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828841/real 1787828841] req@ffff9181c5634380 x1874663043280000/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828866 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828841.569409:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043280000/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828866 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828841.569417:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828841.569420:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828841.569454:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043280000:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828841.569483:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043280128:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828841.569502:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828841.569512:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828841/real 1787828841] req@ffff9181c5635500 x1874663043280128/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828866 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828841.569520:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043280128/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828866 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828841.569530:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828841.569533:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828841.569535:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043280128:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828841.569539:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043280256:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828841.569549:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828841.569555:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828841/real 1787828841] req@ffff9181c5635f80 x1874663043280256/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828866 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828841.569562:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635f80 x1874663043280256/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828866 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828841.569567:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828841.569569:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828841.569571:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043280256:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828842.080828:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828843.104909:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828844.128941:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828845.152964:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828846.176914:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828846.624850:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828846.624862:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828846.624867:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828846.624878:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10913 00000100:00080000:3.0:1787828846.624891:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828846.624899:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828846.624934:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634e00 x1874663043280384/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828846.624965:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828846.624970:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828846.624972:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828846.624978:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10913 00000100:00080000:3.0:1787828846.624982:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828846.625003:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828846.625012:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828846.625030:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635f80 x1874663043280512/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828846.625043:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043280384:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787828846.625043:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828846.625050:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828846.625053:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828846.625058:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10913 00000100:00080000:3.0:1787828846.625061:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828846.625064:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828846.625068:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828846.625079:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043280640/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828846.625132:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828846.625172:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828846/real 1787828846] req@ffff9181c5634e00 x1874663043280384/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828871 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828846.625192:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634e00 x1874663043280384/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828871 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828846.625203:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828846.625208:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828846.625226:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043280384:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828846.625265:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043280512:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828846.625295:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828846.625311:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828846/real 1787828846] req@ffff9181c5635f80 x1874663043280512/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828871 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828846.625325:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635f80 x1874663043280512/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828871 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828846.625334:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828846.625338:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828846.625342:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043280512:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828846.625349:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043280640:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828846.625372:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828846.625385:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828846/real 1787828846] req@ffff9181c5635500 x1874663043280640/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828871 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828846.625416:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043280640/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828871 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828846.625424:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828846.625428:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828846.625432:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043280640:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828847.200910:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828848.224925:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828849.248969:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828850.272905:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828851.296915:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828851.744809:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828851.744818:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828851.744821:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828851.744829:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10918 00000100:00080000:3.0:1787828851.744839:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828851.744845:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828851.744875:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043280768/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828851.744896:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828851.744900:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828851.744901:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828851.744904:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10918 00000100:00080000:3.0:1787828851.744906:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828851.744909:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828851.744915:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828851.744923:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043280896/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828851.744930:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828851.744932:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828851.744933:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828851.744935:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10918 00000100:00080000:3.0:1787828851.744938:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828851.744941:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828851.744944:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828851.744949:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043281024/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828851.744980:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043280768:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787828851.745036:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828851.745068:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828851/real 1787828851] req@ffff9181c5634380 x1874663043280768/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828876 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828851.745081:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043280768/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828876 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828851.745089:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828851.745092:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828851.745105:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043280768:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828851.745110:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043280896:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828851.745127:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828851.745137:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828851/real 1787828851] req@ffff9181c5637100 x1874663043280896/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828876 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828851.745148:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043280896/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828876 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828851.745154:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828851.745156:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828851.745158:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043280896:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828851.745162:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043281024:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828851.745172:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828851.745177:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828851/real 1787828851] req@ffff9181c5637480 x1874663043281024/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828876 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828851.745185:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043281024/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828876 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828851.745195:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828851.745197:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828851.745199:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043281024:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828852.320856:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828853.344929:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828854.368866:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828855.392902:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828856.416861:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828856.864855:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828856.864866:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828856.864870:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828856.864880:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10923 00000100:00080000:3.0:1787828856.864892:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828856.864900:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828856.864938:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043281152/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828856.864967:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828856.864971:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828856.864973:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828856.864978:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10923 00000100:00080000:3.0:1787828856.864982:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828856.864988:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828856.864997:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828856.865014:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043281280/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828856.865016:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043281152:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787828856.865033:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828856.865036:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828856.865049:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828856.865054:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10923 00000100:00080000:3.0:1787828856.865066:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828856.865072:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:2.0:1787828856.865077:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:3.0:1787828856.865078:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828856.865095:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043281408/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828856.865113:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828856/real 1787828856] req@ffff9181c5636a00 x1874663043281152/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828881 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828856.865127:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043281152/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828881 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828856.865146:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828856.865150:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828856.865165:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043281152:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828856.865185:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043281280:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828856.865214:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828856.865225:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828856/real 1787828856] req@ffff9181c5637800 x1874663043281280/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828881 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828856.865234:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043281280/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828881 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828856.865239:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828856.865242:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828856.865244:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043281280:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828856.865248:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043281408:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828856.865259:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828856.865265:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828856/real 1787828856] req@ffff9181c5636d80 x1874663043281408/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828881 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828856.865273:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043281408/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828881 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828856.865278:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828856.865280:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828856.865282:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043281408:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828857.440950:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828858.465006:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828859.488906:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828860.513017:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828861.536930:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828861.984841:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 10000000:01000000:1.0:1787828861.984844:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-cliir' failed (-108). 10000000:00000400:1.0:1787828861.984852:0:159735:0:(mgc_request.c:1900:mgc_process_log()) MGC192.168.203.147@tcp: IR log lustre-cliir failed, not fatal: rc = -108 00000100:00080000:0.0:1787828861.984854:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828861.984859:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828861.984871:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10928 00000100:00080000:0.0:1787828861.984882:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828861.984890:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787828861.984924:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4700 x1874663043281536/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828861.984952:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828861.984957:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828861.984965:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828861.984970:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10928 00000100:00080000:0.0:1787828861.984974:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828861.984980:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828861.984989:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828861.985005:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5f80 x1874663043281664/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828861.985016:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828861.985020:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828861.985022:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828861.985027:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10928 00000100:00080000:0.0:1787828861.985031:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828861.985038:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787828861.985045:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043281536:192.168.203.147@tcp:250:kworker.0 00000008:00100000:0.0:1787828861.985047:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828861.985069:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6d80 x1874663043281792/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828861.985143:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828861.985187:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828861/real 1787828861] req@ffff9181c55a4700 x1874663043281536/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828886 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828861.985207:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4700 x1874663043281536/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828886 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828861.985219:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828861.985224:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828861.985228:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043281536:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828861.985236:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043281664:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828861.985262:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828861.985277:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828861/real 1787828861] req@ffff9181c55a5f80 x1874663043281664/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828886 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828861.985310:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5f80 x1874663043281664/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828886 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828861.985320:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828861.985324:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828861.985327:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043281664:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828861.985356:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043281792:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828861.985381:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828861.985394:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828861/real 1787828861] req@ffff9181c55a6d80 x1874663043281792/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828886 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828861.985408:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6d80 x1874663043281792/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828886 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828861.985418:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828861.985423:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828861.985427:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043281792:192.168.203.147@tcp:38:kworker.0 10000000:01000000:1.0:1787828861.993363:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:1.0:1787828861.993368:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff9181c5620000 from 52 10000000:01000000:1.0:1787828861.993373:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:1.0:1787828861.993412:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f801 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828861.993426:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f801 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828861.993439:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f801 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787828861.993459:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215d180 going to sleep for 0 seconds 00000100:00100000:1.0:1787828861.993470:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043281920:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787828861.993485:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f801 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828861.993495:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f801 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828861.993503:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f801 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828861.993512:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f801 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828861.993521:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f801 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828861.993528:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4800 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787828861.993533:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f801 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828861.993546:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f801 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787828861.993554:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f801 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787828861.993561:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1787828861.993565:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787828861.993572:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f801 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828861.993580:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f801 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787828861.993604:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787828861.993608:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:0.0:1787828861.993656:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828861.993661:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828861.993663:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828861.993667:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10933 00000100:00080000:0.0:1787828861.993672:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828861.993676:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787828861.993697:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7100 x1874663043282048/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828861.993811:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043282048:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787828861.993863:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828861.993893:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828861/real 1787828861] req@ffff9181c55a7100 x1874663043282048/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828886 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828861.993910:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7100 x1874663043282048/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828886 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828861.993927:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828861.993932:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828861.993948:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043282048:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1787828862.561010:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828863.584980:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828864.608899:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828865.632986:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828866.657031:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828867.104932:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828867.104944:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828867.104949:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828867.104962:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10933 00000100:00080000:0.0:1787828867.104977:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828867.104986:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787828867.105042:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7480 x1874663043282176/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828867.105067:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828867.105070:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828867.105072:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828867.105076:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10933 00000100:00080000:0.0:1787828867.105080:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828867.105085:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828867.105095:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828867.105099:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043282176:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787828867.105109:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5880 x1874663043282304/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828867.105134:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828867.105138:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828867.105140:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828867.105145:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10933 00000100:00080000:0.0:1787828867.105149:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828867.105153:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828867.105160:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828867.105175:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6680 x1874663043282432/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828867.105300:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043282304:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828867.105331:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043282432:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828867.680954:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828868.704925:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828869.728862:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1787828870.177194:0:2378:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:2.0:1787828870.177275:0:2378:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1787828870.177298:0:2378:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:2.0:1787828870.177356:0:2378:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:2.0:1787828870.177368:0:2378:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:3.0:1787828870.177468:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828867/real 1787828870] req@ffff9181c55a5880 x1874663043282304/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828892 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828870.177502:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5880 x1874663043282304/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828892 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828870.177518:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828870.177525:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828870.177531:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043282304:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828870.177541:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828867/real 1787828870] req@ffff9181c55a6680 x1874663043282432/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828892 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828870.177556:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6680 x1874663043282432/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828892 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828870.177565:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828870.177569:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828870.177573:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043282432:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828870.177582:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828867/real 1787828870] req@ffff9181c55a7480 x1874663043282176/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828892 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787828870.177598:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7480 x1874663043282176/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828892 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828870.177606:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828870.177611:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828870.177640:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043282176:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1787828870.752915:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828871.776926:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828872.224888:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828872.224897:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828872.224899:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828872.224920:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10938 00000100:00080000:3.0:1787828872.224929:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828872.224936:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828872.224965:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043282560/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828872.224999:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828872.225003:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828872.225005:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828872.225010:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10938 00000100:00080000:3.0:1787828872.225014:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828872.225018:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828872.225028:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828872.225044:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637b80 x1874663043282688/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828872.225056:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1787828872.225059:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043282560:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787828872.225060:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828872.225062:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828872.225070:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10938 00000100:00080000:3.0:1787828872.225074:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828872.225076:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828872.225080:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828872.225094:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043282816/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828872.225143:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828872.225177:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828872/real 1787828872] req@ffff9181c5635c00 x1874663043282560/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828897 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828872.225195:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043282560/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828897 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828872.225231:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828872.225236:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828872.225258:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043282560:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828872.225284:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043282688:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828872.225313:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828872.225329:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828872/real 1787828872] req@ffff9181c5637b80 x1874663043282688/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828897 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828872.225353:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637b80 x1874663043282688/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828897 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828872.225363:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828872.225368:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828872.225373:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043282688:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828872.225382:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043282816:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828872.225403:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828872.225415:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828872/real 1787828872] req@ffff9181c5634000 x1874663043282816/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828897 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828872.225427:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043282816/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828897 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828872.225435:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828872.225438:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828872.225441:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043282816:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828872.800878:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828873.824887:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828874.848875:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828875.872884:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828876.896910:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828877.344864:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828877.344878:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828877.344885:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828877.344896:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10943 00000100:00080000:0.0:1787828877.344908:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828877.344916:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787828877.344957:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7b80 x1874663043282944/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828877.344992:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828877.344996:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828877.344998:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828877.345002:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10943 00000100:00080000:0.0:1787828877.345005:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828877.345009:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828877.345018:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828877.345032:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7480 x1874663043283072/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828877.345042:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828877.345045:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828877.345046:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828877.345050:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10943 00000100:00080000:0.0:1787828877.345052:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828877.345055:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828877.345059:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828877.345069:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6680 x1874663043283200/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828877.345100:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043282944:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787828877.345173:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828877.345253:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828877/real 1787828877] req@ffff9181c55a7b80 x1874663043282944/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828902 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828877.345279:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7b80 x1874663043282944/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828902 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828877.345289:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828877.345293:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828877.345310:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043282944:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828877.345317:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043283072:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828877.345343:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828877.345357:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828877/real 1787828877] req@ffff9181c55a7480 x1874663043283072/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828902 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828877.345369:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7480 x1874663043283072/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828902 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828877.345377:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828877.345381:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828877.345384:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043283072:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828877.345406:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043283200:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828877.345421:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828877.345430:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828877/real 1787828877] req@ffff9181c55a6680 x1874663043283200/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828902 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828877.345447:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6680 x1874663043283200/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828902 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828877.345466:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828877.345471:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828877.345476:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043283200:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828877.920890:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828878.944875:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828879.968912:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828880.992864:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828882.016903:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828882.464852:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828882.464862:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828882.464866:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828882.464876:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10949 00000100:00080000:0.0:1787828882.464886:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828882.464892:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 10000000:01000000:1.0:1787828882.464912:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-client' failed (-108). 10000000:00020000:1.0:1787828882.464918:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:1.0:1787828882.464923:0:159735:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:1.0:1787828882.464925:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff9181c5620000 from 7 00000100:00100000:0.0:1787828882.464927:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5880 x1874663043283328/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 10000000:01000000:1.0:1787828882.464929:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00000100:00080000:0.0:1787828882.464969:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828882.464973:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828882.464974:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828882.464978:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10949 00010000:00010000:1.0:1787828882.464981:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f808 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787828882.464982:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828882.464990:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:1.0:1787828882.464998:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f808 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000008:00100000:0.0:1787828882.465001:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:1.0:1787828882.465030:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f808 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787828882.465036:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7100 x1874663043283584/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828882.465048:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1787828882.465050:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215db40 going to sleep for 0 seconds 00000100:00080000:0.0:1787828882.465052:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828882.465056:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828882.465062:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10949 00000100:00100000:1.0:1787828882.465063:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043283456:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00080000:0.0:1787828882.465067:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1787828882.465070:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043283328:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787828882.465076:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:1.0:1787828882.465081:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f808 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000008:00100000:0.0:1787828882.465083:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:1.0:1787828882.465095:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f808 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787828882.465100:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6d80 x1874663043283712/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787828882.465103:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f808 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828882.465124:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f808 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828882.465133:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f808 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828882.465140:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4800 already unused, calling callback (ffffffffc1273c20) 00000800:00000100:2.0:1787828882.465141:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 10000000:00010000:1.0:1787828882.465160:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f808 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828882.465173:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f808 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787828882.465179:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f808 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787828882.465184:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:1.0:1787828882.465205:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787828882.465212:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f808 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828882.465219:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f808 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787828882.465226:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828882/real 1787828882] req@ffff9181c55a5880 x1874663043283328/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828907 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 10000000:01000000:1.0:1787828882.465244:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00100000:2.0:1787828882.465249:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5880 x1874663043283328/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828907 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828882.465250:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:CONNECTING 00000100:00080000:2.0:1787828882.465279:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828882.465294:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828882.465301:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043283328:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828882.465308:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043283584:192.168.203.147@tcp:38:kworker.0 00000100:00080000:0.0:1787828882.465341:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828882.465346:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828882.465348:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000800:00000100:2.0:1787828882.465354:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:0.0:1787828882.465358:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10954 00000100:00080000:0.0:1787828882.465364:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828882.465368:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787828882.465373:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828882/real 1787828882] req@ffff9181c55a7100 x1874663043283584/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828907 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828882.465384:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5f80 x1874663043283840/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828882.465389:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7100 x1874663043283584/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828907 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828882.465411:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828882.465414:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828882.465418:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043283584:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828882.465437:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043283712:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828882.465453:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828882.465461:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828882/real 1787828882] req@ffff9181c55a6d80 x1874663043283712/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828907 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828882.465470:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6d80 x1874663043283712/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828907 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828882.465476:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828882.465478:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828882.465481:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043283712:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828882.465485:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043283840:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787828882.465496:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828882.465503:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828882/real 1787828882] req@ffff9181c55a5f80 x1874663043283840/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828907 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828882.465511:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5f80 x1874663043283840/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828907 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828882.465518:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828882.465519:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828882.465530:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043283840:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1787828883.040877:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828884.064931:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828885.088885:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828886.112946:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828887.136969:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828887.584900:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828887.584914:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828887.584919:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828887.584941:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10954 00000100:00080000:0.0:1787828887.584956:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828887.584966:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787828887.585021:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4700 x1874663043283968/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828887.585056:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828887.585061:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828887.585063:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828887.585068:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10954 00000100:00080000:0.0:1787828887.585073:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828887.585077:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828887.585088:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828887.585131:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5c00 x1874663043284096/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828887.585146:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828887.585150:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828887.585152:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828887.585157:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10954 00000100:00080000:0.0:1787828887.585162:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828887.585167:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828887.585176:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828887.585187:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043283968:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787828887.585196:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4380 x1874663043284224/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828887.585273:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828887.585324:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828887/real 1787828887] req@ffff9181c55a4700 x1874663043283968/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828912 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828887.585345:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4700 x1874663043283968/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828912 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828887.585354:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828887.585358:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828887.585376:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043283968:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828887.585383:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043284096:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828887.585407:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828887.585420:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828887/real 1787828887] req@ffff9181c55a5c00 x1874663043284096/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828912 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828887.585430:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5c00 x1874663043284096/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828912 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828887.585437:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828887.585441:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828887.585443:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043284096:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828887.585465:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043284224:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828887.585478:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828887.585485:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828887/real 1787828887] req@ffff9181c55a4380 x1874663043284224/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828912 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828887.585494:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4380 x1874663043284224/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828912 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828887.585504:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828887.585507:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828887.585509:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043284224:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828888.160920:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787828889.184821:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828890.208873:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828891.232915:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828892.256967:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828892.704859:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828892.704881:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828892.704900:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828892.704912:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10959 00000100:00080000:3.0:1787828892.704923:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828892.704932:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828892.704990:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634a80 x1874663043284352/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828892.705026:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828892.705032:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828892.705035:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828892.705041:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10959 00000100:00080000:3.0:1787828892.705046:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828892.705053:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828892.705064:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828892.705112:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043284352:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828892.705120:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043284480/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828892.705134:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828892.705138:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828892.705141:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828892.705145:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10959 00000100:00080000:3.0:1787828892.705163:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828892.705169:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:2.0:1787828892.705175:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:3.0:1787828892.705178:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828892.705195:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637b80 x1874663043284608/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828892.705207:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828892/real 1787828892] req@ffff9181c5634a80 x1874663043284352/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828917 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828892.705240:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634a80 x1874663043284352/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828917 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828892.705249:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828892.705252:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828892.705264:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043284352:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828892.705283:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043284480:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828892.705303:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828892.705318:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828892/real 1787828892] req@ffff9181c5634000 x1874663043284480/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828917 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828892.705328:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043284480/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828917 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828892.705338:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828892.705342:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828892.705347:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043284480:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828892.705354:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043284608:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828892.705377:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828892.705390:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828892/real 1787828892] req@ffff9181c5637b80 x1874663043284608/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828917 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828892.705401:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637b80 x1874663043284608/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828917 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828892.705407:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828892.705409:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828892.705412:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043284608:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787828893.280957:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828894.304923:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828895.328887:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828896.352900:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828897.376939:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828897.824951:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828897.824962:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828897.824966:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828897.824977:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10964 00000100:00080000:0.0:1787828897.824989:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828897.824997:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787828897.825061:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5180 x1874663043284736/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828897.825102:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828897.825108:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828897.825109:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828897.825114:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10964 00000100:00080000:0.0:1787828897.825118:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828897.825132:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828897.825149:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828897.825172:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5500 x1874663043284864/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828897.825175:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043284736:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787828897.825198:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828897.825201:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828897.825202:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828897.825209:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10964 00000100:00080000:0.0:1787828897.825214:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828897.825219:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828897.825223:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828897.825235:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4e00 x1874663043284992/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828897.825280:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828897.825331:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828897/real 1787828897] req@ffff9181c55a5180 x1874663043284736/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828922 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828897.825352:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5180 x1874663043284736/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828922 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828897.825362:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828897.825366:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828897.825382:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043284736:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828897.825409:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043284864:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828897.825440:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828897.825455:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828897/real 1787828897] req@ffff9181c55a5500 x1874663043284864/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828922 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828897.825466:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5500 x1874663043284864/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828922 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828897.825473:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828897.825477:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828897.825479:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043284864:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828897.825485:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043284992:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828897.825500:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828897.825508:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828897/real 1787828897] req@ffff9181c55a4e00 x1874663043284992/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828922 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828897.825518:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4e00 x1874663043284992/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828922 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828897.825524:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828897.825527:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828897.825529:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043284992:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828898.400929:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828899.424903:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828900.448852:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828901.472880:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828902.496874:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828902.944896:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 10000000:01000000:1.0:1787828902.944899:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'params' failed (-108). 10000000:00020000:1.0:1787828902.944906:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00080000:3.0:1787828902.944910:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 10000000:01000000:1.0:1787828902.944912:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:1.0:1787828902.944915:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff9181c87b19b8 from 1 00000100:00080000:3.0:1787828902.944916:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 10000000:01000000:1.0:1787828902.944919:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00000100:00080000:3.0:1787828902.944933:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10969 00000100:00080000:3.0:1787828902.944946:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828902.944954:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828902.944986:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043285248/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787828902.944992:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f80f lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828902.945019:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f80f lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787828902.945028:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:1.0:1787828902.945030:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f80f lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787828902.945033:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828902.945037:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828902.945046:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10969 00000100:00080000:3.0:1787828902.945050:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828902.945056:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787828902.945057:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215db40 going to sleep for 0 seconds 00000008:00100000:3.0:1787828902.945067:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828902.945068:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043285120:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00100000:2.0:1787828902.945071:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043285248:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787828902.945086:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043285376/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787828902.945087:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f80f lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828902.945108:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f80f lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787828902.945114:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:1.0:1787828902.945117:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f80f lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787828902.945120:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828902.945125:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00010000:00010000:1.0:1787828902.945128:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f80f lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828902.945135:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f80f lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000800:00000100:2.0:1787828902.945136:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1787828902.945139:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10969 00000100:00080000:3.0:1787828902.945147:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00010000:00010000:1.0:1787828902.945149:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4800 already unused, calling callback (ffffffffc1273c20) 00000100:00080000:3.0:1787828902.945155:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 10000000:00010000:1.0:1787828902.945156:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f80f lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000008:00100000:3.0:1787828902.945162:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787828902.945174:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828902/real 1787828902] req@ffff9181c5635c00 x1874663043285248/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828927 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:1.0:1787828902.945177:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f80f lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787828902.945180:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043285504/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 10000000:00010000:1.0:1787828902.945202:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f80f lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787828902.945226:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1787828902.945232:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00000100:00100000:2.0:1787828902.945241:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043285248/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828927 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:1.0:1787828902.945242:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f80f lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828902.945263:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f80f lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787828902.945270:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828902.945275:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828902.945280:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043285248:192.168.203.147@tcp:250:kworker.0 10000000:01000000:1.0:1787828902.945287:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787828902.945291:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00100000:2.0:1787828902.945300:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043285376:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828902.945320:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828902.945334:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828902/real 1787828902] req@ffff9181c5636d80 x1874663043285376/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828927 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828902.945337:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828902.945346:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828902.945351:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1787828902.945356:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043285376/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828927 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828902.945359:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10974 00000100:00080000:3.0:1787828902.945367:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787828902.945372:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787828902.945374:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00080000:2.0:1787828902.945379:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828902.945383:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043285376:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828902.945390:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043285504:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787828902.945394:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043285632/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828902.945415:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828902.945427:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828902/real 1787828902] req@ffff9181c5637800 x1874663043285504/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828927 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828902.945436:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043285504/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828927 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828902.945447:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828902.945450:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828902.945452:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043285504:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828902.945461:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043285632:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787828902.945472:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828902.945478:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828902/real 1787828902] req@ffff9181c5636a00 x1874663043285632/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828927 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828902.945503:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043285632/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828927 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828902.945508:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828902.945510:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828902.945521:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043285632:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1787828903.520840:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828904.544846:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828905.568977:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828906.592963:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828907.616911:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828908.064892:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828908.064900:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828908.064904:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828908.064913:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10974 00000100:00080000:1.0:1787828908.064922:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828908.064928:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787828908.064963:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0e00 x1874663043285760/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828908.065005:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828908.065008:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828908.065010:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828908.065014:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10974 00000100:00080000:1.0:1787828908.065016:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828908.065020:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828908.065028:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828908.065039:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2680 x1874663043285888/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828908.065049:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828908.065056:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828908.065058:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1787828908.065058:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043285760:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787828908.065073:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10974 00000100:00080000:1.0:1787828908.065076:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828908.065079:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828908.065082:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828908.065092:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3480 x1874663043286016/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828908.065147:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828908.065199:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828908/real 1787828908] req@ffff9181c35b0e00 x1874663043285760/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828933 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828908.065220:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0e00 x1874663043285760/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828933 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828908.065230:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828908.065234:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828908.065247:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043285760:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828908.065279:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043285888:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828908.065300:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828908.065311:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828908/real 1787828908] req@ffff9181c35b2680 x1874663043285888/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828933 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828908.065319:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2680 x1874663043285888/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828933 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828908.065325:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828908.065327:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828908.065330:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043285888:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828908.065334:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043286016:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828908.065345:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828908.065352:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828908/real 1787828908] req@ffff9181c35b3480 x1874663043286016/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828933 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828908.065359:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3480 x1874663043286016/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828933 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828908.065364:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828908.065366:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828908.065368:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043286016:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828908.640938:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828909.664936:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828910.689023:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828911.712849:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828912.736939:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828913.184901:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828913.184912:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828913.184916:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828913.184926:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10979 00000100:00080000:3.0:1787828913.184948:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828913.184955:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828913.185011:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043286144/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828913.185036:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828913.185039:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828913.185040:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828913.185043:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10979 00000100:00080000:3.0:1787828913.185046:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828913.185049:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828913.185057:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828913.185066:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043286272/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828913.185074:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828913.185077:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828913.185078:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828913.185082:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10979 00000100:00080000:3.0:1787828913.185084:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828913.185086:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828913.185089:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828913.185097:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043286400/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828913.185109:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043286144:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787828913.185177:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828913.185212:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828913/real 1787828913] req@ffff9181c5637480 x1874663043286144/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828938 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828913.185226:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043286144/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828938 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828913.185235:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828913.185238:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828913.185253:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043286144:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828913.185259:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043286272:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828913.185274:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828913.185282:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828913/real 1787828913] req@ffff9181c5637100 x1874663043286272/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828938 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828913.185290:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043286272/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828938 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828913.185297:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828913.185300:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828913.185304:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043286272:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828913.185340:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043286400:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828913.185353:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828913.185359:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828913/real 1787828913] req@ffff9181c5634380 x1874663043286400/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828938 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828913.185367:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043286400/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828938 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828913.185377:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828913.185379:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828913.185381:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043286400:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787828913.760890:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828914.784862:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828915.808829:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828916.832921:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828917.856976:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828918.304873:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828918.304884:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828918.304889:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828918.304913:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10984 00000100:00080000:1.0:1787828918.304927:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828918.304935:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787828918.305003:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2a00 x1874663043286528/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828918.305035:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828918.305039:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828918.305041:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828918.305045:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10984 00000100:00080000:1.0:1787828918.305048:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828918.305052:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828918.305064:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828918.305078:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2d80 x1874663043286656/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828918.305088:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828918.305092:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828918.305093:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828918.305097:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10984 00000100:00080000:1.0:1787828918.305099:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828918.305103:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828918.305107:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828918.305119:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1f80 x1874663043286784/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828918.305124:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043286528:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787828918.305240:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828918.305297:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828918/real 1787828918] req@ffff9181c35b2a00 x1874663043286528/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828943 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828918.305325:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2a00 x1874663043286528/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828943 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828918.305350:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828918.305357:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828918.305382:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043286528:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828918.305404:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043286656:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828918.305446:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828918.305472:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828918/real 1787828918] req@ffff9181c35b2d80 x1874663043286656/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828943 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828918.305485:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2d80 x1874663043286656/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828943 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828918.305505:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828918.305510:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828918.305514:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043286656:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828918.305552:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043286784:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828918.305572:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828918.305583:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828918/real 1787828918] req@ffff9181c35b1f80 x1874663043286784/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828943 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828918.305605:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1f80 x1874663043286784/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828943 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828918.305613:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828918.305616:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828918.305628:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043286784:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828918.881001:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828919.904956:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828920.928869:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828921.952921:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828922.976878:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828923.424843:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828923.424851:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828923.424854:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 10000000:01000000:1.0:1787828923.424856:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-sptlrpc' failed (-108). 10000000:00020000:1.0:1787828923.424876:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00080000:3.0:1787828923.424876:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10990 00000100:00080000:3.0:1787828923.424888:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828923.424915:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828923.424946:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043286912/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828923.424968:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828923.424971:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828923.424972:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828923.424976:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10990 00000100:00080000:3.0:1787828923.424978:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828923.424981:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828923.424989:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828923.425000:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635f80 x1874663043287040/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828923.425008:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828923.425011:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828923.425012:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828923.425015:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10990 00000100:00080000:3.0:1787828923.425018:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1787828923.425024:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043286912:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787828923.425024:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828923.425029:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828923.425041:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634e00 x1874663043287168/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828923.425100:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828923.425144:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828923/real 1787828923] req@ffff9181c5635500 x1874663043286912/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828948 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828923.425159:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043286912/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828948 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828923.425167:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828923.425171:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828923.425174:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043286912:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828923.425180:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043287040:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828923.425209:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828923.425221:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828923/real 1787828923] req@ffff9181c5635f80 x1874663043287040/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828948 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828923.425233:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635f80 x1874663043287040/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828948 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828923.425240:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828923.425242:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828923.425244:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043287040:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828923.425273:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043287168:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828923.425289:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828923.425296:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828923/real 1787828923] req@ffff9181c5634e00 x1874663043287168/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828948 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828923.425314:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634e00 x1874663043287168/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828948 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828923.425320:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828923.425323:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828923.425325:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043287168:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828924.000927:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787828924.704793:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:1.0:1787828924.704802:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff9181c5620000 from 341 10000000:01000000:1.0:1787828924.704807:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:1.0:1787828924.704870:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f816 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828924.704884:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f816 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828924.704909:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f816 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787828924.704932:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215db40 going to sleep for 0 seconds 00000100:00100000:1.0:1787828924.704943:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043287296:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787828924.704959:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f816 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828924.704969:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f816 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828924.704977:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f816 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828924.704986:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f816 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828924.704994:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f816 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828924.705002:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b5c00 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787828924.705006:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f816 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828924.705015:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f816 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787828924.705024:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f816 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787828924.705031:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:1.0:1787828924.705046:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787828924.705052:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f816 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828924.705061:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f816 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787828924.705083:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787828924.705087:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:0.0:1787828924.705122:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828924.705127:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828924.705129:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828924.705135:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10995 00000100:00080000:0.0:1787828924.705142:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828924.705147:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787828924.705166:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7800 x1874663043287424/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828924.705183:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828924.705186:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828924.705187:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828924.705190:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10995 00000100:00080000:0.0:1787828924.705192:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828924.705195:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828924.705203:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828924.705213:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4e00 x1874663043287552/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828924.705217:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043287424:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787828924.705226:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828924.705229:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828924.705230:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828924.705234:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10995 00000100:00080000:0.0:1787828924.705235:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828924.705238:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828924.705241:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828924.705260:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5500 x1874663043287680/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828924.705278:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828924.705306:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828924/real 1787828924] req@ffff9181c55a7800 x1874663043287424/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828949 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828924.705337:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7800 x1874663043287424/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828949 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828924.705345:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828924.705348:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828924.705358:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043287424:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828924.705376:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043287552:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828924.705401:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828924.705416:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828924/real 1787828924] req@ffff9181c55a4e00 x1874663043287552/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828949 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828924.705430:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4e00 x1874663043287552/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828949 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828924.705437:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828924.705439:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828924.705442:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043287552:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828924.705446:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043287680:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828924.705462:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828924.705476:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828924/real 1787828924] req@ffff9181c55a5500 x1874663043287680/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828949 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828924.705489:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5500 x1874663043287680/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828949 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828924.705504:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828924.705508:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828924.705511:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043287680:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828925.024917:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828926.048976:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828927.072927:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828928.096897:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828929.120941:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828930.080906:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828930.080921:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828930.080926:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828930.080939:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 10996 00000100:00080000:3.0:1787828930.080953:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828930.080962:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828930.081015:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634e00 x1874663043287808/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828930.081041:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828930.081044:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828930.081046:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828930.081049:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10996 00000100:00080000:3.0:1787828930.081052:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828930.081055:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828930.081064:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828930.081073:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635f80 x1874663043287936/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828930.081081:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828930.081084:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828930.081085:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828930.081091:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 10996 00000100:00080000:3.0:1787828930.081094:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828930.081097:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828930.081100:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828930.081107:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043288064/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828930.081133:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043287808:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828930.081284:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043287936:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828930.081339:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043288064:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828930.144952:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828931.168894:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828932.192920:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1787828933.153078:0:2379:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:1.0:1787828933.153118:0:2379:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787828933.153132:0:2379:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1787828933.153183:0:2379:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1787828933.153203:0:2379:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:2.0:1787828933.153242:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828930/real 1787828933] req@ffff9181c5635500 x1874663043288064/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828955 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828933.153271:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043288064/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828955 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828933.153284:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828933.153290:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828933.153294:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043288064:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828933.153302:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828930/real 1787828933] req@ffff9181c5634e00 x1874663043287808/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828955 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828933.153316:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634e00 x1874663043287808/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828955 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828933.153324:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828933.153328:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828933.153348:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043287808:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828933.153355:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828930/real 1787828933] req@ffff9181c5635f80 x1874663043287936/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828955 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828933.153366:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635f80 x1874663043287936/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828955 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828933.153373:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828933.153377:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828933.153380:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043287936:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828933.216906:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828934.240886:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828935.200880:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828935.200890:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828935.200894:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828935.200904:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11001 00000100:00080000:0.0:1787828935.200913:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828935.200920:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787828935.200950:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5180 x1874663043288192/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828935.200976:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828935.200980:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828935.200982:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828935.200985:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11001 00000100:00080000:0.0:1787828935.200988:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828935.200991:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828935.201001:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828935.201018:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4380 x1874663043288320/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828935.201031:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828935.201036:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828935.201038:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828935.201046:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11001 00000100:00100000:2.0:1787828935.201047:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043288192:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787828935.201051:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828935.201068:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828935.201073:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828935.201086:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5c00 x1874663043288448/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828935.201127:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828935.201170:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828935/real 1787828935] req@ffff9181c55a5180 x1874663043288192/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828960 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828935.201210:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5180 x1874663043288192/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828960 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828935.201225:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828935.201230:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828935.201259:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043288192:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828935.201270:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043288320:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828935.201300:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828935.201317:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828935/real 1787828935] req@ffff9181c55a4380 x1874663043288320/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828960 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828935.201342:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4380 x1874663043288320/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828960 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828935.201353:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828935.201357:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828935.201362:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043288320:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828935.201390:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043288448:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828935.201417:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828935.201433:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828935/real 1787828935] req@ffff9181c55a5c00 x1874663043288448/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828960 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828935.201448:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5c00 x1874663043288448/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828960 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828935.201458:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828935.201463:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828935.201467:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043288448:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828935.264892:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828936.288896:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828937.312909:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828938.336924:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828939.360898:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828940.320849:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828940.320861:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828940.320866:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828940.320878:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11006 00000100:00080000:0.0:1787828940.320890:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828940.320899:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787828940.320948:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4700 x1874663043288576/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828940.320985:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828940.320988:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828940.320990:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828940.320994:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11006 00000100:00080000:0.0:1787828940.320997:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828940.321000:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828940.321010:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828940.321022:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5f80 x1874663043288704/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828940.321031:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828940.321034:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828940.321053:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828940.321058:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11006 00000100:00080000:0.0:1787828940.321062:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828940.321068:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787828940.321073:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043288576:192.168.203.147@tcp:250:kworker.0 00000008:00100000:0.0:1787828940.321075:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828940.321099:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6d80 x1874663043288832/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828940.321162:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828940.321205:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828940/real 1787828940] req@ffff9181c55a4700 x1874663043288576/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828965 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828940.321226:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4700 x1874663043288576/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828965 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828940.321247:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828940.321252:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828940.321271:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043288576:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828940.321280:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043288704:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828940.321307:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828940.321321:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828940/real 1787828940] req@ffff9181c55a5f80 x1874663043288704/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828965 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828940.321338:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5f80 x1874663043288704/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828965 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828940.321360:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828940.321365:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828940.321369:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043288704:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828940.321409:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043288832:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828940.321432:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828940.321444:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828940/real 1787828940] req@ffff9181c55a6d80 x1874663043288832/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828965 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828940.321458:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6d80 x1874663043288832/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828965 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828940.321468:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828940.321472:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828940.321475:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043288832:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828940.385022:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828941.408894:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828942.433007:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828943.456960:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828944.480938:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787828944.928882:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-cliir' failed (-108). 10000000:00000400:1.0:1787828944.928893:0:159735:0:(mgc_request.c:1900:mgc_process_log()) MGC192.168.203.147@tcp: IR log lustre-cliir failed, not fatal: rc = -108 10000000:01000000:1.0:1787828944.928903:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:1.0:1787828944.928906:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff9181c5620000 from 52 10000000:01000000:1.0:1787828944.928909:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:1.0:1787828944.928980:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f81d lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828944.928991:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f81d lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828944.929002:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f81d lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787828944.929025:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215db40 going to sleep for 0 seconds 00000100:00100000:1.0:1787828944.929038:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043288960:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787828944.929051:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f81d lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828944.929058:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f81d lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828944.929063:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f81d lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828944.929073:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f81d lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828944.929079:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f81d lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828944.929084:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b5c00 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787828944.929087:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f81d lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828944.929093:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f81d lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787828944.929098:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f81d lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787828944.929102:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1787828944.929104:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787828944.929111:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f81d lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828944.929115:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f81d lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787828944.929136:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787828944.929139:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:0.0:1787828944.929206:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828944.929213:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828944.929216:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828944.929223:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11012 00000100:00080000:0.0:1787828944.929243:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828944.929250:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787828944.929278:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7100 x1874663043289088/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828944.929317:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828944.929321:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828944.929323:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828944.929329:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11012 00000100:00080000:0.0:1787828944.929333:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828944.929338:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828944.929351:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828944.929372:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5880 x1874663043289216/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828944.929387:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828944.929391:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828944.929393:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828944.929401:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11012 00000100:00080000:0.0:1787828944.929407:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828944.929419:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787828944.929438:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043289088:192.168.203.147@tcp:250:kworker.0 00000008:00100000:0.0:1787828944.929439:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828944.929468:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6680 x1874663043289344/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828944.929551:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828944.929630:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828944/real 1787828944] req@ffff9181c55a7100 x1874663043289088/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828969 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828944.929656:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7100 x1874663043289088/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828969 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828944.929672:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828944.929677:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828944.929702:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043289088:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828944.929822:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043289216:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828944.929887:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828944.929909:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828944/real 1787828944] req@ffff9181c55a5880 x1874663043289216/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828969 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828944.929926:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5880 x1874663043289216/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828969 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828944.929937:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828944.929941:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828944.929945:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043289216:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828944.929953:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043289344:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828944.929977:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828944.929988:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828944/real 1787828944] req@ffff9181c55a6680 x1874663043289344/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828969 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828944.930001:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6680 x1874663043289344/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828969 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828944.930009:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828944.930013:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828944.930016:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043289344:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828945.504954:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828946.528904:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828947.553036:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828948.576883:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828949.600930:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828950.048883:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828950.048898:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828950.048904:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828950.048920:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11016 00000100:00080000:3.0:1787828950.048935:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828950.048944:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828950.048998:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043289472/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828950.049032:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828950.049036:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828950.049039:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828950.049043:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11016 00000100:00080000:3.0:1787828950.049046:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828950.049051:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828950.049061:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828950.049075:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043289600/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828950.049084:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828950.049089:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:2.0:1787828950.049090:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043289472:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787828950.049090:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828950.049097:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11016 00000100:00080000:3.0:1787828950.049102:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828950.049105:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828950.049108:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828950.049117:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043289728/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828950.049178:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828950.049216:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828950/real 1787828950] req@ffff9181c5634380 x1874663043289472/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828975 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828950.049231:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043289472/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828975 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828950.049241:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828950.049244:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828950.049259:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043289472:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828950.049283:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043289600:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828950.049304:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828950.049317:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828950/real 1787828950] req@ffff9181c5637100 x1874663043289600/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828975 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828950.049331:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043289600/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828975 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828950.049337:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828950.049339:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828950.049341:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043289600:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828950.049345:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043289728:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828950.049355:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828950.049361:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828950/real 1787828950] req@ffff9181c5637480 x1874663043289728/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828975 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828950.049369:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043289728/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828975 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828950.049374:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828950.049376:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828950.049378:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043289728:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828950.624979:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828951.648905:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828952.672946:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787828953.696957:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828954.720861:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828955.168852:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828955.168863:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828955.168868:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828955.168879:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11021 00000100:00080000:1.0:1787828955.168891:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828955.168900:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787828955.168941:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0a80 x1874663043289856/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828955.168973:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828955.168978:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828955.168980:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828955.168986:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11021 00000100:00080000:1.0:1787828955.168990:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828955.168996:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828955.169007:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828955.169026:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3800 x1874663043289984/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828955.169042:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1787828955.169064:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043289856:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787828955.169067:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828955.169075:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828955.169085:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11021 00000100:00080000:1.0:1787828955.169091:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828955.169100:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828955.169108:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828955.169148:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3b80 x1874663043290112/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828955.169169:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828955.169225:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828955/real 1787828955] req@ffff9181c35b0a80 x1874663043289856/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828980 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828955.169241:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0a80 x1874663043289856/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828980 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828955.169252:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828955.169256:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828955.169271:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043289856:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828955.169297:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043289984:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828955.169317:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828955.169328:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828955/real 1787828955] req@ffff9181c35b3800 x1874663043289984/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828980 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828955.169338:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3800 x1874663043289984/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828980 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828955.169344:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828955.169347:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828955.169349:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043289984:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828955.169354:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043290112:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828955.169372:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828955.169385:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828955/real 1787828955] req@ffff9181c35b3b80 x1874663043290112/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828980 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828955.169398:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3b80 x1874663043290112/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828980 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828955.169417:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828955.169421:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828955.169425:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043290112:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828955.744924:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828956.768968:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828957.792917:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828958.816858:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828959.840890:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828960.288851:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828960.288860:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828960.288865:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828960.288876:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11026 00000100:00080000:3.0:1787828960.288888:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828960.288896:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828960.288932:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043290240/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828960.288969:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828960.288975:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828960.288978:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828960.288985:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11026 00000100:00080000:3.0:1787828960.288991:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828960.288998:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828960.289012:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828960.289034:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043290368/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828960.289058:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043290240:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787828960.289065:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828960.289073:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828960.289076:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828960.289085:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11026 00000100:00080000:3.0:1787828960.289092:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828960.289097:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828960.289105:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828960.289122:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043290496/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828960.289161:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828960.289208:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828960/real 1787828960] req@ffff9181c5636a00 x1874663043290240/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828985 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828960.289236:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043290240/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828985 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828960.289246:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828960.289250:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828960.289270:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043290240:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828960.289297:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043290368:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828960.289327:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828960.289337:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828960/real 1787828960] req@ffff9181c5637800 x1874663043290368/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828985 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828960.289346:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043290368/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828985 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828960.289351:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828960.289354:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828960.289357:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043290368:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828960.289361:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043290496:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828960.289372:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828960.289378:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828960/real 1787828960] req@ffff9181c5636d80 x1874663043290496/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828985 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828960.289388:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043290496/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828985 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828960.289397:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828960.289400:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828960.289404:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043290496:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787828960.864878:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828961.888899:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828962.912881:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828963.936872:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787828964.960910:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828965.408878:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828965.408888:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828965.408893:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828965.408904:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11032 00000100:00080000:3.0:1787828965.408914:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828965.408922:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 10000000:01000000:1.0:1787828965.408945:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-client' failed (-108). 10000000:00020000:1.0:1787828965.408952:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:1.0:1787828965.408957:0:159735:0:(mgc_request.c:611:do_requeue()) updating log params 00000100:00100000:3.0:1787828965.408958:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043290624/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 10000000:01000000:1.0:1787828965.408960:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff9181c5620000 from 7 10000000:01000000:1.0:1787828965.408967:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00000100:00080000:3.0:1787828965.408992:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828965.408997:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828965.408999:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828965.409003:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11032 00000100:00080000:3.0:1787828965.409007:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828965.409011:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:1.0:1787828965.409015:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f824 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000008:00100000:3.0:1787828965.409020:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:1.0:1787828965.409033:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f824 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787828965.409044:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637b80 x1874663043290880/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787828965.409046:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f824 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787828965.409074:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828965.409081:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:1.0:1787828965.409082:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215db40 going to sleep for 0 seconds 00000100:00100000:2.0:1787828965.409084:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043290624:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787828965.409084:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828965.409097:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11032 00000100:00100000:1.0:1787828965.409098:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043290752:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00080000:3.0:1787828965.409106:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828965.409112:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:1.0:1787828965.409117:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f824 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000008:00100000:3.0:1787828965.409117:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:1.0:1787828965.409132:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f824 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787828965.409139:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043291008/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787828965.409142:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f824 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828965.409166:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f824 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828965.409192:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f824 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000800:00000100:2.0:1787828965.409206:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00010000:00010000:1.0:1787828965.409208:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b5c00 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787828965.409216:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f824 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828965.409229:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f824 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787828965.409240:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f824 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787828965.409247:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:1.0:1787828965.409255:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00000100:00100000:2.0:1787828965.409269:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828965/real 1787828965] req@ffff9181c5635c00 x1874663043290624/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828990 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:1.0:1787828965.409271:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f824 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828965.409296:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f824 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787828965.409309:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043290624/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828990 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828965.409322:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828965.409326:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828965.409332:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043290624:192.168.203.147@tcp:250:kworker.0 10000000:01000000:1.0:1787828965.409337:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787828965.409343:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00100000:2.0:1787828965.409361:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043290880:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828965.409399:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828965.409415:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828965/real 1787828965] req@ffff9181c5637b80 x1874663043290880/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828990 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828965.409424:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1787828965.409427:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637b80 x1874663043290880/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828990 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787828965.409450:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787828965.409451:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828965.409460:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00080000:3.0:1787828965.409461:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1787828965.409471:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043290880:192.168.203.147@tcp:38:kworker.0 00000100:00080000:3.0:1787828965.409478:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11037 00000100:00100000:2.0:1787828965.409480:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043291008:192.168.203.147@tcp:38:kworker.0 00000100:00080000:3.0:1787828965.409502:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828965.409516:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:2.0:1787828965.409517:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828965.409529:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828965/real 1787828965] req@ffff9181c5634000 x1874663043291008/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828990 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828965.409538:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043291008/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828990 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828965.409544:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828965.409552:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787828965.409552:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634a80 x1874663043291136/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828965.409580:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043291008:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828965.409619:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043291136:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787828965.409640:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828965.409649:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828965/real 1787828965] req@ffff9181c5634a80 x1874663043291136/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828990 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828965.409657:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634a80 x1874663043291136/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828990 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828965.409663:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828965.409665:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828965.409681:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043291136:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787828965.984903:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828967.008885:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828968.032933:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828969.056886:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828970.080897:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828970.528890:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828970.528899:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828970.528922:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828970.528931:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11037 00000100:00080000:0.0:1787828970.528941:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828970.528947:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787828970.529002:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7480 x1874663043291264/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828970.529029:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828970.529033:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828970.529035:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828970.529039:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11037 00000100:00080000:0.0:1787828970.529042:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828970.529046:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828970.529054:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828970.529065:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7b80 x1874663043291392/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828970.529082:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828970.529086:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828970.529088:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828970.529094:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11037 00000100:00080000:0.0:1787828970.529096:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828970.529100:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828970.529104:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828970.529116:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6680 x1874663043291520/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828970.529124:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043291264:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787828970.529223:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828970.529266:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828970/real 1787828970] req@ffff9181c55a7480 x1874663043291264/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828995 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828970.529283:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7480 x1874663043291264/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787828995 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828970.529292:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828970.529296:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828970.529316:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043291264:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828970.529322:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043291392:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828970.529339:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828970.529351:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828970/real 1787828970] req@ffff9181c55a7b80 x1874663043291392/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828995 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828970.529364:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7b80 x1874663043291392/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828995 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828970.529371:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828970.529374:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828970.529377:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043291392:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828970.529412:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043291520:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828970.529424:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828970.529432:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828970/real 1787828970] req@ffff9181c55a6680 x1874663043291520/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828995 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828970.529440:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6680 x1874663043291520/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787828995 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828970.529445:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828970.529447:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828970.529449:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043291520:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828971.104912:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828972.128941:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828973.152905:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828974.176882:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828975.200928:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787828975.648834:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828975.648843:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787828975.648847:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828975.648857:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11042 00000100:00080000:1.0:1787828975.648869:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828975.648877:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787828975.648914:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0700 x1874663043291648/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828975.648937:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828975.648940:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787828975.648941:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828975.648944:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11042 00000100:00080000:1.0:1787828975.648947:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828975.648950:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828975.648957:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828975.648968:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3b80 x1874663043291776/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787828975.648976:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787828975.648979:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787828975.648981:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787828975.648984:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11042 00000100:00080000:1.0:1787828975.648992:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787828975.648996:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787828975.649000:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828975.649010:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3800 x1874663043291904/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828975.649011:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043291648:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787828975.649117:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828975.649161:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828975/real 1787828975] req@ffff9181c35b0700 x1874663043291648/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829000 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828975.649179:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0700 x1874663043291648/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829000 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828975.649190:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828975.649194:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828975.649210:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043291648:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828975.649217:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043291776:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828975.649241:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828975.649255:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828975/real 1787828975] req@ffff9181c35b3b80 x1874663043291776/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829000 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828975.649283:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3b80 x1874663043291776/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829000 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828975.649296:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828975.649299:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828975.649302:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043291776:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828975.649327:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043291904:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828975.649343:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828975.649351:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828975/real 1787828975] req@ffff9181c35b3800 x1874663043291904/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829000 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828975.649360:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3800 x1874663043291904/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829000 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828975.649367:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828975.649369:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828975.649372:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043291904:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828976.224900:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828977.248971:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828978.272860:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828979.296901:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828980.320886:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828980.768888:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828980.768896:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828980.768899:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828980.768908:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11047 00000100:00080000:3.0:1787828980.768917:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828980.768924:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828980.768957:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634a80 x1874663043292032/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828980.768981:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828980.768984:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828980.768985:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828980.768988:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11047 00000100:00080000:3.0:1787828980.768990:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828980.768993:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828980.769001:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828980.769009:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043292160/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828980.769017:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828980.769020:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828980.769022:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828980.769027:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11047 00000100:00080000:3.0:1787828980.769031:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828980.769036:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828980.769041:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828980.769054:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637b80 x1874663043292288/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828980.769071:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043292032:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787828980.769165:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828980.769210:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828980/real 1787828980] req@ffff9181c5634a80 x1874663043292032/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829005 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828980.769226:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634a80 x1874663043292032/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829005 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828980.769236:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828980.769239:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828980.769255:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043292032:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828980.769261:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043292160:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828980.769285:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828980.769298:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828980/real 1787828980] req@ffff9181c5634000 x1874663043292160/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829005 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828980.769311:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043292160/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829005 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828980.769317:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828980.769319:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828980.769322:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043292160:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828980.769342:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043292288:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828980.769354:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828980.769361:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828980/real 1787828980] req@ffff9181c5637b80 x1874663043292288/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829005 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828980.769369:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637b80 x1874663043292288/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829005 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828980.769374:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828980.769376:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828980.769378:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043292288:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828981.344903:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828982.368921:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828983.392877:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828984.416898:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828985.440925:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828985.888877:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828985.888887:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828985.888891:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 10000000:01000000:1.0:1787828985.888896:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'params' failed (-108). 00000100:00080000:0.0:1787828985.888901:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11052 10000000:00020000:1.0:1787828985.888905:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00080000:0.0:1787828985.888912:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 10000000:01000000:1.0:1787828985.888912:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:1.0:1787828985.888918:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff9181c87b19b8 from 1 00000100:00080000:0.0:1787828985.888923:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 10000000:01000000:1.0:1787828985.888924:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00000100:00100000:0.0:1787828985.888956:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5880 x1874663043292544/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787828985.888972:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f82b lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787828985.888980:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828985.888987:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00010000:00010000:1.0:1787828985.888988:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f82b lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787828985.888990:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828985.889000:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11052 00000100:00080000:0.0:1787828985.889007:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00010000:00010000:1.0:1787828985.889008:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f82b lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787828985.889014:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787828985.889023:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043292544:192.168.203.147@tcp:250:kworker.0 00000008:00100000:0.0:1787828985.889027:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787828985.889036:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215db40 going to sleep for 0 seconds 00000100:00100000:0.0:1787828985.889042:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7100 x1874663043292672/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787828985.889050:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043292416:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787828985.889066:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f82b lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787828985.889068:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828985.889076:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828985.889080:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00010000:00010000:1.0:1787828985.889086:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f82b lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787828985.889087:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11052 00000800:00000100:2.0:1787828985.889087:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:0.0:1787828985.889092:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828985.889099:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:1.0:1787828985.889105:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f82b lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000008:00100000:0.0:1787828985.889107:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:1.0:1787828985.889121:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f82b lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787828985.889125:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6d80 x1874663043292800/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828985.889130:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828985/real 1787828985] req@ffff9181c55a5880 x1874663043292544/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829010 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:1.0:1787828985.889144:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f82b lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828985.889158:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b5800 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787828985.889162:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f82b lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787828985.889164:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5880 x1874663043292544/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829010 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:1.0:1787828985.889187:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f82b lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787828985.889191:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828985.889198:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828985.889205:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043292544:192.168.203.147@tcp:250:kworker.0 10000000:00010000:1.0:1787828985.889206:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f82b lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787828985.889216:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1787828985.889221:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00000100:00100000:2.0:1787828985.889226:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043292672:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828985.889229:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f82b lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787828985.889239:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f82b lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000800:00000100:2.0:1787828985.889250:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 10000000:01000000:1.0:1787828985.889262:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00100000:2.0:1787828985.889265:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828985/real 1787828985] req@ffff9181c55a7100 x1874663043292672/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829010 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787828985.889267:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00100000:2.0:1787828985.889280:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7100 x1874663043292672/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829010 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828985.889290:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828985.889294:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828985.889298:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043292672:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828985.889304:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043292800:192.168.203.147@tcp:38:kworker.0 00000100:00080000:0.0:1787828985.889316:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828985.889319:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828985.889321:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828985.889325:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11057 00000800:00000100:2.0:1787828985.889325:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:0.0:1787828985.889328:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828985.889332:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787828985.889337:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828985/real 1787828985] req@ffff9181c55a6d80 x1874663043292800/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829010 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787828985.889345:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5f80 x1874663043292928/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828985.889350:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6d80 x1874663043292800/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829010 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828985.889365:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828985.889369:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828985.889372:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043292800:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828985.889387:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043292928:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787828985.889405:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828985.889421:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828985/real 1787828985] req@ffff9181c55a5f80 x1874663043292928/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829010 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828985.889446:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5f80 x1874663043292928/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829010 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828985.889452:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828985.889454:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828985.889463:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043292928:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787828986.464910:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828987.488946:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828988.512915:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828989.536908:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828990.560876:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787828991.008902:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828991.008913:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787828991.008917:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828991.008928:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11057 00000100:00080000:3.0:1787828991.008940:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828991.008948:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787828991.008998:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043293056/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828991.009022:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828991.009025:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787828991.009026:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828991.009030:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11057 00000100:00080000:3.0:1787828991.009032:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787828991.009036:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828991.009044:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828991.009054:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043293184/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787828991.009061:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787828991.009069:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787828991.009071:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787828991.009073:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11057 00000100:00080000:3.0:1787828991.009076:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1787828991.009081:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043293056:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787828991.009082:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787828991.009090:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787828991.009098:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043293312/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787828991.009164:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828991.009205:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828991/real 1787828991] req@ffff9181c5635c00 x1874663043293056/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829016 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828991.009224:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043293056/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829016 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828991.009235:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828991.009239:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828991.009255:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043293056:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828991.009263:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043293184:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828991.009289:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828991.009305:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828991/real 1787828991] req@ffff9181c5636d80 x1874663043293184/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829016 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828991.009317:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043293184/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829016 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828991.009325:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828991.009329:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828991.009332:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043293184:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828991.009358:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043293312:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787828991.009377:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787828991.009401:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828991/real 1787828991] req@ffff9181c5637800 x1874663043293312/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829016 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828991.009415:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043293312/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829016 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828991.009423:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828991.009426:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828991.009430:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043293312:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828991.584882:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828992.608883:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828993.632872:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828994.656944:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828995.680891:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787828996.128863:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828996.128874:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787828996.128878:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828996.128889:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11062 00000100:00080000:0.0:1787828996.128908:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828996.128915:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787828996.128956:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4700 x1874663043293440/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828996.128978:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828996.128981:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787828996.128982:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828996.128985:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11062 00000100:00080000:0.0:1787828996.128988:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828996.128990:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828996.128999:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828996.129010:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5c00 x1874663043293568/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787828996.129017:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787828996.129020:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787828996.129021:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787828996.129025:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11062 00000100:00080000:0.0:1787828996.129027:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787828996.129029:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787828996.129032:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787828996.129041:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4380 x1874663043293696/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787828996.129043:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043293440:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828996.129157:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043293568:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828996.129206:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043293696:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828996.704930:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828997.728874:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787828998.752887:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1787828999.201041:0:2376:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:1.0:1787828999.201086:0:2376:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787828999.201103:0:2376:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1787828999.201143:0:2376:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1787828999.201154:0:2376:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:2.0:1787828999.201189:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828996/real 1787828999] req@ffff9181c55a4380 x1874663043293696/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829021 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828999.201217:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4380 x1874663043293696/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829021 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828999.201229:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828999.201235:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828999.201239:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043293696:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787828999.201247:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828996/real 1787828999] req@ffff9181c55a4700 x1874663043293440/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829021 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828999.201259:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4700 x1874663043293440/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829021 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828999.201267:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828999.201271:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828999.201292:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043293440:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787828999.201299:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787828996/real 1787828999] req@ffff9181c55a5c00 x1874663043293568/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829021 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787828999.201310:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5c00 x1874663043293568/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829021 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787828999.201318:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787828999.201321:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787828999.201325:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043293568:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787828999.776852:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829000.800871:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829001.248856:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829001.248866:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829001.248870:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829001.248881:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11067 00000100:00080000:3.0:1787829001.248892:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829001.248901:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829001.248937:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043293824/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829001.248974:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829001.248978:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829001.248981:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829001.248986:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11067 00000100:00080000:3.0:1787829001.248989:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829001.248994:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829001.249003:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829001.249017:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043293952/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829001.249031:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1787829001.249032:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043293824:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787829001.249036:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829001.249041:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829001.249047:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11067 00000100:00080000:3.0:1787829001.249050:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829001.249054:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829001.249059:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829001.249071:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043294080/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829001.249101:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829001.249135:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829001/real 1787829001] req@ffff9181c5636a00 x1874663043293824/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829026 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829001.249153:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043293824/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829026 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829001.249164:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829001.249168:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829001.249191:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043293824:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829001.249218:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043293952:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829001.249245:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829001.249265:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829001/real 1787829001] req@ffff9181c5637480 x1874663043293952/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829026 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829001.249279:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043293952/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829026 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829001.249288:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829001.249291:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829001.249294:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043293952:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829001.249301:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043294080:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829001.249339:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829001.249353:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829001/real 1787829001] req@ffff9181c5637100 x1874663043294080/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829026 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829001.249366:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043294080/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829026 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829001.249374:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829001.249377:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829001.249380:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043294080:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829001.824896:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829002.848879:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829003.872902:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829004.896921:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829005.920908:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787829006.368880:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-sptlrpc' failed (-108). 10000000:00020000:1.0:1787829006.368888:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00080000:1.0:1787829006.368905:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829006.368908:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829006.368911:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829006.368919:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11072 00000100:00080000:1.0:1787829006.368928:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829006.368934:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787829006.368960:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0a80 x1874663043294208/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829006.368981:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829006.368984:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829006.368985:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829006.368988:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11072 00000100:00080000:1.0:1787829006.368990:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829006.368992:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829006.368999:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829006.369008:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1f80 x1874663043294336/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829006.369016:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829006.369019:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829006.369021:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829006.369026:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11072 00000100:00080000:1.0:1787829006.369030:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1787829006.369032:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043294208:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787829006.369038:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829006.369046:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829006.369061:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2d80 x1874663043294464/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829006.369100:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829006.369133:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829006/real 1787829006] req@ffff9181c35b0a80 x1874663043294208/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829031 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829006.369152:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0a80 x1874663043294208/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829031 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829006.369169:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829006.369174:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829006.369179:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043294208:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829006.369186:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043294336:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829006.369206:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829006.369216:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829006/real 1787829006] req@ffff9181c35b1f80 x1874663043294336/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829031 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829006.369225:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1f80 x1874663043294336/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829031 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829006.369236:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829006.369238:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829006.369240:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043294336:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829006.369256:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043294464:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829006.369268:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829006.369275:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829006/real 1787829006] req@ffff9181c35b2d80 x1874663043294464/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829031 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829006.369297:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2d80 x1874663043294464/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829031 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829006.369304:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829006.369306:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829006.369308:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043294464:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829006.944862:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787829007.712833:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:1.0:1787829007.712840:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff9181c5620000 from 341 10000000:01000000:1.0:1787829007.712843:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:1.0:1787829007.712887:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f832 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829007.712896:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f832 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829007.712905:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f832 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787829007.712922:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215db40 going to sleep for 0 seconds 00000100:00100000:1.0:1787829007.712931:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043294592:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787829007.712943:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f832 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829007.712949:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f832 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829007.712953:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f832 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829007.712959:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f832 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829007.712964:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f832 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829007.712969:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b5800 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787829007.712971:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f832 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829007.712976:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f832 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787829007.712981:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f832 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787829007.712985:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:1.0:1787829007.712987:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787829007.712991:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f832 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829007.712995:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f832 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787829007.713012:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787829007.713014:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:3.0:1787829007.713076:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829007.713081:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829007.713084:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829007.713090:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11078 00000100:00080000:3.0:1787829007.713098:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829007.713103:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829007.713129:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043294720/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829007.713151:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829007.713154:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829007.713155:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829007.713158:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11078 00000100:00080000:3.0:1787829007.713160:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829007.713163:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829007.713170:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829007.713180:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635f80 x1874663043294848/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829007.713187:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829007.713189:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829007.713191:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829007.713193:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11078 00000100:00080000:3.0:1787829007.713195:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1787829007.713200:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043294720:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787829007.713201:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829007.713207:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829007.713225:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634e00 x1874663043294976/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829007.713268:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829007.713311:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829007/real 1787829007] req@ffff9181c5634380 x1874663043294720/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829032 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829007.713338:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043294720/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829032 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829007.713349:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829007.713353:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829007.713365:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043294720:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829007.713372:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043294848:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829007.713396:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829007.713408:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829007/real 1787829007] req@ffff9181c5635f80 x1874663043294848/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829032 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829007.713421:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635f80 x1874663043294848/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829032 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829007.713429:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829007.713433:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829007.713436:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043294848:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829007.713457:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043294976:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829007.713478:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829007.713491:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829007/real 1787829007] req@ffff9181c5634e00 x1874663043294976/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829032 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829007.713503:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634e00 x1874663043294976/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829032 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829007.713511:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829007.713515:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829007.713518:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043294976:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829007.968861:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829008.992879:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829010.016930:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829011.040963:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829012.064888:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829013.024924:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829013.024939:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829013.024945:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829013.024960:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11079 00000100:00080000:0.0:1787829013.024975:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829013.024987:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829013.025048:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5180 x1874663043295104/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829013.025085:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829013.025091:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829013.025094:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829013.025100:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11079 00000100:00080000:0.0:1787829013.025106:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829013.025113:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829013.025126:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829013.025148:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5500 x1874663043295232/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829013.025158:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043295104:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787829013.025174:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829013.025182:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829013.025185:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829013.025193:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11079 00000100:00080000:0.0:1787829013.025200:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829013.025207:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829013.025215:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829013.025234:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4e00 x1874663043295360/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829013.025276:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829013.025330:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829013/real 1787829013] req@ffff9181c55a5180 x1874663043295104/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829038 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829013.025357:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5180 x1874663043295104/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829038 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829013.025374:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829013.025381:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829013.025405:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043295104:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829013.025440:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043295232:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829013.025482:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829013.025502:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829013/real 1787829013] req@ffff9181c55a5500 x1874663043295232/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829038 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829013.025523:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5500 x1874663043295232/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829038 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829013.025537:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829013.025543:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829013.025548:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043295232:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829013.025559:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043295360:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829013.025606:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829013.025627:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829013/real 1787829013] req@ffff9181c55a4e00 x1874663043295360/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829038 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829013.025647:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4e00 x1874663043295360/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829038 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829013.025669:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829013.025674:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829013.025680:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043295360:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829013.088896:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829014.112921:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829015.136872:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829016.160967:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829017.184880:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829018.144839:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829018.144847:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829018.144852:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829018.144862:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11084 00000100:00080000:1.0:1787829018.144874:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829018.144882:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787829018.144923:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2a00 x1874663043295488/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829018.144946:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829018.144950:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829018.144952:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829018.144956:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11084 00000100:00080000:1.0:1787829018.144961:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829018.144965:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829018.144975:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829018.144989:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3480 x1874663043295616/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829018.144997:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829018.144999:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829018.145001:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829018.145004:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11084 00000100:00080000:1.0:1787829018.145007:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829018.145010:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829018.145015:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829018.145022:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2680 x1874663043295744/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829018.145034:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043295488:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787829018.145089:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829018.145118:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829018/real 1787829018] req@ffff9181c35b2a00 x1874663043295488/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829043 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829018.145131:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2a00 x1874663043295488/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829043 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829018.145139:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829018.145141:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829018.145153:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043295488:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829018.145158:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043295616:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829018.145177:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829018.145189:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829018/real 1787829018] req@ffff9181c35b3480 x1874663043295616/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829043 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829018.145197:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3480 x1874663043295616/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829043 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829018.145219:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829018.145222:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829018.145224:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043295616:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829018.145256:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043295744:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829018.145271:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829018.145279:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829018/real 1787829018] req@ffff9181c35b2680 x1874663043295744/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829043 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829018.145287:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2680 x1874663043295744/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829043 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829018.145293:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829018.145295:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829018.145297:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043295744:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829018.208899:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829019.232948:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829020.256915:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829021.280846:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829022.304928:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829023.264858:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829023.264867:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829023.264870:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829023.264878:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11089 00000100:00080000:3.0:1787829023.264888:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829023.264894:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829023.264924:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043295872/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829023.264950:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829023.264957:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829023.264959:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829023.264964:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11089 00000100:00080000:3.0:1787829023.264968:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829023.264973:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829023.264983:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829023.264989:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043295872:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829023.265002:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634e00 x1874663043296000/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829023.265014:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829023.265017:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829023.265018:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829023.265022:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11089 00000100:00080000:3.0:1787829023.265034:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829023.265037:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829023.265041:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000800:00000100:2.0:1787829023.265047:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829023.265053:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635f80 x1874663043296128/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829023.265081:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829023/real 1787829023] req@ffff9181c5635500 x1874663043295872/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829048 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829023.265095:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043295872/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829048 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829023.265103:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829023.265106:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829023.265116:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043295872:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829023.265136:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043296000:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829023.265154:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829023.265162:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829023/real 1787829023] req@ffff9181c5634e00 x1874663043296000/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829048 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829023.265186:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634e00 x1874663043296000/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829048 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829023.265192:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829023.265194:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829023.265196:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043296000:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829023.265201:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043296128:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829023.265215:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829023.265222:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829023/real 1787829023] req@ffff9181c5635f80 x1874663043296128/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829048 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829023.265230:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635f80 x1874663043296128/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829048 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829023.265235:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829023.265237:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829023.265239:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043296128:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829023.328974:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829024.352934:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829025.376888:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829026.400943:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829027.424900:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787829027.872849:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-cliir' failed (-108). 10000000:00000400:1.0:1787829027.872859:0:159735:0:(mgc_request.c:1900:mgc_process_log()) MGC192.168.203.147@tcp: IR log lustre-cliir failed, not fatal: rc = -108 10000000:01000000:1.0:1787829027.872867:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:1.0:1787829027.872870:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff9181c5620000 from 52 10000000:01000000:1.0:1787829027.872874:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:1.0:1787829027.872919:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f839 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829027.872931:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f839 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829027.872952:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f839 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787829027.872980:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215db40 going to sleep for 0 seconds 00000100:00100000:1.0:1787829027.872993:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043296256:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787829027.873006:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f839 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829027.873015:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f839 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829027.873024:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f839 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829027.873032:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f839 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829027.873041:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f839 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829027.873048:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4e00 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787829027.873052:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f839 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829027.873061:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f839 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787829027.873069:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f839 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787829027.873075:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1787829027.873079:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787829027.873086:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f839 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829027.873094:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f839 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787829027.873115:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787829027.873118:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:3.0:1787829027.873183:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829027.873188:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829027.873191:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829027.873196:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11095 00000100:00080000:3.0:1787829027.873204:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829027.873209:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829027.873227:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043296384/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829027.873247:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829027.873250:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829027.873251:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829027.873254:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11095 00000100:00080000:3.0:1787829027.873257:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829027.873259:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829027.873266:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829027.873281:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043296512/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829027.873288:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829027.873291:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829027.873292:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829027.873294:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11095 00000100:00080000:3.0:1787829027.873297:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829027.873303:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787829027.873304:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043296384:192.168.203.147@tcp:250:kworker.0 00000008:00100000:3.0:1787829027.873308:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829027.873323:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043296640/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829027.873372:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829027.873403:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829027/real 1787829027] req@ffff9181c5634380 x1874663043296384/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829052 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829027.873421:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043296384/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829052 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829027.873440:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829027.873445:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829027.873459:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043296384:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829027.873466:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043296512:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829027.873493:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829027.873507:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829027/real 1787829027] req@ffff9181c5637100 x1874663043296512/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829052 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829027.873521:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043296512/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829052 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829027.873530:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829027.873533:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829027.873537:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043296512:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829027.873556:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043296640:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829027.873575:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829027.873588:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829027/real 1787829027] req@ffff9181c5637480 x1874663043296640/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829052 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829027.873602:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043296640/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829052 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829027.873611:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829027.873614:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829027.873618:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043296640:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829028.448887:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829029.472932:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829030.496990:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829031.520954:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829032.544883:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829032.992855:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829032.992866:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829032.992870:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829032.992882:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11099 00000100:00080000:0.0:1787829032.992893:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829032.992900:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829032.993004:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7800 x1874663043296768/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829032.993028:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829032.993031:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829032.993033:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829032.993036:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11099 00000100:00080000:0.0:1787829032.993039:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829032.993043:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829032.993052:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829032.993066:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4e00 x1874663043296896/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829032.993075:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1787829032.993077:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043296768:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787829032.993079:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829032.993083:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829032.993089:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11099 00000100:00080000:0.0:1787829032.993091:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829032.993094:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829032.993113:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829032.993124:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5500 x1874663043297024/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829032.993170:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829032.993211:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829032/real 1787829032] req@ffff9181c55a7800 x1874663043296768/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829057 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829032.993227:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7800 x1874663043296768/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829057 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829032.993237:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829032.993240:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829032.993258:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043296768:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829032.993283:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043296896:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829032.993304:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829032.993318:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829032/real 1787829032] req@ffff9181c55a4e00 x1874663043296896/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829057 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829032.993328:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4e00 x1874663043296896/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829057 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829032.993335:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829032.993338:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829032.993341:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043296896:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829032.993345:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043297024:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829032.993357:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829032.993372:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829032/real 1787829032] req@ffff9181c55a5500 x1874663043297024/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829057 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829032.993382:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5500 x1874663043297024/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829057 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829032.993388:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829032.993391:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829032.993393:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043297024:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829033.568908:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829034.592864:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829035.616851:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829036.640873:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829037.664867:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829038.112862:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829038.112871:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829038.112874:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829038.112882:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11104 00000100:00080000:3.0:1787829038.112892:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829038.112898:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829038.112932:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043297152/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829038.112962:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829038.112965:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829038.112966:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829038.112969:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11104 00000100:00080000:3.0:1787829038.112972:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829038.112974:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829038.112982:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829038.112993:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043297280/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829038.113001:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829038.113003:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829038.113005:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829038.113007:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11104 00000100:00080000:3.0:1787829038.113009:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829038.113011:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829038.113016:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829038.113023:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043297408/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829038.113037:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043297152:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787829038.113154:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829038.113209:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829038/real 1787829038] req@ffff9181c5636a00 x1874663043297152/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829063 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829038.113249:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043297152/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829063 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829038.113265:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829038.113271:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829038.113296:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043297152:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829038.113308:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043297280:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829038.113344:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829038.113365:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829038/real 1787829038] req@ffff9181c5637800 x1874663043297280/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829063 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829038.113382:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043297280/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829063 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829038.113402:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829038.113407:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829038.113412:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043297280:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829038.113446:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043297408:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829038.113477:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829038.113493:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829038/real 1787829038] req@ffff9181c5636d80 x1874663043297408/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829063 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829038.113509:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043297408/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829063 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829038.113520:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829038.113525:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829038.113529:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043297408:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829038.688864:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829039.712962:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829040.736883:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829041.760932:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829042.784902:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829043.232839:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829043.232846:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829043.232850:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829043.232858:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11109 00000100:00080000:3.0:1787829043.232868:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829043.232873:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829043.232904:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043297536/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829043.232927:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829043.232930:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829043.232931:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829043.232934:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11109 00000100:00080000:3.0:1787829043.232936:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829043.232939:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829043.232946:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829043.232954:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637b80 x1874663043297664/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829043.232961:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829043.232963:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829043.232965:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829043.232969:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11109 00000100:00080000:3.0:1787829043.232972:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829043.232976:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829043.232980:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829043.232986:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043297792/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829043.233016:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043297536:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787829043.233114:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829043.233157:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829043/real 1787829043] req@ffff9181c5635c00 x1874663043297536/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829068 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829043.233177:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043297536/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829068 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829043.233188:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829043.233192:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829043.233212:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043297536:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829043.233218:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043297664:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829043.233236:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829043.233247:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829043/real 1787829043] req@ffff9181c5637b80 x1874663043297664/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829068 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829043.233256:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637b80 x1874663043297664/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829068 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829043.233263:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829043.233266:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829043.233269:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043297664:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829043.233273:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043297792:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829043.233285:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829043.233292:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829043/real 1787829043] req@ffff9181c5634000 x1874663043297792/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829068 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829043.233299:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043297792/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829068 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829043.233305:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829043.233306:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829043.233308:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043297792:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829043.808961:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829044.832925:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829045.856889:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829046.880956:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829047.904918:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829048.352831:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829048.352844:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829048.352848:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829048.352859:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11114 10000000:01000000:1.0:1787829048.352871:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-client' failed (-108). 00000100:00080000:3.0:1787829048.352872:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 10000000:00020000:1.0:1787829048.352878:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00080000:3.0:1787829048.352883:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 10000000:01000000:1.0:1787829048.352884:0:159735:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:1.0:1787829048.352888:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff9181c5620000 from 7 10000000:01000000:1.0:1787829048.352892:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00000100:00100000:3.0:1787829048.352924:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634a80 x1874663043298048/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787829048.352940:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f840 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829048.352950:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f840 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787829048.352952:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829048.352960:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829048.352961:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829048.352965:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11114 00010000:00010000:1.0:1787829048.352966:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f840 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787829048.352968:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829048.352981:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787829048.352986:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215db40 going to sleep for 0 seconds 00000008:00100000:3.0:1787829048.353002:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829048.353007:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043298048:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787829048.353009:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043297920:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00100000:3.0:1787829048.353018:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043298176/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787829048.353026:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f840 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829048.353045:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f840 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787829048.353046:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829048.353054:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829048.353058:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00010000:00010000:1.0:1787829048.353059:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f840 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787829048.353063:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11114 00000100:00080000:3.0:1787829048.353070:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00010000:00010000:1.0:1787829048.353071:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f840 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787829048.353076:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:2.0:1787829048.353082:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00010000:00010000:1.0:1787829048.353085:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f840 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000008:00100000:3.0:1787829048.353085:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:1.0:1787829048.353096:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4e00 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787829048.353099:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f840 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787829048.353102:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637b80 x1874663043298304/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787829048.353115:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f840 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787829048.353123:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f840 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787829048.353127:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829048/real 1787829048] req@ffff9181c5634a80 x1874663043298048/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829073 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 10000000:01000000:1.0:1787829048.353129:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:1.0:1787829048.353134:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787829048.353140:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f840 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829048.353147:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f840 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787829048.353156:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634a80 x1874663043298048/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829073 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 10000000:01000000:1.0:1787829048.353166:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787829048.353170:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:CONNECTING 00000100:00080000:2.0:1787829048.353171:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829048.353194:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829048.353205:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043298048:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787829048.353215:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829048.353221:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829048.353223:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1787829048.353229:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043298176:192.168.203.147@tcp:38:kworker.0 00000100:00080000:3.0:1787829048.353233:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11120 00000100:00080000:3.0:1787829048.353240:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829048.353246:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829048.353262:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043298432/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829048.353266:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829048.353284:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829048/real 1787829048] req@ffff9181c5634000 x1874663043298176/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829073 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829048.353295:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043298176/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829073 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829048.353305:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829048.353308:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829048.353313:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043298176:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829048.353319:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043298304:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829048.353342:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829048.353352:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829048/real 1787829048] req@ffff9181c5637b80 x1874663043298304/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829073 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829048.353361:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637b80 x1874663043298304/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829073 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829048.353372:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829048.353374:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829048.353377:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043298304:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829048.353387:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043298432:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787829048.353399:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829048.353405:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829048/real 1787829048] req@ffff9181c5635c00 x1874663043298432/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829073 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829048.353412:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043298432/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829073 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829048.353417:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829048.353419:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829048.353428:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043298432:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787829048.928872:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829049.952944:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829050.976942:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829052.000905:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829053.024938:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829053.472850:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829053.472858:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829053.472861:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829053.472870:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11120 00000100:00080000:3.0:1787829053.472879:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829053.472885:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829053.472919:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043298560/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829053.472969:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829053.472974:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829053.472976:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829053.472981:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11120 00000100:00080000:3.0:1787829053.472986:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829053.472992:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829053.473002:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829053.473018:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043298688/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829053.473031:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1787829053.473045:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043298560:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787829053.473046:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829053.473056:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829053.473063:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11120 00000100:00080000:3.0:1787829053.473067:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829053.473074:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829053.473080:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829053.473096:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043298816/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829053.473156:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829053.473201:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829053/real 1787829053] req@ffff9181c5636d80 x1874663043298560/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829078 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829053.473222:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043298560/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829078 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829053.473235:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829053.473240:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829053.473261:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043298560:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829053.473297:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043298688:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829053.473326:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829053.473337:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829053/real 1787829053] req@ffff9181c5637800 x1874663043298688/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829078 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829053.473346:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043298688/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829078 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829053.473353:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829053.473355:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829053.473358:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043298688:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829053.473362:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043298816:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829053.473375:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829053.473382:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829053/real 1787829053] req@ffff9181c5636a00 x1874663043298816/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829078 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829053.473389:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043298816/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829078 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829053.473394:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829053.473396:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829053.473398:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043298816:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829054.048942:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829055.072977:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829056.096900:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829057.120969:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829058.144870:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829058.592871:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829058.592882:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829058.592886:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829058.592898:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11125 00000100:00080000:3.0:1787829058.592910:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829058.592932:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829058.592977:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043298944/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829058.593005:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829058.593008:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829058.593009:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829058.593016:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11125 00000100:00080000:3.0:1787829058.593019:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829058.593022:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829058.593031:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829058.593044:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043299072/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829058.593061:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1787829058.593067:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043298944:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787829058.593068:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829058.593073:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829058.593081:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11125 00000100:00080000:3.0:1787829058.593084:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829058.593087:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829058.593093:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829058.593104:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043299200/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829058.593176:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043299072:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829058.593200:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043299200:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829059.168928:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829060.192927:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829061.216915:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1787829061.665035:0:2377:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:0.0:1787829061.665082:0:2377:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1787829061.665099:0:2377:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1787829061.665141:0:2377:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1787829061.665151:0:2377:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:2.0:1787829061.665196:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829058/real 1787829061] req@ffff9181c5637100 x1874663043299072/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829083 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829061.665218:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043299072/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829083 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829061.665236:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829061.665242:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829061.665256:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043299072:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829061.665264:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829058/real 1787829061] req@ffff9181c5634380 x1874663043299200/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829083 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829061.665277:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043299200/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829083 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829061.665286:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829061.665290:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829061.665294:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043299200:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829061.665302:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829058/real 1787829061] req@ffff9181c5637480 x1874663043298944/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829083 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829061.665315:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043298944/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829083 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829061.665323:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829061.665328:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829061.665357:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043298944:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787829062.240923:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829063.264883:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829063.712907:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829063.712918:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829063.712932:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829063.712942:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11130 00000100:00080000:0.0:1787829063.712951:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829063.712957:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829063.712990:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5180 x1874663043299328/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829063.713018:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829063.713022:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829063.713024:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829063.713029:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11130 00000100:00080000:0.0:1787829063.713033:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829063.713038:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829063.713048:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829063.713065:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5c00 x1874663043299456/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829063.713074:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043299328:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787829063.713081:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829063.713085:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829063.713087:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829063.713091:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11130 00000100:00080000:0.0:1787829063.713093:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829063.713097:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829063.713101:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829063.713113:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4700 x1874663043299584/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829063.713150:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829063.713188:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829063/real 1787829063] req@ffff9181c55a5180 x1874663043299328/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829088 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829063.713208:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5180 x1874663043299328/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829088 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829063.713220:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829063.713225:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829063.713251:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043299328:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829063.713278:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043299456:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829063.713307:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829063.713323:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829063/real 1787829063] req@ffff9181c55a5c00 x1874663043299456/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829088 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829063.713336:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5c00 x1874663043299456/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829088 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829063.713345:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829063.713349:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829063.713353:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043299456:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829063.713360:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043299584:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829063.713383:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829063.713397:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829063/real 1787829063] req@ffff9181c55a4700 x1874663043299584/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829088 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829063.713410:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4700 x1874663043299584/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829088 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829063.713418:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829063.713421:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829063.713424:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043299584:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829064.288909:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829065.312904:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829066.336924:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829067.360986:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829068.384924:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787829068.832927:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'params' failed (-108). 00000100:00080000:3.0:1787829068.832936:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 10000000:00020000:1.0:1787829068.832938:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00080000:3.0:1787829068.832945:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 10000000:01000000:1.0:1787829068.832946:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 00000100:00080000:3.0:1787829068.832950:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 10000000:01000000:1.0:1787829068.832951:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff9181c87b19b8 from 1 10000000:01000000:1.0:1787829068.832957:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00000100:00080000:3.0:1787829068.832962:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11135 00000100:00080000:3.0:1787829068.832971:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829068.832976:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829068.833006:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635f80 x1874663043299840/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787829068.833013:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f847 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829068.833024:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f847 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787829068.833026:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829068.833032:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829068.833033:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00010000:00010000:1.0:1787829068.833035:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f847 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787829068.833038:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11135 00000100:00080000:3.0:1787829068.833043:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829068.833048:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829068.833056:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829068.833057:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215db40 going to sleep for 0 seconds 00000100:00100000:1.0:1787829068.833068:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043299712:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00100000:3.0:1787829068.833070:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634e00 x1874663043299968/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829068.833079:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043299840:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787829068.833081:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f847 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787829068.833084:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829068.833110:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829068.833114:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00010000:00010000:1.0:1787829068.833116:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f847 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787829068.833120:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11135 00000100:00080000:3.0:1787829068.833127:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00010000:00010000:1.0:1787829068.833128:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f847 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787829068.833134:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829068.833143:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:1.0:1787829068.833145:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f847 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829068.833153:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f847 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787829068.833157:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043300096/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787829068.833159:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4200 already unused, calling callback (ffffffffc1273c20) 00000800:00000100:2.0:1787829068.833163:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 10000000:00010000:1.0:1787829068.833165:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f847 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829068.833185:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f847 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787829068.833192:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f847 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787829068.833196:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00000100:00100000:2.0:1787829068.833201:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829068/real 1787829068] req@ffff9181c5635f80 x1874663043299840/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829093 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:1.0:1787829068.833207:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787829068.833214:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f847 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829068.833222:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f847 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787829068.833224:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635f80 x1874663043299840/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829093 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829068.833244:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829068.833250:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 10000000:01000000:1.0:1787829068.833255:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00100000:2.0:1787829068.833256:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043299840:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787829068.833261:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00100000:2.0:1787829068.833281:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043299968:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829068.833303:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829068.833314:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829068/real 1787829068] req@ffff9181c5634e00 x1874663043299968/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829093 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829068.833323:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634e00 x1874663043299968/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829093 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829068.833352:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829068.833355:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829068.833360:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043299968:192.168.203.147@tcp:38:kworker.0 00000100:00080000:3.0:1787829068.833362:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1787829068.833368:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043300096:192.168.203.147@tcp:38:kworker.0 00000100:00080000:3.0:1787829068.833370:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829068.833377:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829068.833384:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11140 00000100:00080000:3.0:1787829068.833389:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829068.833393:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:2.0:1787829068.833412:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829068.833418:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635f80 x1874663043300224/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829068.833429:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829068/real 1787829068] req@ffff9181c5635500 x1874663043300096/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829093 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829068.833444:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043300096/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829093 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829068.833457:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829068.833460:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829068.833463:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043300096:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829068.833476:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043300224:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787829068.833491:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829068.833499:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829068/real 1787829068] req@ffff9181c5635f80 x1874663043300224/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829093 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829068.833509:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635f80 x1874663043300224/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829093 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829068.833515:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829068.833518:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829068.833532:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043300224:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787829069.408898:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829070.432957:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829071.457055:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829072.480998:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829073.504949:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829073.952877:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829073.952886:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829073.952889:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829073.952897:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11140 00000100:00080000:3.0:1787829073.952907:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829073.952918:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829073.952969:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043300352/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829073.952992:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829073.952995:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829073.952997:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829073.953000:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11140 00000100:00080000:3.0:1787829073.953003:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829073.953007:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829073.953015:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829073.953025:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043300480/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829073.953033:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829073.953036:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829073.953039:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1787829073.953040:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043300352:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787829073.953044:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11140 00000100:00080000:3.0:1787829073.953050:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829073.953054:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829073.953057:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829073.953066:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043300608/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829073.953103:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829073.953134:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829073/real 1787829073] req@ffff9181c5637480 x1874663043300352/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829098 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829073.953147:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043300352/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829098 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829073.953156:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829073.953161:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829073.953177:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043300352:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829073.953203:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043300480:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829073.953231:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829073.953266:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829073/real 1787829073] req@ffff9181c5634380 x1874663043300480/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829098 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829073.953275:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043300480/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829098 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829073.953281:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829073.953283:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829073.953286:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043300480:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829073.953290:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043300608:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829073.953303:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829073.953310:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829073/real 1787829073] req@ffff9181c5637100 x1874663043300608/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829098 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829073.953318:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043300608/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829098 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829073.953323:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829073.953325:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829073.953327:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043300608:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829074.528965:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829075.552906:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829076.576905:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829077.600876:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829078.624860:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829079.072821:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829079.072835:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829079.072838:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829079.072847:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11145 00000100:00080000:3.0:1787829079.072857:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829079.072863:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829079.072897:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043300736/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829079.072923:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829079.072926:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829079.072928:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829079.072931:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11145 00000100:00080000:3.0:1787829079.072934:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829079.072938:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829079.072945:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829079.072954:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043300864/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829079.072963:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043300736:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787829079.072964:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829079.072972:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829079.072974:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829079.072980:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11145 00000100:00080000:3.0:1787829079.072982:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829079.072985:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829079.072989:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829079.072997:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043300992/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829079.073037:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829079.073078:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829079/real 1787829079] req@ffff9181c5636a00 x1874663043300736/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829104 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829079.073093:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043300736/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829104 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829079.073101:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829079.073104:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829079.073118:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043300736:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829079.073145:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043300864:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829079.073167:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829079.073176:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829079/real 1787829079] req@ffff9181c5637800 x1874663043300864/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829104 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829079.073185:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043300864/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829104 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829079.073195:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829079.073198:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829079.073200:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043300864:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829079.073204:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043300992:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829079.073215:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829079.073222:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829079/real 1787829079] req@ffff9181c5636d80 x1874663043300992/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829104 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829079.073229:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043300992/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829104 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829079.073234:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829079.073236:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829079.073238:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043300992:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829079.648943:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829080.672889:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829081.697003:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829082.720990:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829083.744913:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829084.192904:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829084.192915:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829084.192919:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829084.192930:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11150 00000100:00080000:0.0:1787829084.192951:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829084.192958:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829084.192989:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4380 x1874663043301120/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829084.193011:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829084.193014:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829084.193016:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829084.193019:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11150 00000100:00080000:0.0:1787829084.193021:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829084.193027:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829084.193035:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829084.193044:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5f80 x1874663043301248/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829084.193053:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829084.193056:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829084.193058:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1787829084.193060:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043301120:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787829084.193063:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11150 00000100:00080000:0.0:1787829084.193069:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829084.193072:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829084.193075:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829084.193083:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6d80 x1874663043301376/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829084.193119:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829084.193149:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829084/real 1787829084] req@ffff9181c55a4380 x1874663043301120/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829109 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829084.193163:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4380 x1874663043301120/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829109 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829084.193174:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829084.193178:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829084.193195:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043301120:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829084.193221:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043301248:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829084.193252:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829084.193266:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829084/real 1787829084] req@ffff9181c55a5f80 x1874663043301248/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829109 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829084.193275:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5f80 x1874663043301248/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829109 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829084.193280:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829084.193282:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829084.193285:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043301248:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829084.193289:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043301376:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829084.193301:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829084.193308:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829084/real 1787829084] req@ffff9181c55a6d80 x1874663043301376/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829109 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829084.193315:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6d80 x1874663043301376/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829109 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829084.193320:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829084.193322:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829084.193324:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043301376:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829084.768931:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829085.792880:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829086.816854:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829087.840982:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829088.864890:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829089.312827:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829089.312837:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 10000000:01000000:1.0:1787829089.312839:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-sptlrpc' failed (-108). 00000100:00080000:0.0:1787829089.312842:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 10000000:00020000:1.0:1787829089.312844:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00080000:0.0:1787829089.312856:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11155 00000100:00080000:0.0:1787829089.312869:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829089.312877:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829089.312909:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7100 x1874663043301504/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829089.312942:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829089.312945:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829089.312946:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829089.312951:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11155 00000100:00080000:0.0:1787829089.312954:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829089.312957:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829089.312964:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829089.312973:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5880 x1874663043301632/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829089.312981:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829089.312984:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829089.312985:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829089.312988:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11155 00000100:00080000:0.0:1787829089.312990:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829089.312993:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829089.312998:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829089.312998:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043301504:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787829089.313009:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6680 x1874663043301760/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829089.313056:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829089.313084:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829089/real 1787829089] req@ffff9181c55a7100 x1874663043301504/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829114 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829089.313098:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7100 x1874663043301504/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829114 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829089.313106:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829089.313109:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829089.313112:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043301504:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829089.313117:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043301632:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829089.313131:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829089.313138:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829089/real 1787829089] req@ffff9181c55a5880 x1874663043301632/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829114 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829089.313147:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5880 x1874663043301632/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829114 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829089.313152:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829089.313154:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829089.313156:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043301632:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829089.313172:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043301760:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829089.313184:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829089.313190:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829089/real 1787829089] req@ffff9181c55a6680 x1874663043301760/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829114 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829089.313198:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6680 x1874663043301760/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829114 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829089.313208:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829089.313210:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829089.313213:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043301760:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829089.888913:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787829090.528822:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:1.0:1787829090.528828:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff9181c5620000 from 341 10000000:01000000:1.0:1787829090.528833:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:1.0:1787829090.528877:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f84e lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829090.528886:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f84e lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829090.528895:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f84e lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787829090.528912:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215db40 going to sleep for 0 seconds 00000100:00100000:1.0:1787829090.528923:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043301888:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787829090.528935:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f84e lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829090.528940:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f84e lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829090.528945:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f84e lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829090.528950:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f84e lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829090.528955:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f84e lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829090.528959:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4200 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787829090.528962:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f84e lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829090.528967:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f84e lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787829090.528971:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f84e lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787829090.528975:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:1.0:1787829090.528977:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787829090.528981:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f84e lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829090.528990:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b4200/0xbef42c0221d6f84e lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787829090.529007:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787829090.529009:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:3.0:1787829090.529093:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829090.529098:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829090.529101:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829090.529107:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11161 00000100:00080000:3.0:1787829090.529115:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829090.529120:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829090.529138:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043302016/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829090.529163:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829090.529166:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829090.529167:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829090.529170:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11161 00000100:00080000:3.0:1787829090.529172:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829090.529175:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829090.529183:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829090.529190:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637b80 x1874663043302144/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829090.529197:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829090.529199:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829090.529201:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829090.529204:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11161 00000100:00080000:3.0:1787829090.529206:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829090.529208:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829090.529211:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829090.529217:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043302272/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829090.529224:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043302016:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787829090.529290:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829090.529322:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829090/real 1787829090] req@ffff9181c5635c00 x1874663043302016/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829115 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829090.529340:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043302016/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829115 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829090.529351:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829090.529356:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829090.529370:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043302016:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829090.529379:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043302144:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829090.529405:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829090.529421:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829090/real 1787829090] req@ffff9181c5637b80 x1874663043302144/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829115 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829090.529435:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637b80 x1874663043302144/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829115 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829090.529445:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829090.529449:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829090.529453:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043302144:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829090.529477:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043302272:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829090.529500:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829090.529512:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829090/real 1787829090] req@ffff9181c5634000 x1874663043302272/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829115 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829090.529526:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043302272/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829115 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829090.529535:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829090.529539:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829090.529543:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043302272:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829090.912893:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829091.936890:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829092.960888:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829093.984919:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829095.008879:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829095.968850:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829095.968862:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829095.968866:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829095.968876:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11162 00000100:00080000:3.0:1787829095.968889:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829095.968897:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829095.968931:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634a80 x1874663043302400/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829095.968977:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829095.968980:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829095.968982:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829095.968986:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11162 00000100:00080000:3.0:1787829095.968989:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829095.968993:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829095.969000:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829095.969017:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043302528/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787829095.969027:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043302400:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787829095.969033:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829095.969038:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829095.969041:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829095.969047:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11162 00000100:00080000:3.0:1787829095.969052:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829095.969057:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829095.969063:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829095.969078:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637b80 x1874663043302656/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787829095.969096:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787829095.969136:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829095/real 1787829095] req@ffff9181c5634a80 x1874663043302400/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829120 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829095.969159:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634a80 x1874663043302400/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829120 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829095.969174:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829095.969179:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829095.969192:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043302400:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787829095.969215:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043302528:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787829095.969237:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787829095.969250:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829095/real 1787829095] req@ffff9181c5634000 x1874663043302528/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829120 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829095.969265:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043302528/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829120 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829095.969273:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829095.969277:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829095.969280:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043302528:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787829095.969285:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043302656:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787829095.969302:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787829095.969315:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829095/real 1787829095] req@ffff9181c5637b80 x1874663043302656/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829120 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829095.969328:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637b80 x1874663043302656/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829120 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829095.969338:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829095.969342:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829095.969346:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043302656:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829096.032883:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829097.056852:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829098.080876:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829099.104866:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829100.128875:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829101.088897:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829101.088908:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829101.088913:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829101.088925:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11167 00000100:00080000:0.0:1787829101.088937:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829101.088944:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829101.088990:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7b80 x1874663043302784/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829101.089024:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829101.089027:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829101.089029:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829101.089032:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11167 00000100:00080000:0.0:1787829101.089034:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829101.089037:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829101.089047:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829101.089057:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7480 x1874663043302912/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829101.089066:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829101.089069:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829101.089071:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829101.089078:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11167 00000100:00080000:0.0:1787829101.089082:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829101.089089:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787829101.089089:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043302784:192.168.203.147@tcp:250:kworker.0 00000008:00100000:0.0:1787829101.089095:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829101.089113:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6680 x1874663043303040/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787829101.089168:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787829101.089199:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829101/real 1787829101] req@ffff9181c55a7b80 x1874663043302784/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829126 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829101.089218:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7b80 x1874663043302784/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829126 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829101.089231:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829101.089236:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829101.089254:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043302784:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787829101.089285:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043302912:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787829101.089324:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787829101.089340:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829101/real 1787829101] req@ffff9181c55a7480 x1874663043302912/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829126 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829101.089355:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7480 x1874663043302912/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829126 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829101.089362:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829101.089365:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829101.089367:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043302912:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787829101.089372:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043303040:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787829101.089390:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787829101.089398:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829101/real 1787829101] req@ffff9181c55a6680 x1874663043303040/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829126 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829101.089406:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6680 x1874663043303040/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829126 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829101.089411:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829101.089413:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829101.089415:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043303040:192.168.203.147@tcp:38:kworker.0 00010000:00010000:0.0:1787829101.152904:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829102.176866:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829103.200871:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829104.224873:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829105.248852:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829106.208831:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829106.208839:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829106.208842:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829106.208849:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11172 00000100:00080000:3.0:1787829106.208858:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829106.208864:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829106.208895:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043303168/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829106.208921:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829106.208926:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829106.208928:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829106.208933:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11172 00000100:00080000:3.0:1787829106.208938:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829106.208953:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787829106.208954:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043303168:192.168.203.147@tcp:250:kworker.0 00000008:00100000:3.0:1787829106.208963:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829106.208977:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043303296/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829106.208986:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829106.208989:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829106.208991:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829106.208994:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11172 00000100:00080000:3.0:1787829106.208997:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829106.209000:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829106.209004:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000800:00000100:1.0:1787829106.209011:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829106.209015:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043303424/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787829106.209058:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829106/real 1787829106] req@ffff9181c5635c00 x1874663043303168/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829131 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829106.209077:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043303168/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829131 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829106.209084:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829106.209087:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829106.209108:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043303168:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787829106.209137:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043303296:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787829106.209162:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787829106.209176:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829106/real 1787829106] req@ffff9181c5636d80 x1874663043303296/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829131 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829106.209188:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043303296/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829131 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829106.209198:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829106.209201:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829106.209204:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043303296:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787829106.209210:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043303424:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787829106.209234:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787829106.209246:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829106/real 1787829106] req@ffff9181c5637800 x1874663043303424/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829131 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829106.209257:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043303424/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829131 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829106.209265:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829106.209269:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829106.209272:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043303424:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829106.272883:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829107.296867:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829108.320845:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829109.344983:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829110.368910:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:3.0:1787829110.816863:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-cliir' failed (-108). 10000000:00000400:3.0:1787829110.816874:0:159735:0:(mgc_request.c:1900:mgc_process_log()) MGC192.168.203.147@tcp: IR log lustre-cliir failed, not fatal: rc = -108 10000000:01000000:3.0:1787829110.816882:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:3.0:1787829110.816885:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff9181c5620000 from 52 10000000:01000000:3.0:1787829110.816889:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:3.0:1787829110.816935:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f855 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787829110.816948:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f855 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787829110.816961:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f855 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787829110.816983:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d225ff00 going to sleep for 0 seconds 00000100:00100000:3.0:1787829110.816994:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043303552:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:3.0:1787829110.817007:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f855 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787829110.817016:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f855 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787829110.817024:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f855 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787829110.817031:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f855 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787829110.817040:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f855 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787829110.817048:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3480000 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:3.0:1787829110.817052:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f855 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787829110.817061:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f855 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1787829110.817068:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f855 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787829110.817075:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:3.0:1787829110.817079:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787829110.817085:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f855 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787829110.817099:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f855 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787829110.817119:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:3.0:1787829110.817123:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:0.0:1787829110.817203:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829110.817210:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829110.817214:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829110.817221:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11177 00000100:00080000:0.0:1787829110.817230:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829110.817237:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829110.817260:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5880 x1874663043303680/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829110.817286:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829110.817291:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829110.817293:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829110.817297:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11177 00000100:00080000:0.0:1787829110.817301:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829110.817307:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829110.817316:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829110.817324:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043303680:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787829110.817332:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7100 x1874663043303808/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829110.817344:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829110.817347:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829110.817349:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829110.817354:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11177 00000100:00080000:0.0:1787829110.817358:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829110.817362:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829110.817367:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829110.817379:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6d80 x1874663043303936/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787829110.817390:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787829110.817429:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829110/real 1787829110] req@ffff9181c55a5880 x1874663043303680/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829135 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829110.817444:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5880 x1874663043303680/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829135 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829110.817452:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829110.817455:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829110.817475:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043303680:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787829110.817491:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043303808:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787829110.817511:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787829110.817521:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829110/real 1787829110] req@ffff9181c55a7100 x1874663043303808/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829135 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829110.817530:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7100 x1874663043303808/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829135 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829110.817536:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829110.817538:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829110.817540:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043303808:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787829110.817544:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043303936:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787829110.817570:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787829110.817579:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829110/real 1787829110] req@ffff9181c55a6d80 x1874663043303936/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829135 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829110.817587:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6d80 x1874663043303936/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829135 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829110.817593:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829110.817595:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829110.817608:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043303936:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787829111.392883:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829112.416896:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829113.440925:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829114.464870:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829115.488886:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829115.936860:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829115.936875:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829115.936879:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829115.936887:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11182 00000100:00080000:0.0:1787829115.936897:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829115.936903:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829115.936934:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5f80 x1874663043304064/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829115.936962:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829115.936967:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829115.936969:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829115.936974:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11182 00000100:00080000:0.0:1787829115.936978:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829115.936983:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829115.936993:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829115.937010:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4380 x1874663043304192/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787829115.937012:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043304064:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787829115.937027:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829115.937038:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829115.937039:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829115.937042:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11182 00000100:00080000:0.0:1787829115.937046:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829115.937049:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829115.937054:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829115.937063:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4700 x1874663043304320/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787829115.937078:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787829115.937109:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829115/real 1787829115] req@ffff9181c55a5f80 x1874663043304064/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829140 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829115.937126:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5f80 x1874663043304064/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829140 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829115.937133:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829115.937136:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829115.937148:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043304064:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787829115.937167:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043304192:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787829115.937185:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787829115.937200:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829115/real 1787829115] req@ffff9181c55a4380 x1874663043304192/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829140 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829115.937212:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4380 x1874663043304192/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829140 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829115.937218:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829115.937221:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829115.937223:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043304192:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787829115.937227:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043304320:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787829115.937238:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787829115.937245:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829115/real 1787829115] req@ffff9181c55a4700 x1874663043304320/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829140 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829115.937255:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4700 x1874663043304320/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829140 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829115.937277:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829115.937281:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829115.937284:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043304320:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829116.512855:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829117.536943:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829118.560894:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829119.584885:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829120.608908:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829121.056870:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829121.056879:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829121.056883:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829121.056892:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11187 00000100:00080000:3.0:1787829121.056901:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829121.056908:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829121.056941:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043304448/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829121.056966:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829121.056969:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829121.056971:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829121.056974:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11187 00000100:00080000:3.0:1787829121.056976:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829121.056979:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829121.056986:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829121.056994:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043304576/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829121.057002:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829121.057005:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829121.057006:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829121.057009:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11187 00000100:00080000:3.0:1787829121.057013:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1787829121.057014:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043304448:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787829121.057018:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829121.057024:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829121.057031:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043304704/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787829121.057072:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787829121.057104:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829121/real 1787829121] req@ffff9181c5636a00 x1874663043304448/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829146 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829121.057123:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043304448/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829146 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829121.057134:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829121.057139:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829121.057156:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043304448:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787829121.057165:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043304576:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787829121.057192:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787829121.057205:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829121/real 1787829121] req@ffff9181c5637100 x1874663043304576/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829146 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829121.057214:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043304576/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829146 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829121.057224:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829121.057227:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829121.057229:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043304576:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787829121.057247:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043304704:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787829121.057261:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787829121.057268:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829121/real 1787829121] req@ffff9181c5634380 x1874663043304704/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829146 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829121.057275:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043304704/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829146 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829121.057280:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829121.057282:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829121.057284:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043304704:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787829121.632879:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829122.656886:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829123.680924:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829124.704897:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829125.728879:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829126.176867:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829126.176878:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829126.176882:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829126.176893:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11192 00000100:00080000:3.0:1787829126.176904:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829126.176911:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829126.176953:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043304832/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829126.176981:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829126.176984:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829126.176985:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829126.176989:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11192 00000100:00080000:3.0:1787829126.176991:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829126.176995:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829126.177005:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829126.177025:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635f80 x1874663043304960/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829126.177037:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829126.177042:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829126.177044:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829126.177051:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11192 00000100:00080000:3.0:1787829126.177056:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1787829126.177060:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043304832:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787829126.177063:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829126.177073:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829126.177088:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043305088/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787829126.177172:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043304960:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787829126.177206:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043305088:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829126.752916:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829127.776877:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829128.800913:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1787829129.249007:0:2378:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:2.0:1787829129.249044:0:2378:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1787829129.249061:0:2378:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:2.0:1787829129.249102:0:2378:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:2.0:1787829129.249112:0:2378:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:1.0:1787829129.249154:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829126/real 1787829129] req@ffff9181c5635500 x1874663043305088/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829151 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829129.249189:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043305088/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829151 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829129.249201:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829129.249205:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829129.249210:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043305088:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787829129.249218:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829126/real 1787829129] req@ffff9181c5637480 x1874663043304832/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829151 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829129.249231:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043304832/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829151 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829129.249240:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829129.249244:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829129.249258:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043304832:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787829129.249266:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829126/real 1787829129] req@ffff9181c5635f80 x1874663043304960/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829151 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829129.249277:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635f80 x1874663043304960/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829151 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829129.249286:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829129.249289:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829129.249293:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043304960:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787829129.824927:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:3.0:1787829130.848871:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-client' failed (-108). 10000000:00020000:3.0:1787829130.848879:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:3.0:1787829130.848884:0:159735:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:3.0:1787829130.848886:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff9181c5620000 from 7 10000000:01000000:3.0:1787829130.848889:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00010000:00010000:1.0:1787829130.848890:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829130.848928:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f85c lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787829130.848937:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f85c lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787829130.848946:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f85c lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787829130.848961:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d225ff00 going to sleep for 0 seconds 00000100:00100000:3.0:1787829130.848970:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043305216:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:3.0:1787829130.848980:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f85c lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787829130.848986:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f85c lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787829130.848992:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f85c lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787829130.848997:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f85c lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787829130.849002:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f85c lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787829130.849006:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3480000 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:3.0:1787829130.849008:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f85c lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787829130.849014:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f85c lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1787829130.849018:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f85c lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787829130.849022:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:3.0:1787829130.849024:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787829130.849029:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f85c lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787829130.849033:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f85c lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787829130.849048:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:3.0:1787829130.849051:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:0.0:1787829130.849095:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829130.849100:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829130.849103:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829130.849108:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11197 00000100:00080000:0.0:1787829130.849115:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829130.849120:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829130.849144:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5c00 x1874663043305344/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829130.849160:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829130.849163:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829130.849164:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829130.849167:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11197 00000100:00080000:0.0:1787829130.849170:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829130.849173:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829130.849180:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829130.849188:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5180 x1874663043305472/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829130.849196:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829130.849200:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:1.0:1787829130.849201:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043305344:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787829130.849202:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829130.849212:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11197 00000100:00080000:0.0:1787829130.849216:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829130.849220:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829130.849226:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829130.849238:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5500 x1874663043305600/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787829130.849281:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787829130.849314:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829130/real 1787829130] req@ffff9181c55a5c00 x1874663043305344/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829155 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829130.849345:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5c00 x1874663043305344/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829155 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829130.849357:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829130.849361:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829130.849377:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043305344:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787829130.849396:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043305472:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787829130.849438:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787829130.849454:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829130/real 1787829130] req@ffff9181c55a5180 x1874663043305472/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829155 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829130.849479:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5180 x1874663043305472/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829155 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829130.849489:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829130.849492:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829130.849501:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043305472:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787829130.849509:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043305600:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787829130.849532:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787829130.849545:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829130/real 1787829130] req@ffff9181c55a5500 x1874663043305600/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829155 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829130.849567:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5500 x1874663043305600/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829155 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829130.849576:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829130.849579:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829130.849583:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043305600:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787829131.872855:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829132.896894:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829133.920860:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829134.944922:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829135.904841:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829135.904852:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829135.904856:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829135.904866:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11202 00000100:00080000:0.0:1787829135.904877:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829135.904885:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829135.904924:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4e00 x1874663043305728/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829135.904953:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829135.904956:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829135.904958:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829135.904963:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11202 00000100:00080000:0.0:1787829135.904968:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829135.904972:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829135.904982:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829135.904982:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043305728:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787829135.905002:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7800 x1874663043305856/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829135.905021:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829135.905025:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829135.905027:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829135.905032:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11202 00000100:00080000:0.0:1787829135.905037:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000800:00000100:1.0:1787829135.905041:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:0.0:1787829135.905046:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829135.905053:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829135.905069:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5500 x1874663043305984/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787829135.905074:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829135/real 1787829135] req@ffff9181c55a4e00 x1874663043305728/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829160 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829135.905100:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4e00 x1874663043305728/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829160 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829135.905109:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829135.905113:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829135.905127:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043305728:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787829135.905145:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043305856:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787829135.905162:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787829135.905176:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829135/real 1787829135] req@ffff9181c55a7800 x1874663043305856/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829160 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829135.905191:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7800 x1874663043305856/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829160 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829135.905201:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829135.905205:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829135.905209:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043305856:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787829135.905216:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043305984:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787829135.905235:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787829135.905244:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829135/real 1787829135] req@ffff9181c55a5500 x1874663043305984/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829160 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829135.905252:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5500 x1874663043305984/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829160 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829135.905257:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829135.905260:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829135.905262:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043305984:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787829135.968902:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829136.992880:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829138.016852:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829139.040887:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829140.064949:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829141.024835:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829141.024844:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829141.024852:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829141.024862:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11207 00000100:00080000:3.0:1787829141.024872:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829141.024878:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829141.024910:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634e00 x1874663043306112/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829141.024931:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829141.024934:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829141.024936:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829141.024939:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11207 00000100:00080000:3.0:1787829141.024941:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829141.024944:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829141.024951:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829141.024965:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635f80 x1874663043306240/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787829141.024967:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043306112:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787829141.024974:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829141.024977:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829141.024978:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829141.024985:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11207 00000100:00080000:3.0:1787829141.024989:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829141.024991:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829141.024994:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829141.025004:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043306368/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787829141.025037:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787829141.025065:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829141/real 1787829141] req@ffff9181c5634e00 x1874663043306112/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829166 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829141.025083:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634e00 x1874663043306112/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829166 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829141.025094:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829141.025099:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829141.025113:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043306112:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787829141.025140:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043306240:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787829141.025166:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787829141.025179:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829141/real 1787829141] req@ffff9181c5635f80 x1874663043306240/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829166 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829141.025190:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635f80 x1874663043306240/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829166 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829141.025195:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829141.025198:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829141.025200:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043306240:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787829141.025204:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043306368:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787829141.025219:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787829141.025230:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829141/real 1787829141] req@ffff9181c5637480 x1874663043306368/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829166 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829141.025238:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043306368/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829166 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829141.025245:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829141.025248:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829141.025252:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043306368:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829141.088893:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829142.112907:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829143.136891:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829144.160890:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829145.184852:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829146.144834:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829146.144854:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829146.144859:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829146.144867:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11212 00000100:00080000:3.0:1787829146.144877:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829146.144884:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829146.144915:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043306496/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829146.144937:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829146.144940:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829146.144941:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829146.144945:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11212 00000100:00080000:3.0:1787829146.144947:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829146.144950:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829146.144956:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829146.144965:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043306624/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829146.144973:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829146.144976:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829146.144977:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1787829146.144981:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043306496:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787829146.144983:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11212 00000100:00080000:3.0:1787829146.144988:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829146.144992:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829146.144995:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829146.145003:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043306752/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787829146.145042:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787829146.145073:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829146/real 1787829146] req@ffff9181c5635500 x1874663043306496/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829171 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829146.145087:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043306496/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829171 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829146.145095:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829146.145099:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829146.145115:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043306496:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787829146.145136:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043306624:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787829146.145159:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787829146.145172:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829146/real 1787829146] req@ffff9181c5634380 x1874663043306624/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829171 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829146.145188:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043306624/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829171 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829146.145195:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829146.145198:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829146.145200:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043306624:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787829146.145206:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043306752:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787829146.145223:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787829146.145236:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829146/real 1787829146] req@ffff9181c5637100 x1874663043306752/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829171 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829146.145249:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043306752/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829171 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829146.145255:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829146.145258:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829146.145261:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043306752:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787829146.208898:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829147.232859:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829148.256913:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829149.280876:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829150.304929:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829151.264846:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829151.264854:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829151.264857:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829151.264865:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11217 10000000:01000000:0.0:1787829151.264874:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'params' failed (-108). 00000100:00080000:3.0:1787829151.264875:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 10000000:00020000:0.0:1787829151.264881:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00080000:3.0:1787829151.264885:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 10000000:01000000:0.0:1787829151.264886:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:0.0:1787829151.264891:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff9181c87b19b8 from 1 10000000:01000000:0.0:1787829151.264893:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00000100:00100000:3.0:1787829151.264920:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043306880/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:0.0:1787829151.264933:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512a00/0xbef42c0221d6f863 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787829151.264942:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e0512a00/0xbef42c0221d6f863 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787829151.264944:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829151.264950:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829151.264951:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829151.264959:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11217 00000100:00080000:3.0:1787829151.264962:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00010000:00010000:0.0:1787829151.264963:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e0512a00/0xbef42c0221d6f863 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787829151.264968:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829151.264979:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829151.264982:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043306880:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787829151.264989:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d01a50c0 going to sleep for 0 seconds 00000100:00100000:3.0:1787829151.264993:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043307136/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787829151.264999:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043307008:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00080000:3.0:1787829151.265011:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:0.0:1787829151.265012:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e0512a00/0xbef42c0221d6f863 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787829151.265017:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829151.265021:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00010000:00010000:0.0:1787829151.265024:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e0512a00/0xbef42c0221d6f863 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787829151.265027:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11217 00000100:00080000:3.0:1787829151.265044:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00010000:00010000:0.0:1787829151.265046:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512a00/0xbef42c0221d6f863 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787829151.265050:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829151.265057:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:0.0:1787829151.265058:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e0512a00/0xbef42c0221d6f863 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787829151.265067:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e0512a00/0xbef42c0221d6f863 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787829151.265071:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043307264/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787829151.265075:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00010000:00010000:0.0:1787829151.265085:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0512a00 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:0.0:1787829151.265090:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512a00/0xbef42c0221d6f863 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787829151.265096:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e0512a00/0xbef42c0221d6f863 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1787829151.265101:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512a00/0xbef42c0221d6f863 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787829151.265104:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:0.0:1787829151.265109:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787829151.265116:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e0512a00/0xbef42c0221d6f863 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787829151.265124:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e0512a00/0xbef42c0221d6f863 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787829151.265125:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829151/real 1787829151] req@ffff9181c5636a00 x1874663043306880/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829176 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 10000000:01000000:0.0:1787829151.265140:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:0.0:1787829151.265144:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:CONNECTING 00000100:00100000:1.0:1787829151.265154:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043306880/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829176 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829151.265163:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829151.265175:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829151.265182:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043306880:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787829151.265199:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043307136:192.168.203.147@tcp:38:kworker.0 00000100:00080000:3.0:1787829151.265208:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829151.265213:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829151.265216:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829151.265221:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11223 00000800:00000100:1.0:1787829151.265224:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1787829151.265229:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829151.265236:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787829151.265239:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829151/real 1787829151] req@ffff9181c5637800 x1874663043307136/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829176 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829151.265250:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043307136/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829176 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829151.265256:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043307392/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829151.265298:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829151.265302:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829151.265306:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043307136:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787829151.265313:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043307264:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787829151.265339:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787829151.265351:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829151/real 1787829151] req@ffff9181c5636d80 x1874663043307264/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829176 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829151.265359:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043307264/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829176 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829151.265364:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829151.265367:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829151.265369:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043307264:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787829151.265378:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043307392:192.168.203.147@tcp:250:kworker.0 00000800:00000100:1.0:1787829151.265388:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787829151.265394:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829151/real 1787829151] req@ffff9181c5635c00 x1874663043307392/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829176 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829151.265401:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043307392/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829176 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829151.265406:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829151.265408:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829151.265418:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043307392:192.168.203.147@tcp:250:kworker.0 00010000:00010000:0.0:1787829151.328883:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829152.352918:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829153.376840:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829154.400923:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829155.424831:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829156.384833:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829156.384841:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829156.384844:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829156.384853:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11223 00000100:00080000:1.0:1787829156.384863:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829156.384877:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787829156.384910:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0e00 x1874663043307520/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829156.384932:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829156.384935:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829156.384936:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829156.384939:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11223 00000100:00080000:1.0:1787829156.384941:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829156.384944:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829156.384951:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829156.384959:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0000 x1874663043307648/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829156.384967:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829156.384969:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829156.384971:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829156.384973:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11223 00000100:00080000:1.0:1787829156.384981:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829156.384985:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829156.384989:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829156.384992:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043307520:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787829156.384999:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1180 x1874663043307776/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829156.385045:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829156.385075:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829156/real 1787829156] req@ffff9181c35b0e00 x1874663043307520/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829181 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829156.385088:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0e00 x1874663043307520/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829181 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829156.385095:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829156.385098:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829156.385111:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043307520:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829156.385119:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043307648:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829156.385138:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829156.385169:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829156/real 1787829156] req@ffff9181c35b0000 x1874663043307648/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829181 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829156.385177:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0000 x1874663043307648/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829181 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829156.385183:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829156.385185:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829156.385187:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043307648:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829156.385204:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043307776:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829156.385225:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829156.385232:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829156/real 1787829156] req@ffff9181c35b1180 x1874663043307776/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829181 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829156.385264:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1180 x1874663043307776/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829181 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829156.385270:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829156.385272:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829156.385274:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043307776:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787829156.448897:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829157.472875:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829158.496913:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829159.520860:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829160.544857:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829161.504854:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829161.504862:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829161.504865:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829161.504873:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11228 00000100:00080000:3.0:1787829161.504883:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829161.504889:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829161.504921:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637b80 x1874663043307904/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829161.504944:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829161.504947:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829161.504948:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829161.504951:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11228 00000100:00080000:3.0:1787829161.504953:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829161.504956:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829161.504963:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829161.504971:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043308032/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829161.504978:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829161.504981:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829161.504983:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829161.504986:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11228 00000100:00080000:3.0:1787829161.504988:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829161.504994:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787829161.504995:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043307904:192.168.203.147@tcp:250:kworker.0 00000008:00100000:3.0:1787829161.504997:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829161.505007:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634a80 x1874663043308160/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829161.505061:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829161.505098:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829161/real 1787829161] req@ffff9181c5637b80 x1874663043307904/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829186 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829161.505119:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637b80 x1874663043307904/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829186 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829161.505131:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829161.505136:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829161.505149:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043307904:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829161.505159:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043308032:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829161.505187:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829161.505201:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829161/real 1787829161] req@ffff9181c5634000 x1874663043308032/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829186 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829161.505228:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043308032/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829186 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829161.505237:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829161.505241:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829161.505245:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043308032:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829161.505270:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043308160:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829161.505297:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829161.505311:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829161/real 1787829161] req@ffff9181c5634a80 x1874663043308160/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829186 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829161.505319:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634a80 x1874663043308160/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829186 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829161.505330:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829161.505332:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829161.505334:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043308160:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829161.568849:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829162.592874:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829163.616840:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829164.640855:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829165.664849:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829166.624849:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829166.624875:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829166.624878:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829166.624889:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11233 00000100:00080000:0.0:1787829166.624899:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829166.624906:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829166.624940:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5180 x1874663043308288/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829166.624965:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829166.624968:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829166.624970:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829166.624975:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11233 00000100:00080000:0.0:1787829166.624979:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829166.624985:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829166.624997:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829166.625013:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043308288:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787829166.625020:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5c00 x1874663043308416/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829166.625037:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829166.625042:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829166.625044:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829166.625050:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11233 00000100:00080000:0.0:1787829166.625065:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829166.625072:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:1.0:1787829166.625093:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:0.0:1787829166.625100:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829166.625122:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4700 x1874663043308544/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787829166.625147:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829166/real 1787829166] req@ffff9181c55a5180 x1874663043308288/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829191 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829166.625195:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5180 x1874663043308288/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829191 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829166.625224:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829166.625227:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829166.625238:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043308288:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787829166.625263:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043308416:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787829166.625283:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787829166.625293:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829166/real 1787829166] req@ffff9181c55a5c00 x1874663043308416/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829191 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829166.625302:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5c00 x1874663043308416/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829191 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829166.625308:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829166.625310:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829166.625313:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043308416:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787829166.625317:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043308544:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787829166.625329:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787829166.625335:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829166/real 1787829166] req@ffff9181c55a4700 x1874663043308544/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829191 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829166.625343:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4700 x1874663043308544/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829191 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829166.625348:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829166.625350:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829166.625352:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043308544:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787829166.688835:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829167.712874:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829168.736873:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829169.760841:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829170.784852:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829171.744849:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 10000000:01000000:3.0:1787829171.744850:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-sptlrpc' failed (-108). 10000000:00020000:3.0:1787829171.744859:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00080000:0.0:1787829171.744860:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829171.744866:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829171.744875:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11238 00000100:00080000:0.0:1787829171.744884:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829171.744890:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829171.744919:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4380 x1874663043308672/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829171.744939:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829171.744942:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829171.744943:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829171.744948:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11238 00000100:00080000:0.0:1787829171.744951:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829171.744955:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829171.744963:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829171.744975:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5f80 x1874663043308800/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787829171.744976:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043308672:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787829171.744990:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829171.744992:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829171.744993:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829171.744996:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11238 00000100:00080000:0.0:1787829171.744998:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829171.745000:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829171.745004:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829171.745013:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6d80 x1874663043308928/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787829171.745038:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787829171.745064:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829171/real 1787829171] req@ffff9181c55a4380 x1874663043308672/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829196 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829171.745077:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4380 x1874663043308672/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829196 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829171.745084:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829171.745087:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829171.745090:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043308672:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787829171.745106:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043308800:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787829171.745120:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787829171.745128:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829171/real 1787829171] req@ffff9181c55a5f80 x1874663043308800/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829196 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829171.745153:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5f80 x1874663043308800/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829196 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829171.745159:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829171.745161:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829171.745163:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043308800:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787829171.745167:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043308928:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787829171.745181:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787829171.745188:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829171/real 1787829171] req@ffff9181c55a6d80 x1874663043308928/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829196 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829171.745196:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6d80 x1874663043308928/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829196 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829171.745201:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829171.745202:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829171.745204:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043308928:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787829171.808885:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829172.832837:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:3.0:1787829173.152819:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:3.0:1787829173.152826:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff9181c5620000 from 341 10000000:01000000:3.0:1787829173.152830:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:3.0:1787829173.152881:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f86a lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787829173.152890:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f86a lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787829173.152900:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f86a lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787829173.152935:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d225ff00 going to sleep for 0 seconds 00000100:00100000:3.0:1787829173.152946:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043309056:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:3.0:1787829173.152960:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f86a lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787829173.152967:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f86a lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787829173.152973:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f86a lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787829173.152979:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f86a lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787829173.152985:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f86a lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787829173.152990:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3480000 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:3.0:1787829173.152994:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f86a lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787829173.153000:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f86a lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1787829173.153005:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f86a lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787829173.153010:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:3.0:1787829173.153013:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787829173.153018:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f86a lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787829173.153025:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f86a lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787829173.153043:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:3.0:1787829173.153046:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:0.0:1787829173.153171:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829173.153177:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829173.153180:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829173.153185:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11243 00000100:00080000:0.0:1787829173.153194:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829173.153199:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829173.153218:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7100 x1874663043309184/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829173.153244:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829173.153247:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829173.153248:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829173.153252:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11243 00000100:00080000:0.0:1787829173.153254:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829173.153257:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829173.153264:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829173.153272:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5880 x1874663043309312/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829173.153280:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829173.153282:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829173.153284:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829173.153286:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11243 00000100:00080000:0.0:1787829173.153288:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829173.153291:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829173.153294:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829173.153301:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043309184:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787829173.153302:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6680 x1874663043309440/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787829173.153368:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787829173.153394:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829173/real 1787829173] req@ffff9181c55a7100 x1874663043309184/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829198 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829173.153406:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7100 x1874663043309184/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829198 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829173.153427:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829173.153430:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829173.153440:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043309184:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787829173.153445:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043309312:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787829173.153465:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787829173.153474:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829173/real 1787829173] req@ffff9181c55a5880 x1874663043309312/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829198 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829173.153483:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5880 x1874663043309312/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829198 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829173.153488:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829173.153491:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829173.153493:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043309312:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787829173.153509:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043309440:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787829173.153520:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787829173.153526:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829173/real 1787829173] req@ffff9181c55a6680 x1874663043309440/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829198 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829173.153534:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6680 x1874663043309440/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829198 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829173.153539:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829173.153541:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829173.153543:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043309440:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829173.856880:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829174.880950:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829175.904948:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829176.928892:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829177.952921:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829178.400928:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829178.400939:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829178.400947:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829178.400956:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11244 00000100:00080000:1.0:1787829178.400965:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829178.400988:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787829178.401019:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3100 x1874663043309568/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829178.401046:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829178.401049:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829178.401050:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829178.401053:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11244 00000100:00080000:1.0:1787829178.401055:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829178.401058:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829178.401066:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829178.401074:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1500 x1874663043309696/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829178.401081:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829178.401100:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829178.401102:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829178.401106:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11244 00000100:00080000:1.0:1787829178.401109:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829178.401112:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829178.401133:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829178.401148:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0380 x1874663043309824/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829178.401157:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043309568:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787829178.401262:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829178.401314:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829178/real 1787829178] req@ffff9181c35b3100 x1874663043309568/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829203 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829178.401334:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3100 x1874663043309568/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829203 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829178.401358:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829178.401364:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829178.401382:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043309568:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829178.401391:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043309696:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829178.401419:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829178.401436:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829178/real 1787829178] req@ffff9181c35b1500 x1874663043309696/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829203 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829178.401462:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1500 x1874663043309696/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829203 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829178.401478:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829178.401482:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829178.401486:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043309696:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829178.401516:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043309824:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829178.401541:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829178.401553:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829178/real 1787829178] req@ffff9181c35b0380 x1874663043309824/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829203 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829178.401567:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0380 x1874663043309824/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829203 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829178.401576:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829178.401580:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829178.401583:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043309824:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787829178.977009:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829180.000907:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829181.024927:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829182.048870:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829183.072919:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829183.520872:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829183.520881:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829183.520884:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829183.520893:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11250 00000100:00080000:0.0:1787829183.520903:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829183.520909:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829183.520940:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7480 x1874663043309952/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829183.520967:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829183.520971:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829183.520973:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829183.520978:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11250 00000100:00080000:0.0:1787829183.520982:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829183.520987:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829183.520997:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829183.521013:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043309952:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787829183.521015:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7b80 x1874663043310080/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829183.521030:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829183.521033:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829183.521034:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829183.521038:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11250 00000100:00080000:0.0:1787829183.521041:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829183.521044:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829183.521064:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829183.521077:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6680 x1874663043310208/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829183.521092:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829183.521125:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829183/real 1787829183] req@ffff9181c55a7480 x1874663043309952/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829208 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829183.521139:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7480 x1874663043309952/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829208 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829183.521147:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829183.521150:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829183.521162:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043309952:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829183.521182:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043310080:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829183.521202:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829183.521214:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829183/real 1787829183] req@ffff9181c55a7b80 x1874663043310080/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829208 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829183.521223:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7b80 x1874663043310080/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829208 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829183.521228:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829183.521231:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829183.521233:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043310080:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829183.521238:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043310208:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829183.521258:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829183.521269:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829183/real 1787829183] req@ffff9181c55a6680 x1874663043310208/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829208 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829183.521282:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6680 x1874663043310208/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829208 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829183.521291:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829183.521295:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829183.521299:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043310208:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829184.096860:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829185.120912:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829186.144879:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829187.168911:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829188.192872:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829188.640849:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829188.640865:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829188.640870:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829188.640881:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11255 00000100:00080000:0.0:1787829188.640893:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829188.640900:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829188.640944:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5880 x1874663043310336/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829188.640971:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829188.640974:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829188.640976:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829188.640979:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11255 00000100:00080000:0.0:1787829188.640981:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829188.640984:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829188.640993:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829188.641003:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7100 x1874663043310464/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829188.641010:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829188.641027:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829188.641030:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829188.641034:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11255 00000100:00080000:0.0:1787829188.641039:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829188.641043:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829188.641047:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829188.641064:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6d80 x1874663043310592/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829188.641071:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043310336:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829188.641172:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043310464:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829188.641203:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043310592:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787829189.216933:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829190.241060:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829191.264983:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1787829191.713119:0:2379:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:1.0:1787829191.713158:0:2379:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787829191.713172:0:2379:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1787829191.713207:0:2379:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1787829191.713214:0:2379:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:2.0:1787829191.713311:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829188/real 1787829191] req@ffff9181c55a6d80 x1874663043310592/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829213 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829191.713353:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6d80 x1874663043310592/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829213 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829191.713368:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829191.713375:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829191.713380:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043310592:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829191.713391:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829188/real 1787829191] req@ffff9181c55a5880 x1874663043310336/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829213 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829191.713405:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5880 x1874663043310336/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829213 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829191.713413:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829191.713418:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829191.713442:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043310336:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829191.713451:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829188/real 1787829191] req@ffff9181c55a7100 x1874663043310464/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829213 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829191.713467:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7100 x1874663043310464/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829213 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829191.713487:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829191.713493:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829191.713499:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043310464:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787829192.288941:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:0.0:1787829193.185018:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-cliir' failed (-108). 10000000:00000400:0.0:1787829193.185030:0:159735:0:(mgc_request.c:1900:mgc_process_log()) MGC192.168.203.147@tcp: IR log lustre-cliir failed, not fatal: rc = -108 10000000:01000000:0.0:1787829193.185040:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:0.0:1787829193.185043:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff9181c5620000 from 52 10000000:01000000:0.0:1787829193.185048:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:0.0:1787829193.185104:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0513000/0xbef42c0221d6f871 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787829193.185118:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e0513000/0xbef42c0221d6f871 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787829193.185133:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e0513000/0xbef42c0221d6f871 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787829193.185158:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d01a50c0 going to sleep for 0 seconds 00000100:00100000:0.0:1787829193.185173:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043310720:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:0.0:1787829193.185188:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e0513000/0xbef42c0221d6f871 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787829193.185198:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e0513000/0xbef42c0221d6f871 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787829193.185207:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0513000/0xbef42c0221d6f871 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787829193.185216:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e0513000/0xbef42c0221d6f871 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787829193.185226:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e0513000/0xbef42c0221d6f871 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787829193.185234:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0513000 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:0.0:1787829193.185238:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e0513000/0xbef42c0221d6f871 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787829193.185248:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e0513000/0xbef42c0221d6f871 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1787829193.185256:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e0513000/0xbef42c0221d6f871 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787829193.185262:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:0.0:1787829193.185266:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787829193.185273:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e0513000/0xbef42c0221d6f871 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787829193.185281:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e0513000/0xbef42c0221d6f871 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787829193.185305:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:0.0:1787829193.185309:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:3.0:1787829193.185401:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829193.185414:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829193.185419:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829193.185431:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11260 00000100:00080000:3.0:1787829193.185444:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829193.185454:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829193.185507:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634a80 x1874663043310848/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829193.185541:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829193.185546:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829193.185549:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829193.185556:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11260 00000100:00080000:3.0:1787829193.185578:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1787829193.185590:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043310848:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787829193.185593:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829193.185608:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829193.185631:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043310976/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829193.185647:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829193.185669:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829193.185671:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829193.185677:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11260 00000800:00000100:2.0:1787829193.185685:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1787829193.185688:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829193.185697:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829193.185705:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829193.185725:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829193/real 1787829193] req@ffff9181c5634a80 x1874663043310848/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829218 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829193.185727:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637b80 x1874663043311104/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829193.185803:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634a80 x1874663043310848/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829218 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829193.185816:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829193.185822:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829193.185846:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043310848:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829193.185870:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043310976:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829193.185896:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829193.185909:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829193/real 1787829193] req@ffff9181c5634000 x1874663043310976/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829218 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829193.185918:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043310976/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829218 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829193.185924:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829193.185926:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829193.185928:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043310976:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829193.185934:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043311104:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829193.185957:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829193.185965:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829193/real 1787829193] req@ffff9181c5637b80 x1874663043311104/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829218 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829193.185997:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637b80 x1874663043311104/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829218 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829193.186004:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829193.186006:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829193.186009:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043311104:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787829193.312994:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829194.336885:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829195.361000:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829196.384861:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829197.408873:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829198.369059:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829198.369070:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829198.369074:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829198.369085:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11264 00000100:00080000:1.0:1787829198.369098:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829198.369105:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787829198.369158:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1c00 x1874663043311232/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829198.369207:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829198.369210:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829198.369212:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829198.369217:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11264 00000100:00080000:1.0:1787829198.369220:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829198.369224:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829198.369233:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829198.369248:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0380 x1874663043311360/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829198.369259:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829198.369262:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829198.369264:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829198.369268:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11264 00000100:00080000:1.0:1787829198.369271:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829198.369280:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787829198.369281:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043311232:192.168.203.147@tcp:250:kworker.0 00000008:00100000:1.0:1787829198.369285:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829198.369300:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1500 x1874663043311488/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829198.369381:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829198.369423:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829198/real 1787829198] req@ffff9181c35b1c00 x1874663043311232/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829223 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829198.369443:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1c00 x1874663043311232/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829223 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829198.369452:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829198.369455:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829198.369468:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043311232:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829198.369475:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043311360:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829198.369497:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829198.369510:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829198/real 1787829198] req@ffff9181c35b0380 x1874663043311360/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829223 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829198.369541:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0380 x1874663043311360/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829223 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829198.369550:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829198.369553:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829198.369556:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043311360:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829198.369579:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043311488:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829198.369607:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829198.369616:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829198/real 1787829198] req@ffff9181c35b1500 x1874663043311488/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829223 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829198.369643:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1500 x1874663043311488/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829223 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829198.369659:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829198.369661:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829198.369663:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043311488:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787829198.432983:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829199.457049:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829200.480904:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829201.504873:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829202.528935:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829203.488827:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829203.488837:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829203.488841:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829203.488851:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11270 00000100:00080000:1.0:1787829203.488863:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829203.488872:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787829203.488906:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3100 x1874663043311616/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829203.488973:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829203.488976:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829203.488978:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829203.488982:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11270 00000100:00080000:1.0:1787829203.488985:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829203.488989:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829203.488998:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829203.489029:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1180 x1874663043311744/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787829203.489039:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043311616:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787829203.489043:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829203.489052:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829203.489055:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829203.489061:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11270 00000100:00080000:1.0:1787829203.489066:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829203.489073:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829203.489080:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829203.489098:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0000 x1874663043311872/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829203.489112:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829203.489149:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829203/real 1787829203] req@ffff9181c35b3100 x1874663043311616/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829228 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829203.489169:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3100 x1874663043311616/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829228 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829203.489177:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829203.489181:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829203.489211:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043311616:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787829203.489232:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043311744:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787829203.489255:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829203.489266:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829203/real 1787829203] req@ffff9181c35b1180 x1874663043311744/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829228 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829203.489278:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1180 x1874663043311744/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829228 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829203.489286:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829203.489289:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829203.489292:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043311744:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787829203.489296:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043311872:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787829203.489307:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829203.489313:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829203/real 1787829203] req@ffff9181c35b0000 x1874663043311872/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829228 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829203.489322:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0000 x1874663043311872/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829228 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829203.489327:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829203.489329:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829203.489331:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043311872:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787829203.552896:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829204.576877:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829205.600885:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829206.624885:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829207.648922:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829208.608843:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829208.608854:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829208.608859:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829208.608870:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11275 00000100:00080000:0.0:1787829208.608893:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829208.608900:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829208.608937:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5f80 x1874663043312000/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829208.608966:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829208.608970:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829208.608972:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829208.608976:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11275 00000100:00080000:0.0:1787829208.608980:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829208.608984:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829208.608994:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829208.609007:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4380 x1874663043312128/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829208.609014:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043312000:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787829208.609024:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829208.609029:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829208.609031:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829208.609036:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11275 00000100:00080000:0.0:1787829208.609040:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829208.609045:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829208.609050:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829208.609063:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4700 x1874663043312256/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829208.609076:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829208.609113:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829208/real 1787829208] req@ffff9181c55a5f80 x1874663043312000/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829233 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829208.609133:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5f80 x1874663043312000/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829233 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829208.609149:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829208.609152:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829208.609163:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043312000:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829208.609192:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043312128:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829208.609211:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829208.609221:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829208/real 1787829208] req@ffff9181c55a4380 x1874663043312128/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829233 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829208.609229:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4380 x1874663043312128/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829233 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829208.609235:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829208.609237:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829208.609239:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043312128:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829208.609243:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043312256:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829208.609254:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829208.609260:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829208/real 1787829208] req@ffff9181c55a4700 x1874663043312256/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829233 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829208.609273:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4700 x1874663043312256/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829233 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829208.609279:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829208.609281:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829208.609283:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043312256:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787829208.672911:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829209.696857:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829210.720886:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829211.744866:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829212.768933:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787829213.216892:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-client' failed (-108). 10000000:00020000:1.0:1787829213.216901:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:1.0:1787829213.216906:0:159735:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:1.0:1787829213.216908:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff9181c5620000 from 7 10000000:01000000:1.0:1787829213.216911:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00010000:00010000:1.0:1787829213.216950:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f878 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829213.216959:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f878 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829213.216969:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f878 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787829213.216985:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215db40 going to sleep for 0 seconds 00000100:00100000:1.0:1787829213.216996:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043312384:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787829213.217006:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f878 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829213.217012:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f878 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829213.217017:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f878 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829213.217022:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f878 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829213.217027:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f878 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829213.217031:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4a00 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787829213.217033:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f878 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829213.217038:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f878 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787829213.217043:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f878 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787829213.217047:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:1.0:1787829213.217049:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787829213.217054:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f878 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829213.217062:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f878 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787829213.217078:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787829213.217080:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:0.0:1787829213.217146:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829213.217150:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829213.217152:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829213.217157:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11280 00000100:00080000:0.0:1787829213.217165:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829213.217170:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829213.217195:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5c00 x1874663043312512/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829213.217213:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829213.217216:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829213.217217:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829213.217220:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11280 00000100:00080000:0.0:1787829213.217222:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829213.217225:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829213.217232:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829213.217240:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5180 x1874663043312640/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829213.217248:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829213.217251:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829213.217254:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1787829213.217255:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043312512:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787829213.217259:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11280 00000100:00080000:0.0:1787829213.217266:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829213.217269:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829213.217273:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829213.217281:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5500 x1874663043312768/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829213.217325:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829213.217352:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829213/real 1787829213] req@ffff9181c55a5c00 x1874663043312512/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829238 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829213.217364:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5c00 x1874663043312512/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829238 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829213.217372:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829213.217375:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829213.217388:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043312512:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829213.217405:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043312640:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829213.217422:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829213.217430:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829213/real 1787829213] req@ffff9181c55a5180 x1874663043312640/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829238 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829213.217438:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5180 x1874663043312640/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829238 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829213.217444:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829213.217447:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829213.217449:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043312640:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829213.217453:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043312768:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829213.217463:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829213.217469:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829213/real 1787829213] req@ffff9181c55a5500 x1874663043312768/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829238 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829213.217477:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5500 x1874663043312768/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829238 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829213.217482:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829213.217483:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829213.217485:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043312768:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787829213.792969:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829214.816987:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829215.840904:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829216.864943:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829217.888892:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829218.336916:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829218.336924:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829218.336928:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829218.336942:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11284 00000100:00080000:3.0:1787829218.336950:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829218.336956:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829218.336986:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043312896/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829218.337006:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829218.337010:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829218.337011:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829218.337014:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11284 00000100:00080000:3.0:1787829218.337017:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829218.337020:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829218.337027:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829218.337035:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043313024/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829218.337043:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829218.337045:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829218.337046:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829218.337049:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11284 00000100:00080000:3.0:1787829218.337051:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829218.337054:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829218.337058:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829218.337060:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043312896:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829218.337066:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043313152/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829218.337127:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829218.337157:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829218/real 1787829218] req@ffff9181c5635c00 x1874663043312896/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829243 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829218.337173:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043312896/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829243 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829218.337184:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829218.337189:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829218.337207:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043312896:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829218.337216:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043313024:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829218.337244:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829218.337259:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829218/real 1787829218] req@ffff9181c5636d80 x1874663043313024/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829243 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829218.337272:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043313024/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829243 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829218.337288:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829218.337290:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829218.337293:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043313024:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829218.337312:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043313152:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829218.337327:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829218.337334:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829218/real 1787829218] req@ffff9181c5637800 x1874663043313152/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829243 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829218.337342:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043313152/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829243 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829218.337348:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829218.337350:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829218.337352:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043313152:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787829218.912972:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829219.936901:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829220.960916:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829221.984876:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829223.008938:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829223.456844:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829223.456852:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829223.456856:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829223.456865:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11290 00000100:00080000:0.0:1787829223.456873:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829223.456880:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829223.456911:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7800 x1874663043313280/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829223.456931:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829223.456934:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829223.456935:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829223.456938:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11290 00000100:00080000:0.0:1787829223.456940:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829223.456943:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829223.456950:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829223.456959:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4e00 x1874663043313408/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829223.456967:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829223.456971:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:2.0:1787829223.456972:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043313280:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787829223.456973:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829223.456984:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11290 00000100:00080000:0.0:1787829223.456987:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829223.456989:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829223.456993:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829223.457002:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5500 x1874663043313536/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829223.457042:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829223.457069:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829223/real 1787829223] req@ffff9181c55a7800 x1874663043313280/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829248 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829223.457082:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7800 x1874663043313280/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829248 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829223.457090:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829223.457093:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829223.457104:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043313280:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829223.457122:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043313408:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829223.457149:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829223.457166:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829223/real 1787829223] req@ffff9181c55a4e00 x1874663043313408/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829248 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829223.457181:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4e00 x1874663043313408/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829248 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829223.457188:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829223.457190:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829223.457192:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043313408:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829223.457197:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043313536:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829223.457212:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829223.457221:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829223/real 1787829223] req@ffff9181c55a5500 x1874663043313536/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829248 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829223.457233:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5500 x1874663043313536/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829248 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829223.457241:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829223.457245:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829223.457249:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043313536:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787829224.032917:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829225.056901:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829226.080921:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829227.104901:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829228.128918:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829228.576891:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829228.576902:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829228.576906:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829228.576919:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11295 00000100:00080000:3.0:1787829228.576930:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829228.576937:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829228.576983:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043313664/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829228.577010:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829228.577013:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829228.577015:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829228.577018:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11295 00000100:00080000:3.0:1787829228.577021:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829228.577024:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829228.577032:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829228.577041:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043313792/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829228.577049:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829228.577052:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829228.577053:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829228.577056:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11295 00000100:00080000:3.0:1787829228.577059:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829228.577062:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829228.577065:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829228.577071:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043313920/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829228.577097:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043313664:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787829228.577163:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829228.577205:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829228/real 1787829228] req@ffff9181c5636a00 x1874663043313664/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829253 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829228.577219:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043313664/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829253 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829228.577228:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829228.577231:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829228.577247:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043313664:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829228.577253:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043313792:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829228.577271:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829228.577281:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829228/real 1787829228] req@ffff9181c5637100 x1874663043313792/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829253 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829228.577295:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043313792/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829253 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829228.577304:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829228.577309:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829228.577313:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043313792:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829228.577339:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043313920:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829228.577362:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829228.577371:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829228/real 1787829228] req@ffff9181c5634380 x1874663043313920/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829253 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829228.577384:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043313920/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829253 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829228.577389:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829228.577392:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829228.577394:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043313920:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829229.152904:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829230.176878:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829231.200839:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829232.224856:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829233.248869:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787829233.696889:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'params' failed (-108). 10000000:00020000:1.0:1787829233.696897:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:1.0:1787829233.696902:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:1.0:1787829233.696904:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff9181c87b19b8 from 1 10000000:01000000:1.0:1787829233.696907:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00000100:00080000:0.0:1787829233.696911:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829233.696919:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829233.696923:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829233.696935:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11300 00000100:00080000:0.0:1787829233.696945:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829233.696953:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:1.0:1787829233.696976:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f87f lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787829233.696979:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5180 x1874663043314176/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787829233.696998:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f87f lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829233.697010:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f87f lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787829233.697015:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829233.697019:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829233.697021:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829233.697025:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11300 00000100:00100000:1.0:1787829233.697029:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215db40 going to sleep for 0 seconds 00000100:00080000:0.0:1787829233.697030:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829233.697037:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787829233.697041:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043314048:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000008:00100000:0.0:1787829233.697050:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:1.0:1787829233.697057:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f87f lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829233.697065:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f87f lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787829233.697070:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5c00 x1874663043314304/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787829233.697072:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f87f lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787829233.697078:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043314176:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787829233.697093:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f87f lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787829233.697103:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:1.0:1787829233.697103:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f87f lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787829233.697114:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829233.697119:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00010000:00010000:1.0:1787829233.697119:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4a00 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787829233.697127:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f87f lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787829233.697130:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11300 00000100:00080000:0.0:1787829233.697139:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00010000:00010000:1.0:1787829233.697145:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f87f lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000800:00000100:2.0:1787829233.697147:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:0.0:1787829233.697149:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829233.697162:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 10000000:00010000:1.0:1787829233.697163:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f87f lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787829233.697173:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1787829233.697177:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00000100:00100000:0.0:1787829233.697184:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4700 x1874663043314432/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787829233.697186:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f87f lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787829233.697187:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829233/real 1787829233] req@ffff9181c55a5180 x1874663043314176/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829258 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:1.0:1787829233.697211:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f87f lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787829233.697220:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5180 x1874663043314176/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829258 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829233.697235:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829233.697239:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829233.697243:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043314176:192.168.203.147@tcp:250:kworker.0 10000000:01000000:1.0:1787829233.697245:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787829233.697249:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00100000:2.0:1787829233.697261:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043314304:192.168.203.147@tcp:38:kworker.0 00000100:00080000:0.0:1787829233.697286:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:2.0:1787829233.697289:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:0.0:1787829233.697291:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829233.697294:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829233.697300:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11305 00000100:00080000:0.0:1787829233.697304:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829233.697310:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787829233.697311:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829233/real 1787829233] req@ffff9181c55a5c00 x1874663043314304/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829258 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829233.697322:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4380 x1874663043314560/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829233.697339:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5c00 x1874663043314304/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829258 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829233.697349:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829233.697352:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829233.697357:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043314304:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829233.697363:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043314432:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829233.697379:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829233.697387:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829233/real 1787829233] req@ffff9181c55a4700 x1874663043314432/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829258 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829233.697395:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4700 x1874663043314432/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829258 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829233.697400:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829233.697403:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829233.697405:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043314432:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829233.697414:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043314560:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787829233.697424:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829233.697429:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829233/real 1787829233] req@ffff9181c55a4380 x1874663043314560/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829258 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829233.697437:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4380 x1874663043314560/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829258 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829233.697446:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829233.697448:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829233.697458:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043314560:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787829234.272870:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829235.296837:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829236.320862:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829237.344856:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829238.368849:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829238.816856:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829238.816867:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829238.816871:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829238.816880:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11305 00000100:00080000:3.0:1787829238.816891:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829238.816898:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829238.816932:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043314688/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829238.816958:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829238.816962:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829238.816963:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829238.816968:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11305 00000100:00080000:3.0:1787829238.816971:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829238.816975:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829238.816985:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829238.816997:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043314816/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829238.817009:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829238.817012:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829238.817014:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829238.817020:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11305 00000100:00100000:2.0:1787829238.817024:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043314688:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787829238.817025:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829238.817033:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829238.817039:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829238.817050:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635f80 x1874663043314944/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829238.817098:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829238.817142:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829238/real 1787829238] req@ffff9181c5635500 x1874663043314688/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829263 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829238.817161:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043314688/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829263 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829238.817173:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829238.817178:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829238.817204:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043314688:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829238.817228:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043314816:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829238.817255:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829238.817271:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829238/real 1787829238] req@ffff9181c5637480 x1874663043314816/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829263 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829238.817285:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043314816/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829263 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829238.817301:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829238.817305:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829238.817309:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043314816:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829238.817316:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043314944:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829238.817338:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829238.817351:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829238/real 1787829238] req@ffff9181c5635f80 x1874663043314944/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829263 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829238.817366:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635f80 x1874663043314944/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829263 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829238.817375:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829238.817379:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829238.817383:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043314944:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787829239.392875:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829240.416904:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829241.440944:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829242.464869:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829243.488936:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829243.936868:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829243.936877:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829243.936880:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829243.936889:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11310 00000100:00080000:0.0:1787829243.936899:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829243.936906:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829243.936945:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5f80 x1874663043315072/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829243.936972:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829243.936975:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829243.936976:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829243.936980:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11310 00000100:00080000:0.0:1787829243.936984:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829243.936989:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829243.936999:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829243.937015:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043315072:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787829243.937030:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7100 x1874663043315200/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829243.937039:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829243.937042:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829243.937043:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829243.937046:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11310 00000100:00080000:0.0:1787829243.937050:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829243.937053:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829243.937057:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829243.937065:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5880 x1874663043315328/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829243.937085:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829243.937115:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829243/real 1787829243] req@ffff9181c55a5f80 x1874663043315072/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829268 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829243.937128:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5f80 x1874663043315072/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829268 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829243.937136:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829243.937138:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829243.937151:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043315072:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829243.937177:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043315200:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829243.937195:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829243.937206:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829243/real 1787829243] req@ffff9181c55a7100 x1874663043315200/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829268 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829243.937216:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7100 x1874663043315200/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829268 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829243.937222:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829243.937225:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829243.937227:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043315200:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829243.937231:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043315328:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829243.937242:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829243.937247:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829243/real 1787829243] req@ffff9181c55a5880 x1874663043315328/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829268 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829243.937255:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5880 x1874663043315328/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829268 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829243.937260:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829243.937262:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829243.937263:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043315328:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829244.512992:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829245.536973:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829246.560882:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829247.584957:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829248.608899:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829249.056921:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829249.056939:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829249.056946:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829249.056964:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11315 00000100:00080000:3.0:1787829249.056981:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829249.056994:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829249.057096:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634e00 x1874663043315456/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829249.057147:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829249.057166:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829249.057170:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829249.057178:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11315 00000100:00080000:3.0:1787829249.057184:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829249.057192:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829249.057210:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829249.057249:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043315456:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829249.057252:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635f80 x1874663043315584/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829249.057282:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829249.057289:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829249.057294:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829249.057301:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11315 00000100:00080000:3.0:1787829249.057321:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829249.057331:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:2.0:1787829249.057337:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:3.0:1787829249.057346:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829249.057390:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043315712/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829249.057402:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829249/real 1787829249] req@ffff9181c5634e00 x1874663043315456/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829274 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829249.057465:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634e00 x1874663043315456/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829274 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829249.057488:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829249.057495:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829249.057523:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043315456:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829249.057553:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043315584:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829249.057585:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829249.057658:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829249/real 1787829249] req@ffff9181c5635f80 x1874663043315584/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829274 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829249.057677:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635f80 x1874663043315584/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829274 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829249.057699:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829249.057704:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829249.057708:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043315584:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829249.057716:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043315712:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829249.057743:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829249.057756:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829249/real 1787829249] req@ffff9181c5637480 x1874663043315712/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829274 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829249.057809:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043315712/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829274 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829249.057820:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829249.057824:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829249.057840:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043315712:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787829249.632903:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829250.656915:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829251.680900:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829252.704852:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829253.728862:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829254.176890:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829254.176908:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829254.176913:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829254.176927:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11320 00000100:00080000:3.0:1787829254.176940:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829254.176964:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 10000000:01000000:1.0:1787829254.176989:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-sptlrpc' failed (-108). 10000000:00020000:1.0:1787829254.176995:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00100000:3.0:1787829254.177037:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043315840/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829254.177071:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829254.177077:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829254.177079:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829254.177084:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11320 00000100:00080000:3.0:1787829254.177089:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829254.177095:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829254.177108:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829254.177152:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043315968/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829254.177164:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829254.177169:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829254.177171:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829254.177176:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11320 00000100:00100000:2.0:1787829254.177183:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043315840:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787829254.177183:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829254.177191:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829254.177197:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829254.177208:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043316096/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829254.177311:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043315968:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829254.177340:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043316096:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829254.752857:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829255.776907:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787829256.160824:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:1.0:1787829256.160833:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff9181c5620000 from 341 10000000:01000000:1.0:1787829256.160840:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:1.0:1787829256.160901:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f886 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829256.160912:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f886 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829256.160923:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f886 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787829256.160944:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215db40 going to sleep for 0 seconds 00000100:00100000:1.0:1787829256.160956:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043316224:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787829256.160973:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f886 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829256.160979:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f886 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829256.160984:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f886 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829256.160994:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f886 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829256.161000:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f886 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829256.161005:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4e00 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787829256.161008:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f886 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829256.161013:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f886 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787829256.161018:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f886 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787829256.161022:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:1.0:1787829256.161024:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787829256.161030:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f886 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829256.161034:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f886 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787829256.161064:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787829256.161067:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:CONNECTING 00010000:00010000:1.0:1787829256.800913:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1787829257.249030:0:2376:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:1.0:1787829257.249072:0:2376:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787829257.249091:0:2376:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1787829257.249130:0:2376:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1787829257.249141:0:2376:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:2.0:1787829257.249224:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829254/real 1787829257] req@ffff9181c5637100 x1874663043316096/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829279 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829257.249275:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043316096/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829279 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829257.249291:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829257.249297:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829257.249303:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043316096:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829257.249313:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829254/real 1787829257] req@ffff9181c5635500 x1874663043315840/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829279 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829257.249329:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043315840/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829279 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829257.249338:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829257.249372:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829257.249396:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043315840:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829257.249407:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829254/real 1787829257] req@ffff9181c5634380 x1874663043315968/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829279 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829257.249422:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043315968/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829279 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829257.249432:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829257.249434:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829257.249443:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00080000:3.0:1787829257.249445:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00100000:2.0:1787829257.249451:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043315968:192.168.203.147@tcp:38:kworker.0 00000100:00080000:3.0:1787829257.249452:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829257.249466:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11325 00000100:00080000:3.0:1787829257.249477:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829257.249487:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829257.250136:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043316352/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829257.250163:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829257.250167:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829257.250169:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829257.250174:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11325 00000100:00080000:3.0:1787829257.250180:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829257.250186:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829257.250200:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829257.250214:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043316352:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829257.250218:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043316480/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829257.250233:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829257.250238:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829257.250240:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829257.250244:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11325 00000100:00080000:3.0:1787829257.250248:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829257.250254:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829257.250260:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829257.250274:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043316608/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829257.250298:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829257.250334:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829257/real 1787829257] req@ffff9181c5636a00 x1874663043316352/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829282 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829257.250348:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043316352/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829282 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829257.250357:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829257.250360:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829257.250377:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043316352:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829257.250398:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043316480:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829257.250425:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829257.250440:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829257/real 1787829257] req@ffff9181c5637800 x1874663043316480/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829282 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829257.250452:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043316480/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829282 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829257.250461:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829257.250465:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829257.250469:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043316480:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829257.250477:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043316608:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829257.250495:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829257.250503:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829257/real 1787829257] req@ffff9181c5636d80 x1874663043316608/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829282 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829257.250511:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043316608/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829282 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829257.250517:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829257.250519:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829257.250521:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043316608:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787829257.824947:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829258.848986:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829259.872979:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829260.896885:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829261.920930:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829262.368808:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829262.368817:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829262.368821:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829262.368828:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11328 00000100:00080000:3.0:1787829262.368838:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829262.368844:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829262.368875:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043316736/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829262.368896:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829262.368900:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829262.368901:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829262.368906:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11328 00000100:00080000:3.0:1787829262.368910:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829262.368914:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829262.368923:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829262.368938:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637b80 x1874663043316864/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829262.368949:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829262.368953:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829262.368956:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1787829262.368969:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043316736:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787829262.368970:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11328 00000100:00080000:3.0:1787829262.368978:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829262.368984:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829262.368990:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829262.369002:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043316992/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829262.369032:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829262.369064:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829262/real 1787829262] req@ffff9181c5635c00 x1874663043316736/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829287 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829262.369076:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043316736/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829287 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829262.369085:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829262.369087:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829262.369100:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043316736:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829262.369105:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043316864:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829262.369124:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829262.369135:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829262/real 1787829262] req@ffff9181c5637b80 x1874663043316864/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829287 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829262.369149:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637b80 x1874663043316864/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829287 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829262.369158:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829262.369162:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829262.369166:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043316864:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829262.369189:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043316992:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829262.369209:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829262.369217:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829262/real 1787829262] req@ffff9181c5634000 x1874663043316992/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829287 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829262.369225:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043316992/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829287 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829262.369231:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829262.369233:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829262.369235:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043316992:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829262.944871:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829263.968869:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829264.992986:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829266.016924:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829267.040862:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829267.488869:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829267.488883:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829267.488887:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829267.488900:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11334 00000100:00080000:3.0:1787829267.488936:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829267.488944:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829267.489000:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634a80 x1874663043317120/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829267.489035:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829267.489040:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829267.489042:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829267.489048:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11334 00000100:00080000:3.0:1787829267.489053:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829267.489059:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829267.489070:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829267.489090:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043317248/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829267.489128:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043317120:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787829267.489131:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829267.489137:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829267.489139:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829267.489145:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11334 00000100:00080000:3.0:1787829267.489150:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829267.489154:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829267.489160:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829267.489175:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637b80 x1874663043317376/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829267.489210:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829267.489253:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829267/real 1787829267] req@ffff9181c5634a80 x1874663043317120/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829292 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829267.489277:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634a80 x1874663043317120/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829292 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829267.489302:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829267.489305:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829267.489318:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043317120:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829267.489338:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043317248:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829267.489376:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829267.489395:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829267/real 1787829267] req@ffff9181c5634000 x1874663043317248/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829292 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829267.489407:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043317248/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829292 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829267.489415:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829267.489417:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829267.489420:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043317248:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829267.489425:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043317376:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829267.489438:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829267.489446:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829267/real 1787829267] req@ffff9181c5637b80 x1874663043317376/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829292 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829267.489456:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637b80 x1874663043317376/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829292 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829267.489502:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829267.489505:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829267.489507:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043317376:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787829268.064880:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829269.088888:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829270.112868:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829271.136921:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829272.160893:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829272.608838:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829272.608848:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829272.608852:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829272.608863:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11339 00000100:00080000:0.0:1787829272.608874:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829272.608880:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829272.608916:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6d80 x1874663043317504/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829272.608948:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829272.608951:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829272.608953:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829272.608958:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11339 00000100:00080000:0.0:1787829272.608962:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829272.608967:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829272.608978:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829272.608986:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043317504:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787829272.608993:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6680 x1874663043317632/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829272.609007:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829272.609010:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829272.609012:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829272.609016:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11339 00000100:00080000:0.0:1787829272.609019:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829272.609024:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829272.609029:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829272.609042:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7b80 x1874663043317760/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829272.609044:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829272.609080:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829272/real 1787829272] req@ffff9181c55a6d80 x1874663043317504/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829297 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829272.609094:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6d80 x1874663043317504/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829297 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829272.609102:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829272.609112:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829272.609122:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043317504:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829272.609143:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043317632:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829272.609162:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829272.609177:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829272/real 1787829272] req@ffff9181c55a6680 x1874663043317632/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829297 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829272.609186:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6680 x1874663043317632/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829297 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829272.609192:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829272.609194:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829272.609196:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043317632:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829272.609201:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043317760:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829272.609214:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829272.609220:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829272/real 1787829272] req@ffff9181c55a7b80 x1874663043317760/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829297 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829272.609228:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7b80 x1874663043317760/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829297 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829272.609233:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829272.609235:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829272.609237:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043317760:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829273.184889:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829274.208872:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829275.232886:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787829276.192863:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-cliir' failed (-108). 10000000:00000400:1.0:1787829276.192871:0:159735:0:(mgc_request.c:1900:mgc_process_log()) MGC192.168.203.147@tcp: IR log lustre-cliir failed, not fatal: rc = -108 10000000:01000000:1.0:1787829276.192877:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:1.0:1787829276.192879:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff9181c5620000 from 52 10000000:01000000:1.0:1787829276.192882:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:1.0:1787829276.192920:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f88d lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829276.192931:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f88d lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829276.192944:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f88d lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787829276.192965:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215db40 going to sleep for 0 seconds 00000100:00100000:1.0:1787829276.192977:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043317888:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787829276.192990:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f88d lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829276.193000:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f88d lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829276.193005:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f88d lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829276.193010:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f88d lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829276.193015:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f88d lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829276.193020:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4a00 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787829276.193022:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f88d lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829276.193027:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f88d lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787829276.193032:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f88d lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787829276.193036:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1787829276.193038:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787829276.193043:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f88d lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829276.193047:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f88d lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787829276.193067:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787829276.193070:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:3.0:1787829276.193128:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829276.193134:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829276.193136:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829276.193141:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11344 00000100:00080000:3.0:1787829276.193150:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829276.193154:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829276.193171:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043318016/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829276.193189:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829276.193192:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829276.193193:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829276.193196:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11344 00000100:00080000:3.0:1787829276.193198:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829276.193201:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829276.193213:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829276.193222:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043318144/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829276.193230:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829276.193232:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829276.193233:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829276.193236:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11344 00000100:00080000:3.0:1787829276.193238:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829276.193241:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829276.193244:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829276.193251:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043318016:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829276.193253:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043318272/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829276.193321:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829276.193355:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829276/real 1787829276] req@ffff9181c5635c00 x1874663043318016/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829301 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829276.193374:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043318016/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829301 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829276.193386:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829276.193390:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829276.193403:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043318016:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829276.193424:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043318144:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829276.193451:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829276.193464:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829276/real 1787829276] req@ffff9181c5636d80 x1874663043318144/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829301 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829276.193478:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043318144/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829301 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829276.193486:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829276.193489:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829276.193493:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043318144:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829276.193511:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043318272:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829276.193528:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829276.193539:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829276/real 1787829276] req@ffff9181c5637800 x1874663043318272/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829301 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829276.193552:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043318272/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829301 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829276.193561:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829276.193565:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829276.193569:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043318272:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787829276.256917:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829277.280947:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829278.304958:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829279.328911:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829280.352948:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829281.312870:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829281.312881:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829281.312885:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829281.312896:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11347 00000100:00080000:0.0:1787829281.312907:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829281.312913:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829281.312963:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7480 x1874663043318400/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829281.312988:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829281.312992:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829281.312994:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829281.312997:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11347 00000100:00080000:0.0:1787829281.313000:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829281.313004:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829281.313012:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829281.313025:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7b80 x1874663043318528/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829281.313035:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829281.313038:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829281.313040:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829281.313043:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11347 00000100:00080000:0.0:1787829281.313046:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829281.313049:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829281.313058:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829281.313058:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043318400:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787829281.313070:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6680 x1874663043318656/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829281.313143:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829281.313200:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829281/real 1787829281] req@ffff9181c55a7480 x1874663043318400/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829306 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829281.313236:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7480 x1874663043318400/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829306 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829281.313259:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829281.313265:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829281.313284:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043318400:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829281.313294:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043318528:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829281.313335:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829281.313356:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829281/real 1787829281] req@ffff9181c55a7b80 x1874663043318528/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829306 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829281.313395:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7b80 x1874663043318528/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829306 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829281.313404:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829281.313407:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829281.313410:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043318528:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829281.313444:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043318656:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829281.313488:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829281.313506:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829281/real 1787829281] req@ffff9181c55a6680 x1874663043318656/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829306 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829281.313524:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6680 x1874663043318656/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829306 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829281.313536:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829281.313542:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829281.313563:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043318656:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829281.376936:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829282.400852:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829283.424862:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829284.448906:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829285.472958:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829286.432837:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829286.432845:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829286.432848:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829286.432857:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11353 00000100:00080000:0.0:1787829286.432867:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829286.432874:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829286.432908:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6d80 x1874663043318784/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829286.432928:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829286.432931:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829286.432932:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829286.432935:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11353 00000100:00080000:0.0:1787829286.432938:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829286.432940:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829286.432948:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829286.432956:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5880 x1874663043318912/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829286.432964:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829286.432967:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829286.432968:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829286.432973:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11353 00000100:00080000:0.0:1787829286.432976:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1787829286.432980:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043318784:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787829286.432981:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829286.432988:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829286.432996:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7100 x1874663043319040/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829286.433067:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829286.433101:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829286/real 1787829286] req@ffff9181c55a6d80 x1874663043318784/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829311 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829286.433116:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6d80 x1874663043318784/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829311 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829286.433124:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829286.433127:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829286.433139:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043318784:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829286.433145:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043318912:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829286.433166:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829286.433177:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829286/real 1787829286] req@ffff9181c55a5880 x1874663043318912/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829311 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829286.433187:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5880 x1874663043318912/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829311 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829286.433194:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829286.433197:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829286.433199:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043318912:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829286.433218:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043319040:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829286.433229:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829286.433236:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829286/real 1787829286] req@ffff9181c55a7100 x1874663043319040/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829311 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829286.433245:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7100 x1874663043319040/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829311 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829286.433250:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829286.433252:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829286.433254:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043319040:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787829286.496897:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829287.520905:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829288.544896:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829289.568884:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829290.592913:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829291.552920:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829291.552928:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829291.552932:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829291.552940:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11358 00000100:00080000:3.0:1787829291.552949:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829291.552955:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829291.552988:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043319168/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829291.553024:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829291.553029:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829291.553031:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829291.553035:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11358 00000100:00080000:3.0:1787829291.553038:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829291.553041:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829291.553049:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829291.553062:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043319296/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829291.553070:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829291.553073:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829291.553074:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829291.553079:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11358 00000100:00080000:3.0:1787829291.553082:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1787829291.553083:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043319168:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787829291.553086:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829291.553093:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829291.553107:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043319424/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829291.553148:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829291.553179:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829291/real 1787829291] req@ffff9181c5636a00 x1874663043319168/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829316 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829291.553193:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043319168/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829316 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829291.553222:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829291.553226:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829291.553250:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043319168:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829291.553278:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043319296:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829291.553357:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829291.553370:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829291/real 1787829291] req@ffff9181c5634380 x1874663043319296/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829316 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829291.553379:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043319296/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829316 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829291.553390:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829291.553393:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829291.553395:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043319296:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829291.553399:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043319424:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829291.553434:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829291.553447:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829291/real 1787829291] req@ffff9181c5635500 x1874663043319424/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829316 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829291.553464:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043319424/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829316 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829291.553477:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829291.553482:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829291.553488:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043319424:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829291.616855:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829292.640886:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829293.664906:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829294.688935:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829295.712884:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787829296.672878:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-client' failed (-108). 10000000:00020000:1.0:1787829296.672887:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:1.0:1787829296.672892:0:159735:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:1.0:1787829296.672896:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff9181c5620000 from 7 00000100:00080000:3.0:1787829296.672900:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 10000000:01000000:1.0:1787829296.672901:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00000100:00080000:3.0:1787829296.672910:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829296.672915:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829296.672950:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11363 00000100:00080000:3.0:1787829296.672960:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829296.672966:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829296.672990:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043319680/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787829296.672994:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f894 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829296.673029:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f894 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829296.673042:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f894 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787829296.673043:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829296.673055:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829296.673059:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1787829296.673067:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215db40 going to sleep for 0 seconds 00000100:00080000:3.0:1787829296.673068:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11363 00000100:00080000:3.0:1787829296.673078:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1787829296.673080:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043319552:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00080000:3.0:1787829296.673103:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787829296.673113:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043319680:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787829296.673114:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f894 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000008:00100000:3.0:1787829296.673115:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:1.0:1787829296.673130:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f894 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787829296.673137:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043319808/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787829296.673143:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f894 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787829296.673150:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829296.673158:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829296.673161:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00010000:00010000:1.0:1787829296.673162:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f894 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787829296.673166:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11363 00000100:00080000:3.0:1787829296.673173:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00010000:00010000:1.0:1787829296.673179:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f894 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000800:00000100:2.0:1787829296.673179:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1787829296.673183:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:1.0:1787829296.673195:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4a00 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787829296.673200:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f894 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829296.673209:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f894 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787829296.673215:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f894 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787829296.673221:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00000100:00100000:2.0:1787829296.673223:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829296/real 1787829296] req@ffff9181c5637100 x1874663043319680/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829321 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:1.0:1787829296.673225:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787829296.673232:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f894 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829296.673244:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b4a00/0xbef42c0221d6f894 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000008:00100000:3.0:1787829296.673251:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829296.673253:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043319680/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829321 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 10000000:01000000:1.0:1787829296.673282:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:2.0:1787829296.673283:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00100000:3.0:1787829296.673283:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635f80 x1874663043319936/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829296.673287:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:CONNECTING 00000100:00080000:2.0:1787829296.673293:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829296.673300:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043319680:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829296.673322:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043319808:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829296.673343:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829296.673354:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829296/real 1787829296] req@ffff9181c5637480 x1874663043319808/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829321 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829296.673364:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043319808/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829321 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829296.673370:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829296.673372:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829296.673375:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043319808:192.168.203.147@tcp:38:kworker.0 00000100:00080000:3.0:1787829296.673463:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829296.673466:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829296.673468:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829296.673472:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11368 00000100:00080000:3.0:1787829296.673477:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829296.673480:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787829296.673487:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043319936:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829296.673510:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829296.673521:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829296/real 1787829296] req@ffff9181c5635f80 x1874663043319936/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829321 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829296.673533:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635f80 x1874663043319936/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829321 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829296.673540:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829296.673543:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829296.673547:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634e00 x1874663043320064/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829296.673551:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043319936:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829296.673570:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043320064:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787829296.673585:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829296.673593:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829296/real 1787829296] req@ffff9181c5634e00 x1874663043320064/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829321 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829296.673601:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634e00 x1874663043320064/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829321 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829296.673606:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829296.673608:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829296.673618:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043320064:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1787829296.736934:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829297.760888:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829298.784853:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829299.808828:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829300.832833:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829301.792844:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829301.792853:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829301.792856:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829301.792866:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11368 00000100:00080000:3.0:1787829301.792878:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829301.792886:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829301.792928:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634e00 x1874663043320192/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829301.792954:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829301.792957:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829301.792958:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829301.792961:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11368 00000100:00080000:3.0:1787829301.792964:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829301.792968:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829301.792975:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829301.792984:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635f80 x1874663043320320/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829301.792992:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829301.792995:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829301.792996:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829301.792998:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11368 00000100:00080000:3.0:1787829301.793000:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829301.793003:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829301.793006:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829301.793012:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043320448/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829301.793020:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043320192:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787829301.793088:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829301.793126:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829301/real 1787829301] req@ffff9181c5634e00 x1874663043320192/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829326 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829301.793144:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634e00 x1874663043320192/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829326 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829301.793155:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829301.793160:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829301.793176:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043320192:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829301.793183:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043320320:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829301.793208:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829301.793223:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829301/real 1787829301] req@ffff9181c5635f80 x1874663043320320/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829326 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829301.793254:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635f80 x1874663043320320/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829326 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829301.793271:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829301.793275:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829301.793279:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043320320:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829301.793304:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043320448:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829301.793327:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829301.793340:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829301/real 1787829301] req@ffff9181c5637480 x1874663043320448/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829326 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829301.793352:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043320448/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829326 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829301.793360:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829301.793364:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829301.793367:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043320448:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829301.856851:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829302.880835:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829303.904856:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829304.928882:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829305.952876:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829306.912878:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829306.912886:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829306.912890:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829306.912898:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11373 00000100:00080000:1.0:1787829306.912907:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829306.912913:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787829306.912965:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0e00 x1874663043320576/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829306.912995:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829306.912998:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829306.913000:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829306.913004:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11373 00000100:00080000:1.0:1787829306.913007:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829306.913011:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829306.913035:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829306.913045:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2680 x1874663043320704/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829306.913051:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043320576:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787829306.913060:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829306.913064:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829306.913065:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829306.913069:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11373 00000100:00080000:1.0:1787829306.913072:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829306.913075:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829306.913078:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829306.913087:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3480 x1874663043320832/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829306.913114:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829306.913145:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829306/real 1787829306] req@ffff9181c35b0e00 x1874663043320576/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829331 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829306.913158:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0e00 x1874663043320576/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829331 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829306.913166:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829306.913169:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829306.913180:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043320576:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829306.913214:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043320704:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829306.913252:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829306.913263:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829306/real 1787829306] req@ffff9181c35b2680 x1874663043320704/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829331 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829306.913273:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2680 x1874663043320704/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829331 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829306.913279:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829306.913281:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829306.913284:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043320704:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829306.913288:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043320832:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829306.913318:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829306.913329:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829306/real 1787829306] req@ffff9181c35b3480 x1874663043320832/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829331 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829306.913341:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3480 x1874663043320832/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829331 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829306.913348:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829306.913351:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829306.913354:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043320832:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787829306.976899:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829308.000912:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829309.024867:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829310.048879:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829311.072873:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829312.032837:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829312.032846:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829312.032850:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829312.032858:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11378 00000100:00080000:3.0:1787829312.032867:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829312.032873:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829312.032908:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043320960/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829312.032939:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829312.032943:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829312.032945:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829312.032949:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11378 00000100:00080000:3.0:1787829312.032952:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829312.032955:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829312.032962:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829312.032972:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043321088/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829312.032979:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829312.032981:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829312.032983:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829312.032985:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11378 00000100:00080000:3.0:1787829312.032988:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829312.032991:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829312.032994:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829312.033000:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043321216/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829312.033020:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043320960:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787829312.033077:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829312.033108:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829312/real 1787829312] req@ffff9181c5637100 x1874663043320960/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829337 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829312.033121:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043320960/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829337 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829312.033129:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829312.033132:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829312.033148:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043320960:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829312.033154:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043321088:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829312.033184:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829312.033194:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829312/real 1787829312] req@ffff9181c5635500 x1874663043321088/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829337 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829312.033205:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043321088/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829337 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829312.033210:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829312.033214:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829312.033217:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043321088:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829312.033222:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043321216:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829312.033236:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829312.033242:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829312/real 1787829312] req@ffff9181c5634380 x1874663043321216/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829337 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829312.033249:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043321216/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829337 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829312.033259:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829312.033261:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829312.033263:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043321216:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829312.096884:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829313.120872:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829314.144876:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829315.168941:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829316.192893:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829317.152838:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829317.152850:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829317.152854:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 10000000:01000000:1.0:1787829317.152864:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'params' failed (-108). 00000100:00080000:3.0:1787829317.152865:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11383 10000000:00020000:1.0:1787829317.152870:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:1.0:1787829317.152875:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:1.0:1787829317.152877:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff9181c87b19b8 from 1 00000100:00080000:3.0:1787829317.152880:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 10000000:01000000:1.0:1787829317.152881:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00000100:00080000:3.0:1787829317.152891:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829317.152921:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043321472/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787829317.152933:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f89b lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829317.152947:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f89b lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787829317.152949:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829317.152961:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829317.152964:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00010000:00010000:1.0:1787829317.152966:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f89b lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787829317.152972:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11383 00000100:00080000:3.0:1787829317.152977:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829317.152983:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787829317.152992:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215db40 going to sleep for 0 seconds 00000100:00100000:2.0:1787829317.152992:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043321472:192.168.203.147@tcp:250:kworker.0 00000008:00100000:3.0:1787829317.152993:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829317.153003:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043321344:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787829317.153018:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f89b lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787829317.153018:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043321600/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787829317.153035:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f89b lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787829317.153044:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:1.0:1787829317.153045:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f89b lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787829317.153053:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00010000:00010000:1.0:1787829317.153056:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f89b lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787829317.153058:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00010000:00010000:1.0:1787829317.153068:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f89b lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787829317.153068:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11383 00000100:00100000:2.0:1787829317.153078:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043321600:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829317.153079:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4e00 already unused, calling callback (ffffffffc1273c20) 00000100:00080000:3.0:1787829317.153079:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 10000000:00010000:1.0:1787829317.153086:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f89b lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787829317.153091:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:1.0:1787829317.153096:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f89b lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000008:00100000:3.0:1787829317.153101:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 10000000:00010000:1.0:1787829317.153103:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f89b lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787829317.153111:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1787829317.153115:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787829317.153133:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f89b lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787829317.153136:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043321728/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787829317.153164:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b4e00/0xbef42c0221d6f89b lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787829317.153185:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787829317.153187:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:CONNECTING 00000100:00100000:2.0:1787829317.153204:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043321728:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829317.216879:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829318.240873:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829319.264903:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1787829320.224974:0:2377:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:0.0:1787829320.225017:0:2377:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1787829320.225033:0:2377:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1787829320.225072:0:2377:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1787829320.225082:0:2377:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:2.0:1787829320.225125:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829317/real 1787829320] req@ffff9181c5636d80 x1874663043321728/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829342 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829320.225153:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043321728/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829342 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829320.225166:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829320.225172:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829320.225176:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043321728:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829320.225183:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829317/real 1787829320] req@ffff9181c5637800 x1874663043321600/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829342 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829320.225201:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043321600/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829342 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829320.225216:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829320.225220:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829320.225224:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043321600:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829320.225231:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829317/real 1787829320] req@ffff9181c5636a00 x1874663043321472/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829342 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829320.225243:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043321472/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829342 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829320.225250:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829320.225280:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829320.225294:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043321472:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787829320.225388:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829320.225393:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829320.225396:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829320.225402:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11388 00000100:00080000:1.0:1787829320.225409:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829320.225415:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787829320.225447:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2a00 x1874663043321856/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829320.225509:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043321856:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787829320.225565:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:1.0:1787829320.225803:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829320.225808:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829320.225810:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829320.225815:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11388 00000100:00080000:1.0:1787829320.225820:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829320.225825:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829320.225835:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829320.225850:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2d80 x1874663043321984/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829320.225929:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829320/real 1787829320] req@ffff9181c35b2a00 x1874663043321856/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829345 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829320.225943:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2a00 x1874663043321856/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829345 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829320.225952:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829320.225955:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829320.225961:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043321856:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829320.225979:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043321984:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829320.226012:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829320.226027:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829320/real 1787829320] req@ffff9181c35b2d80 x1874663043321984/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829345 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829320.226039:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2d80 x1874663043321984/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829345 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829320.226048:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829320.226052:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829320.226056:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043321984:192.168.203.147@tcp:38:kworker.0 00000100:00080000:1.0:1787829320.226121:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829320.226126:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829320.226128:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829320.226132:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11388 00000100:00080000:1.0:1787829320.226137:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829320.226141:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829320.226147:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829320.226162:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1f80 x1874663043322112/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829320.226430:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043322112:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829320.226459:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829320.226473:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829320/real 1787829320] req@ffff9181c35b1f80 x1874663043322112/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829345 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829320.226486:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1f80 x1874663043322112/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829345 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829320.226494:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829320.226505:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829320.226509:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043322112:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829320.288878:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829321.312888:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829322.336873:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829323.360902:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829324.384873:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829325.344891:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829325.344897:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829325.344900:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829325.344909:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11391 00000100:00080000:3.0:1787829325.344919:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829325.344926:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829325.344963:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043322240/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829325.344993:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829325.344996:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829325.344997:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829325.345001:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11391 00000100:00080000:3.0:1787829325.345004:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829325.345008:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829325.345015:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829325.345029:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637b80 x1874663043322368/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829325.345037:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829325.345040:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829325.345042:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829325.345046:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11391 00000100:00080000:3.0:1787829325.345048:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829325.345053:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787829325.345055:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043322240:192.168.203.147@tcp:250:kworker.0 00000008:00100000:3.0:1787829325.345056:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829325.345081:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043322496/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829325.345138:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829325.345172:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829325/real 1787829325] req@ffff9181c5635c00 x1874663043322240/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829350 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829325.345191:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043322240/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829350 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829325.345201:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829325.345205:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829325.345221:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043322240:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829325.345229:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043322368:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829325.345257:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829325.345270:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829325/real 1787829325] req@ffff9181c5637b80 x1874663043322368/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829350 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829325.345279:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637b80 x1874663043322368/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829350 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829325.345285:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829325.345287:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829325.345290:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043322368:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829325.345309:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043322496:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829325.345323:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829325.345330:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829325/real 1787829325] req@ffff9181c5634000 x1874663043322496/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829350 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829325.345339:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043322496/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829350 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829325.345349:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829325.345351:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829325.345353:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043322496:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829325.408912:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829326.432938:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829327.456928:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829328.480885:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829329.504902:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829330.464864:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829330.464873:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829330.464876:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829330.464885:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11397 00000100:00080000:0.0:1787829330.464895:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829330.464902:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829330.464935:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5f80 x1874663043322624/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829330.464956:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829330.464959:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829330.464960:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829330.464963:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11397 00000100:00080000:0.0:1787829330.464965:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829330.464968:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829330.464976:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829330.464984:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4380 x1874663043322752/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829330.464992:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829330.464996:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829330.464998:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1787829330.464999:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043322624:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787829330.465002:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11397 00000100:00080000:0.0:1787829330.465007:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829330.465011:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829330.465015:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829330.465021:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4700 x1874663043322880/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829330.465082:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829330.465129:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829330/real 1787829330] req@ffff9181c55a5f80 x1874663043322624/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829355 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829330.465148:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5f80 x1874663043322624/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829355 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829330.465159:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829330.465164:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829330.465186:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043322624:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829330.465213:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043322752:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829330.465243:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829330.465258:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829330/real 1787829330] req@ffff9181c55a4380 x1874663043322752/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829355 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829330.465271:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4380 x1874663043322752/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829355 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829330.465285:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829330.465289:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829330.465293:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043322752:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829330.465300:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043322880:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829330.465322:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829330.465334:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829330/real 1787829330] req@ffff9181c55a4700 x1874663043322880/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829355 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829330.465346:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4700 x1874663043322880/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829355 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829330.465355:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829330.465359:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829330.465363:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043322880:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829330.528908:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829331.552879:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829332.576865:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829333.600895:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829334.624896:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829335.584832:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829335.584840:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829335.584843:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829335.584851:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11402 00000100:00080000:3.0:1787829335.584860:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829335.584866:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829335.584899:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634a80 x1874663043323008/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829335.584922:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829335.584926:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829335.584927:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829335.584930:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11402 00000100:00080000:3.0:1787829335.584932:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829335.584935:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829335.584942:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829335.584954:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043323136/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829335.584963:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829335.584966:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829335.584967:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1787829335.584973:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043323008:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787829335.584973:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11402 00000100:00080000:3.0:1787829335.584979:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829335.584983:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829335.584987:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829335.585013:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637b80 x1874663043323264/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829335.585048:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829335.585087:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829335/real 1787829335] req@ffff9181c5634a80 x1874663043323008/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829360 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829335.585105:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634a80 x1874663043323008/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829360 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829335.585114:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829335.585117:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829335.585129:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043323008:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829335.585158:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043323136:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829335.585180:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829335.585192:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829335/real 1787829335] req@ffff9181c5634000 x1874663043323136/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829360 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829335.585201:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043323136/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829360 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829335.585207:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829335.585209:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829335.585211:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043323136:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829335.585216:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043323264:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829335.585227:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829335.585233:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829335/real 1787829335] req@ffff9181c5637b80 x1874663043323264/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829360 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829335.585240:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637b80 x1874663043323264/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829360 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829335.585246:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829335.585249:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829335.585252:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043323264:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829335.648910:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829336.672889:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787829337.184875:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-sptlrpc' failed (-108). 10000000:00020000:1.0:1787829337.184884:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00010000:00010000:2.0:1787829337.696868:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829338.720884:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787829338.976856:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:1.0:1787829338.976862:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff9181c5620000 from 341 10000000:01000000:1.0:1787829338.976866:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:1.0:1787829338.976922:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f8a2 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829338.976931:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f8a2 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829338.976940:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f8a2 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787829338.976956:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215db40 going to sleep for 0 seconds 00000100:00100000:1.0:1787829338.976966:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043323392:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787829338.977006:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f8a2 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829338.977012:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f8a2 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829338.977017:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f8a2 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829338.977022:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f8a2 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829338.977027:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f8a2 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829338.977031:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b5800 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787829338.977034:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f8a2 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829338.977039:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f8a2 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787829338.977044:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f8a2 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787829338.977048:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:1.0:1787829338.977050:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787829338.977055:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f8a2 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829338.977062:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f8a2 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787829338.977080:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787829338.977082:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:2.0:1787829338.977135:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829338.977141:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829338.977144:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829338.977149:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11407 00000100:00080000:2.0:1787829338.977158:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829338.977162:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787829338.977181:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ce00 x1874663043323520/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829338.977203:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829338.977206:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829338.977208:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829338.977211:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11407 00000100:00080000:2.0:1787829338.977213:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829338.977216:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829338.977223:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829338.977232:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d880 x1874663043323648/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829338.977240:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829338.977243:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829338.977244:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829338.977249:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11407 00000100:00080000:2.0:1787829338.977251:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829338.977257:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829338.977258:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043323520:192.168.203.147@tcp:250:kworker.0 00000008:00100000:2.0:1787829338.977261:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829338.977270:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e680 x1874663043323776/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829338.977317:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829338.977344:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829338/real 1787829338] req@ffff9181e007ce00 x1874663043323520/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829363 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829338.977357:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ce00 x1874663043323520/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829363 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829338.977364:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829338.977367:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829338.977378:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043323520:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787829338.977385:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043323648:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787829338.977404:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829338.977414:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829338/real 1787829338] req@ffff9181e007d880 x1874663043323648/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829363 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829338.977428:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d880 x1874663043323648/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829363 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829338.977436:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829338.977440:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829338.977444:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043323648:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787829338.977465:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043323776:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787829338.977483:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829338.977491:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829338/real 1787829338] req@ffff9181e007e680 x1874663043323776/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829363 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829338.977500:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e680 x1874663043323776/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829363 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829338.977505:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829338.977507:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829338.977509:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043323776:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829339.744906:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829340.768930:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829341.792885:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829342.816923:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829343.840889:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829344.288820:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829344.288832:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829344.288836:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829344.288846:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11410 00000100:00080000:2.0:1787829344.288856:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829344.288862:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787829344.288893:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007fb80 x1874663043323904/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829344.288912:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829344.288915:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829344.288916:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829344.288919:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11410 00000100:00080000:2.0:1787829344.288925:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829344.288929:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829344.288952:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829344.288962:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043323904:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829344.288965:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ca80 x1874663043324032/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829344.288976:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829344.288978:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829344.288980:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829344.288982:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11410 00000100:00080000:2.0:1787829344.288985:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829344.288988:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829344.288992:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829344.289001:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007dc00 x1874663043324160/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829344.289027:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829344.289057:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829344/real 1787829344] req@ffff9181e007fb80 x1874663043323904/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829369 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829344.289071:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007fb80 x1874663043323904/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829369 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829344.289083:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829344.289087:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829344.289104:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043323904:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787829344.289132:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043324032:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787829344.289157:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829344.289168:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829344/real 1787829344] req@ffff9181e007ca80 x1874663043324032/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829369 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829344.289176:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ca80 x1874663043324032/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829369 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829344.289183:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829344.289185:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829344.289187:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043324032:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787829344.289191:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043324160:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787829344.289202:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829344.289208:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829344/real 1787829344] req@ffff9181e007dc00 x1874663043324160/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829369 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829344.289216:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007dc00 x1874663043324160/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829369 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829344.289225:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829344.289227:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829344.289229:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043324160:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829344.864910:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829345.888865:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829346.912892:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829347.936887:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829348.960928:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829349.408848:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829349.408857:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829349.408860:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829349.408869:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11416 00000100:00080000:1.0:1787829349.408880:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829349.408885:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787829349.408931:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0a80 x1874663043324288/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829349.408953:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829349.408956:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829349.408958:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829349.408961:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11416 00000100:00080000:1.0:1787829349.408963:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829349.408966:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829349.408973:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829349.408982:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3800 x1874663043324416/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829349.408990:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829349.408993:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829349.408995:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829349.409000:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11416 00000100:00100000:0.0:1787829349.409003:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043324288:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787829349.409004:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829349.409011:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829349.409015:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829349.409023:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3b80 x1874663043324544/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829349.409063:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829349.409092:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829349/real 1787829349] req@ffff9181c35b0a80 x1874663043324288/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829374 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829349.409106:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0a80 x1874663043324288/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829374 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829349.409114:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829349.409117:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829349.409128:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043324288:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787829349.409134:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043324416:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787829349.409158:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829349.409167:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829349/real 1787829349] req@ffff9181c35b3800 x1874663043324416/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829374 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829349.409175:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3800 x1874663043324416/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829374 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829349.409181:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829349.409183:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829349.409185:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043324416:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787829349.409203:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043324544:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787829349.409215:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829349.409221:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829349/real 1787829349] req@ffff9181c35b3b80 x1874663043324544/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829374 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829349.409229:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3b80 x1874663043324544/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829374 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829349.409233:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829349.409235:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829349.409237:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043324544:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829349.984910:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829351.008869:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829352.032920:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829353.056865:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829354.080934:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829354.528927:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829354.528938:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829354.528942:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829354.528953:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11421 00000100:00080000:0.0:1787829354.528964:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829354.528971:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829354.529015:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5c00 x1874663043324672/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829354.529045:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829354.529048:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829354.529049:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829354.529052:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11421 00000100:00080000:0.0:1787829354.529054:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829354.529058:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829354.529068:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829354.529077:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5180 x1874663043324800/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829354.529085:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829354.529088:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829354.529089:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829354.529092:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11421 00000100:00080000:0.0:1787829354.529094:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1787829354.529099:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043324672:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787829354.529100:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829354.529107:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829354.529115:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5500 x1874663043324928/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829354.529177:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829354.529214:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829354/real 1787829354] req@ffff9181c55a5c00 x1874663043324672/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829379 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829354.529234:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5c00 x1874663043324672/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829379 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829354.529243:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829354.529246:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829354.529260:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043324672:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829354.529266:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043324800:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829354.529285:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829354.529295:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829354/real 1787829354] req@ffff9181c55a5180 x1874663043324800/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829379 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829354.529305:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5180 x1874663043324800/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829379 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829354.529312:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829354.529314:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829354.529316:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043324800:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829354.529337:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043324928:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829354.529347:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829354.529354:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829354/real 1787829354] req@ffff9181c55a5500 x1874663043324928/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829379 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829354.529362:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5500 x1874663043324928/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829379 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829354.529371:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829354.529374:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829354.529376:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043324928:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829355.104920:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829356.128921:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829357.152920:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829358.176904:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787829359.136868:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-cliir' failed (-108). 10000000:00000400:1.0:1787829359.136879:0:159735:0:(mgc_request.c:1900:mgc_process_log()) MGC192.168.203.147@tcp: IR log lustre-cliir failed, not fatal: rc = -108 10000000:01000000:1.0:1787829359.136887:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:1.0:1787829359.136891:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff9181c5620000 from 52 10000000:01000000:1.0:1787829359.136895:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:1.0:1787829359.136939:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f8a9 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829359.136949:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f8a9 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829359.136962:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f8a9 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787829359.136995:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215db40 going to sleep for 0 seconds 00000100:00100000:1.0:1787829359.137008:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043325056:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787829359.137021:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f8a9 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829359.137030:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f8a9 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829359.137043:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f8a9 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829359.137052:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f8a9 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829359.137060:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f8a9 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829359.137068:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b5800 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787829359.137071:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f8a9 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829359.137077:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f8a9 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787829359.137082:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f8a9 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787829359.137086:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1787829359.137089:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787829359.137094:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f8a9 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829359.137099:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f8a9 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787829359.137115:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787829359.137118:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:0.0:1787829359.137183:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829359.137190:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829359.137193:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829359.137200:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11426 00000100:00080000:0.0:1787829359.137209:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829359.137215:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829359.137232:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4e00 x1874663043325184/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829359.137255:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829359.137259:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829359.137262:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829359.137266:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11426 00000100:00080000:0.0:1787829359.137275:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829359.137281:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829359.137291:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829359.137302:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043325184:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787829359.137309:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7800 x1874663043325312/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829359.137324:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829359.137328:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829359.137330:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829359.137335:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11426 00000100:00080000:0.0:1787829359.137340:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829359.137344:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829359.137348:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829359.137360:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5500 x1874663043325440/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829359.137369:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829359.137406:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829359/real 1787829359] req@ffff9181c55a4e00 x1874663043325184/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829384 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829359.137420:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4e00 x1874663043325184/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829384 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829359.137427:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829359.137430:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829359.137444:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043325184:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829359.137464:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043325312:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829359.137490:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829359.137503:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829359/real 1787829359] req@ffff9181c55a7800 x1874663043325312/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829384 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829359.137512:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7800 x1874663043325312/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829384 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829359.137522:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829359.137524:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829359.137527:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043325312:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829359.137531:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043325440:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829359.137544:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829359.137550:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829359/real 1787829359] req@ffff9181c55a5500 x1874663043325440/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829384 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829359.137558:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5500 x1874663043325440/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829384 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829359.137563:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829359.137564:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829359.137566:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043325440:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829359.200897:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829360.224893:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829361.248903:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829362.272900:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829363.296889:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829364.256851:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829364.256859:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829364.256862:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829364.256870:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11430 00000100:00080000:1.0:1787829364.256879:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829364.256887:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787829364.256928:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0700 x1874663043325568/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829364.256966:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829364.256970:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829364.256971:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829364.256975:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11430 00000100:00080000:1.0:1787829364.256978:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829364.256982:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829364.256989:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829364.257001:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3b80 x1874663043325696/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829364.257009:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829364.257012:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829364.257013:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829364.257016:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11430 00000100:00080000:1.0:1787829364.257019:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1787829364.257021:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043325568:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787829364.257024:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829364.257032:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829364.257041:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3800 x1874663043325824/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829364.257089:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829364.257124:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829364/real 1787829364] req@ffff9181c35b0700 x1874663043325568/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829389 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829364.257137:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0700 x1874663043325568/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829389 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829364.257145:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829364.257148:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829364.257161:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043325568:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829364.257167:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043325696:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829364.257187:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829364.257199:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829364/real 1787829364] req@ffff9181c35b3b80 x1874663043325696/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829389 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829364.257207:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3b80 x1874663043325696/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829389 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829364.257212:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829364.257215:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829364.257217:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043325696:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829364.257234:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043325824:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829364.257244:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829364.257250:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829364/real 1787829364] req@ffff9181c35b3800 x1874663043325824/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829389 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829364.257257:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3800 x1874663043325824/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829389 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829364.257262:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829364.257264:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829364.257266:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043325824:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829364.320866:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829365.344874:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829366.368920:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829367.392878:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829368.416947:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829369.376888:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829369.376899:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829369.376903:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829369.376914:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11435 00000100:00080000:2.0:1787829369.376925:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829369.376932:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787829369.376977:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c000 x1874663043325952/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829369.377007:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829369.377010:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829369.377011:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829369.377014:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11435 00000100:00080000:2.0:1787829369.377017:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829369.377019:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829369.377028:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829369.377037:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c380 x1874663043326080/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829369.377045:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829369.377048:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829369.377050:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829369.377053:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11435 00000100:00080000:2.0:1787829369.377055:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829369.377058:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829369.377061:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829369.377073:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007dc00 x1874663043326208/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787829369.377074:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043325952:192.168.203.147@tcp:250:kworker.0 00000800:00000100:0.0:1787829369.377157:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829369.377198:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829369/real 1787829369] req@ffff9181e007c000 x1874663043325952/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829394 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829369.377226:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c000 x1874663043325952/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829394 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829369.377239:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829369.377245:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829369.377260:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043325952:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787829369.377270:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043326080:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787829369.377296:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829369.377314:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829369/real 1787829369] req@ffff9181e007c380 x1874663043326080/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829394 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829369.377330:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c380 x1874663043326080/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829394 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829369.377340:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829369.377344:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829369.377348:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043326080:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787829369.377375:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043326208:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787829369.377398:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829369.377411:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829369/real 1787829369] req@ffff9181e007dc00 x1874663043326208/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829394 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829369.377424:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007dc00 x1874663043326208/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829394 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829369.377433:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829369.377436:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829369.377440:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043326208:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829369.440885:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829370.464874:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829371.488869:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829372.512919:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829373.536939:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829374.496844:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829374.496852:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829374.496855:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829374.496863:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11441 00000100:00080000:3.0:1787829374.496873:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829374.496880:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829374.496911:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043326336/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829374.496934:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829374.496937:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829374.496938:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829374.496941:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11441 00000100:00080000:3.0:1787829374.496943:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829374.496946:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829374.496952:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829374.496961:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043326464/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829374.496968:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829374.496971:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829374.496972:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829374.496978:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11441 00000100:00100000:0.0:1787829374.496981:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043326336:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787829374.496982:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829374.496988:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829374.496993:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829374.497000:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043326592/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829374.497040:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829374.497070:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829374/real 1787829374] req@ffff9181c5635c00 x1874663043326336/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829399 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829374.497084:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043326336/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829399 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829374.497093:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829374.497095:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829374.497107:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043326336:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787829374.497112:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043326464:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787829374.497128:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829374.497136:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829374/real 1787829374] req@ffff9181c5636a00 x1874663043326464/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829399 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829374.497145:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043326464/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829399 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829374.497150:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829374.497152:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829374.497155:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043326464:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787829374.497174:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043326592:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787829374.497188:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829374.497195:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829374/real 1787829374] req@ffff9181c5637800 x1874663043326592/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829399 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829374.497208:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043326592/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829399 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829374.497213:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829374.497215:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829374.497217:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043326592:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829374.560899:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829375.584893:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829376.608878:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829377.632882:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829378.656869:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829379.616850:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829379.616861:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829379.616864:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 10000000:01000000:1.0:1787829379.616876:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-client' failed (-108). 00000100:00080000:2.0:1787829379.616876:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11446 10000000:00020000:1.0:1787829379.616882:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:1.0:1787829379.616887:0:159735:0:(mgc_request.c:611:do_requeue()) updating log params 00000100:00080000:2.0:1787829379.616889:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 10000000:01000000:1.0:1787829379.616890:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff9181c5620000 from 7 10000000:01000000:1.0:1787829379.616895:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00000100:00080000:2.0:1787829379.616902:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787829379.616929:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ca80 x1874663043326848/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787829379.616941:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f8b0 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829379.616950:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f8b0 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787829379.616959:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:1.0:1787829379.616960:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f8b0 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787829379.616977:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829379.616980:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1787829379.616987:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215db40 going to sleep for 0 seconds 00000100:00080000:2.0:1787829379.616987:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11446 00000100:00100000:0.0:1787829379.616994:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043326848:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787829379.616994:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1787829379.616999:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043326720:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00080000:2.0:1787829379.617002:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:1.0:1787829379.617013:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f8b0 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000008:00100000:2.0:1787829379.617014:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:1.0:1787829379.617025:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f8b0 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829379.617031:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f8b0 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787829379.617034:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007fb80 x1874663043326976/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787829379.617047:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f8b0 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787829379.617053:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829379.617060:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829379.617065:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00010000:00010000:1.0:1787829379.617072:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f8b0 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787829379.617072:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11446 00000100:00080000:2.0:1787829379.617080:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00010000:00010000:1.0:1787829379.617088:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4800 already unused, calling callback (ffffffffc1273c20) 00000100:00080000:2.0:1787829379.617089:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 10000000:00010000:1.0:1787829379.617094:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f8b0 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787829379.617097:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043326976:192.168.203.147@tcp:38:kworker.0 00000008:00100000:2.0:1787829379.617097:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:1.0:1787829379.617108:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f8b0 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787829379.617114:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f8b0 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787829379.617116:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e680 x1874663043327104/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 10000000:01000000:1.0:1787829379.617127:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:1.0:1787829379.617135:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787829379.617143:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f8b0 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829379.617149:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f8b0 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787829379.617150:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043327104:192.168.203.147@tcp:38:kworker.0 10000000:01000000:1.0:1787829379.617174:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787829379.617177:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:CONNECTING 00010000:00010000:2.0:1787829379.680906:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829380.704868:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829381.728898:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1787829382.688995:0:2378:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:2.0:1787829382.689045:0:2378:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1787829382.689064:0:2378:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:2.0:1787829382.689110:0:2378:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:2.0:1787829382.689122:0:2378:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:0.0:1787829382.689171:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829379/real 1787829382] req@ffff9181e007e680 x1874663043327104/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829404 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829382.689200:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e680 x1874663043327104/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829404 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829382.689222:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829382.689227:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829382.689232:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043327104:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787829382.689241:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829379/real 1787829382] req@ffff9181e007fb80 x1874663043326976/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829404 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829382.689254:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007fb80 x1874663043326976/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829404 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829382.689269:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829382.689274:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829382.689277:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043326976:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787829382.689284:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829379/real 1787829382] req@ffff9181e007ca80 x1874663043326848/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829404 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829382.689296:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ca80 x1874663043326848/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829404 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829382.689304:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829382.689322:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829382.689339:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043326848:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787829382.689630:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829382.689638:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829382.689642:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829382.689649:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11451 00000100:00080000:3.0:1787829382.689656:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829382.689680:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829382.689714:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043327232/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829382.689784:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829382.689789:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829382.689792:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829382.689798:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11451 00000100:00080000:3.0:1787829382.689803:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829382.689808:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829382.689819:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829382.689834:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043327360/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829382.689855:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829382.689859:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829382.689861:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829382.689866:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11451 00000100:00080000:3.0:1787829382.689870:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829382.689876:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829382.689882:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829382.689896:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043327488/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787829382.689943:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043327232:192.168.203.147@tcp:250:kworker.0 00000800:00000100:0.0:1787829382.689995:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829382.690019:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829382/real 1787829382] req@ffff9181c5636d80 x1874663043327232/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829407 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829382.690034:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043327232/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829407 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829382.690043:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829382.690047:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829382.690060:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043327232:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787829382.690073:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043327360:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787829382.690099:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829382.690111:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829382/real 1787829382] req@ffff9181c5634380 x1874663043327360/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829407 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829382.690124:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043327360/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829407 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829382.690133:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829382.690136:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829382.690140:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043327360:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787829382.690145:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043327488:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787829382.690160:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829382.690169:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829382/real 1787829382] req@ffff9181c5635500 x1874663043327488/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829407 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829382.690187:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043327488/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829407 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829382.690196:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829382.690199:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829382.690203:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043327488:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829382.752880:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829383.776917:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829384.800886:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829385.824893:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829386.848880:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829387.808847:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829387.808855:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829387.808858:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829387.808866:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11454 00000100:00080000:0.0:1787829387.808875:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829387.808882:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829387.808912:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5180 x1874663043327616/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829387.808935:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829387.808938:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829387.808939:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829387.808942:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11454 00000100:00080000:0.0:1787829387.808945:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829387.808948:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829387.808954:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829387.808962:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5c00 x1874663043327744/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829387.808970:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829387.808973:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829387.808974:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829387.808977:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11454 00000100:00080000:0.0:1787829387.808980:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829387.808983:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829387.808986:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829387.808994:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043327616:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787829387.808998:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4700 x1874663043327872/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829387.809062:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829387.809099:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829387/real 1787829387] req@ffff9181c55a5180 x1874663043327616/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829412 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829387.809119:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5180 x1874663043327616/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829412 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829387.809130:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829387.809135:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829387.809151:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043327616:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829387.809160:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043327744:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829387.809189:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829387.809204:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829387/real 1787829387] req@ffff9181c55a5c00 x1874663043327744/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829412 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829387.809217:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5c00 x1874663043327744/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829412 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829387.809225:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829387.809229:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829387.809232:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043327744:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829387.809257:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043327872:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829387.809277:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829387.809290:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829387/real 1787829387] req@ffff9181c55a4700 x1874663043327872/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829412 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829387.809303:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4700 x1874663043327872/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829412 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829387.809311:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829387.809314:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829387.809318:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043327872:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829387.872896:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829388.896868:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829389.920862:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829390.944925:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829391.968942:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829392.928875:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829392.928885:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829392.928889:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829392.928900:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11459 00000100:00080000:1.0:1787829392.928911:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829392.928919:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787829392.928953:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0a80 x1874663043328000/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829392.928982:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829392.928986:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829392.928988:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829392.928992:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11459 00000100:00080000:1.0:1787829392.928996:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829392.929001:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829392.929011:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829392.929023:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043328000:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787829392.929028:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1f80 x1874663043328128/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829392.929041:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829392.929045:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829392.929047:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829392.929052:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11459 00000100:00080000:1.0:1787829392.929055:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829392.929059:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829392.929065:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829392.929078:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2d80 x1874663043328256/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829392.929081:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829392.929116:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829392/real 1787829392] req@ffff9181c35b0a80 x1874663043328000/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829417 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829392.929129:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0a80 x1874663043328000/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829417 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829392.929137:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829392.929140:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829392.929155:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043328000:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829392.929181:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043328128:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829392.929208:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829392.929219:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829392/real 1787829392] req@ffff9181c35b1f80 x1874663043328128/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829417 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829392.929227:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1f80 x1874663043328128/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829417 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829392.929233:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829392.929236:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829392.929238:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043328128:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829392.929242:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043328256:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829392.929255:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829392.929261:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829392/real 1787829392] req@ffff9181c35b2d80 x1874663043328256/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829417 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829392.929268:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2d80 x1874663043328256/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829417 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829392.929274:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829392.929276:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829392.929278:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043328256:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829392.992906:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829394.016881:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829395.040872:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829396.064882:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829397.088866:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829398.048843:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829398.048854:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829398.048858:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829398.048868:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11464 00000100:00080000:0.0:1787829398.048878:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829398.048886:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829398.048924:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4380 x1874663043328384/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829398.048952:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829398.048956:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829398.048958:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829398.048962:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11464 00000100:00080000:0.0:1787829398.048967:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829398.048973:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829398.048983:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829398.048997:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043328384:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787829398.049001:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5f80 x1874663043328512/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829398.049015:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829398.049019:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829398.049021:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829398.049026:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11464 00000100:00080000:0.0:1787829398.049030:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829398.049035:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829398.049041:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829398.049056:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7100 x1874663043328640/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829398.049060:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829398.049105:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829398/real 1787829398] req@ffff9181c55a4380 x1874663043328384/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829423 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829398.049124:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4380 x1874663043328384/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829423 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829398.049133:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829398.049136:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829398.049148:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043328384:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829398.049170:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043328512:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829398.049190:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829398.049199:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829398/real 1787829398] req@ffff9181c55a5f80 x1874663043328512/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829423 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829398.049207:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5f80 x1874663043328512/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829423 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829398.049213:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829398.049215:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829398.049217:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043328512:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829398.049221:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043328640:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829398.049231:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829398.049238:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829398/real 1787829398] req@ffff9181c55a7100 x1874663043328640/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829423 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829398.049245:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7100 x1874663043328640/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829423 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829398.049254:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829398.049256:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829398.049259:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043328640:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787829398.112853:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829399.136903:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787829399.648864:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'params' failed (-108). 10000000:00020000:1.0:1787829399.648875:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:1.0:1787829399.648880:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:1.0:1787829399.648883:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff9181c87b19b8 from 1 10000000:01000000:1.0:1787829399.648887:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00010000:00010000:1.0:1787829399.648930:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f8b7 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829399.648942:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f8b7 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829399.648954:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f8b7 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787829399.648977:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215db40 going to sleep for 0 seconds 00000100:00100000:1.0:1787829399.648988:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043328768:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787829399.649002:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f8b7 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829399.649016:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f8b7 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829399.649023:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f8b7 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829399.649031:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f8b7 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829399.649038:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f8b7 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829399.649045:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4800 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787829399.649049:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f8b7 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829399.649057:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f8b7 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787829399.649064:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f8b7 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787829399.649070:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1787829399.649073:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787829399.649079:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f8b7 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829399.649087:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b4800/0xbef42c0221d6f8b7 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787829399.649107:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787829399.649110:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:0.0:1787829399.649168:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829399.649173:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829399.649176:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829399.649181:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11469 00000100:00080000:0.0:1787829399.649189:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829399.649193:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829399.649212:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5880 x1874663043328896/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829399.649231:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829399.649234:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829399.649235:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829399.649239:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11469 00000100:00080000:0.0:1787829399.649241:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829399.649244:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829399.649251:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829399.649261:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6d80 x1874663043329024/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829399.649268:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829399.649270:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829399.649272:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829399.649277:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11469 00000100:00100000:2.0:1787829399.649278:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043328896:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787829399.649280:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829399.649287:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829399.649290:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829399.649298:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6680 x1874663043329152/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829399.649334:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829399.649360:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829399/real 1787829399] req@ffff9181c55a5880 x1874663043328896/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829424 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829399.649372:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5880 x1874663043328896/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829424 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829399.649380:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829399.649382:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829399.649392:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043328896:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829399.649407:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043329024:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829399.649425:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829399.649434:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829399/real 1787829399] req@ffff9181c55a6d80 x1874663043329024/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829424 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829399.649443:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6d80 x1874663043329024/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829424 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829399.649448:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829399.649451:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829399.649454:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043329024:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829399.649457:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043329152:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829399.649468:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829399.649474:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829399/real 1787829399] req@ffff9181c55a6680 x1874663043329152/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829424 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829399.649482:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6680 x1874663043329152/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829424 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829399.649487:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829399.649488:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829399.649490:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043329152:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829400.160917:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829401.184887:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829402.208877:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829403.232873:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829404.256900:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829404.704839:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829404.704848:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829404.704851:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829404.704859:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11471 00000100:00080000:2.0:1787829404.704868:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829404.704874:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787829404.704906:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d880 x1874663043329280/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829404.704929:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829404.704933:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829404.704935:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829404.704940:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11471 00000100:00080000:2.0:1787829404.704943:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829404.704948:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829404.704958:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829404.704963:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043329280:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829404.704970:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ce00 x1874663043329408/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829404.704980:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829404.704982:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829404.704984:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829404.704987:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11471 00000100:00080000:2.0:1787829404.704990:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829404.704993:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829404.704998:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829404.705007:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f800 x1874663043329536/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829404.705022:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829404.705058:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829404/real 1787829404] req@ffff9181e007d880 x1874663043329280/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829429 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829404.705072:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d880 x1874663043329280/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829429 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829404.705080:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829404.705082:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829404.705093:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043329280:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787829404.705112:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043329408:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787829404.705134:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829404.705147:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829404/real 1787829404] req@ffff9181e007ce00 x1874663043329408/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829429 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829404.705159:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ce00 x1874663043329408/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829429 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829404.705168:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829404.705172:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829404.705175:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043329408:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787829404.705183:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043329536:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787829404.705209:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829404.705218:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829404/real 1787829404] req@ffff9181e007f800 x1874663043329536/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829429 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829404.705227:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f800 x1874663043329536/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829429 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829404.705232:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829404.705234:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829404.705237:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043329536:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829405.280858:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829406.304880:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829407.328904:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829408.352894:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829409.376908:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829409.824835:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829409.824844:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829409.824848:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829409.824857:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11476 00000100:00080000:2.0:1787829409.824867:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829409.824875:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787829409.824914:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f480 x1874663043329664/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829409.824944:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829409.824949:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829409.824951:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829409.824955:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11476 00000100:00080000:2.0:1787829409.824959:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829409.824963:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829409.824973:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829409.824990:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f100 x1874663043329792/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829409.825002:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829409.825007:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:0.0:1787829409.825015:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043329664:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787829409.825016:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829409.825035:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11476 00000100:00080000:2.0:1787829409.825040:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829409.825046:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829409.825053:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829409.825070:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ea00 x1874663043329920/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829409.825102:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829409.825151:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829409/real 1787829409] req@ffff9181e007f480 x1874663043329664/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829434 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829409.825172:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f480 x1874663043329664/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829434 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829409.825182:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829409.825186:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829409.825207:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043329664:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787829409.825235:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043329792:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787829409.825257:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829409.825267:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829409/real 1787829409] req@ffff9181e007f100 x1874663043329792/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829434 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829409.825277:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f100 x1874663043329792/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829434 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829409.825284:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829409.825286:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829409.825289:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043329792:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787829409.825294:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043329920:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787829409.825306:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829409.825312:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829409/real 1787829409] req@ffff9181e007ea00 x1874663043329920/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829434 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829409.825320:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ea00 x1874663043329920/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829434 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829409.825325:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829409.825327:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829409.825329:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043329920:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829410.400928:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829411.424960:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829412.448964:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829413.472974:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829414.496934:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829414.944878:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829414.944888:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829414.944908:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829414.944919:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11481 00000100:00080000:2.0:1787829414.944931:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829414.944938:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787829414.944980:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c700 x1874663043330048/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829414.945011:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829414.945015:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829414.945017:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829414.945022:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11481 00000100:00080000:2.0:1787829414.945026:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829414.945030:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829414.945042:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829414.945056:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e300 x1874663043330176/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829414.945064:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829414.945067:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829414.945070:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829414.945073:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11481 00000100:00080000:2.0:1787829414.945076:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829414.945079:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829414.945083:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829414.945089:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043330048:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829414.945094:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d180 x1874663043330304/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829414.945169:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829414.945215:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829414/real 1787829414] req@ffff9181e007c700 x1874663043330048/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829439 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829414.945232:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c700 x1874663043330048/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829439 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829414.945246:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829414.945249:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829414.945261:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043330048:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787829414.945267:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043330176:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787829414.945292:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829414.945312:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829414/real 1787829414] req@ffff9181e007e300 x1874663043330176/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829439 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829414.945327:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e300 x1874663043330176/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829439 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829414.945336:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829414.945339:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829414.945342:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043330176:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787829414.945371:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043330304:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787829414.945396:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829414.945410:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829414/real 1787829414] req@ffff9181e007d180 x1874663043330304/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829439 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829414.945424:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d180 x1874663043330304/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829439 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829414.945432:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829414.945435:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829414.945437:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043330304:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829415.520933:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829416.544902:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829417.568898:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829418.592881:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829419.616895:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829420.064835:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829420.064845:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829420.064849:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829420.064860:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11486 00000100:00080000:2.0:1787829420.064872:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 10000000:01000000:1.0:1787829420.064876:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-sptlrpc' failed (-108). 00000100:00080000:2.0:1787829420.064881:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 10000000:00020000:1.0:1787829420.064883:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00100000:2.0:1787829420.064922:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d500 x1874663043330432/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829420.064953:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829420.064956:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829420.064959:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829420.064963:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11486 00000100:00080000:2.0:1787829420.064967:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829420.064972:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829420.064982:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829420.064994:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007df80 x1874663043330560/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829420.065002:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829420.065005:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829420.065006:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829420.065011:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11486 00000100:00100000:0.0:1787829420.065012:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043330432:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787829420.065014:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829420.065021:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829420.065025:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829420.065033:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ed80 x1874663043330688/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829420.065082:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829420.065118:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829420/real 1787829420] req@ffff9181e007d500 x1874663043330432/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829445 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829420.065139:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d500 x1874663043330432/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829445 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829420.065150:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829420.065154:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829420.065159:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043330432:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787829420.065180:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043330560:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787829420.065204:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829420.065217:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829420/real 1787829420] req@ffff9181e007df80 x1874663043330560/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829445 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829420.065230:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007df80 x1874663043330560/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829445 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829420.065239:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829420.065242:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829420.065246:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043330560:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787829420.065253:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043330688:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787829420.065272:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829420.065283:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829420/real 1787829420] req@ffff9181e007ed80 x1874663043330688/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829445 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829420.065298:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ed80 x1874663043330688/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829445 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829420.065306:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829420.065310:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829420.065313:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043330688:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829420.640948:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787829421.344891:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:1.0:1787829421.344902:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff9181c5620000 from 341 10000000:01000000:1.0:1787829421.344911:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:1.0:1787829421.344982:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8be lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829421.344994:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8be lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829421.345005:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8be lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787829421.345040:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215db40 going to sleep for 0 seconds 00000100:00100000:1.0:1787829421.345055:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043330816:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787829421.345074:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8be lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829421.345081:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8be lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829421.345086:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8be lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829421.345091:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8be lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829421.345096:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8be lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829421.345101:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b5000 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787829421.345104:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8be lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829421.345110:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8be lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787829421.345115:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8be lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787829421.345119:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:1.0:1787829421.345123:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787829421.345129:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8be lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829421.345134:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8be lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787829421.345156:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787829421.345159:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:2.0:1787829421.345244:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829421.345252:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829421.345256:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829421.345264:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11491 00000100:00080000:2.0:1787829421.345273:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829421.345281:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787829421.345307:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ed80 x1874663043330944/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829421.345334:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829421.345338:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829421.345340:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829421.345345:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11491 00000100:00080000:2.0:1787829421.345348:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829421.345354:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829421.345365:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829421.345382:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007df80 x1874663043331072/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787829421.345385:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043330944:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787829421.345419:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829421.345424:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829421.345426:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829421.345434:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11491 00000100:00080000:2.0:1787829421.345439:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829421.345445:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829421.345450:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829421.345465:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d500 x1874663043331200/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829421.345476:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829421.345524:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829421/real 1787829421] req@ffff9181e007ed80 x1874663043330944/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829446 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829421.345540:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ed80 x1874663043330944/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829446 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829421.345549:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829421.345552:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829421.345566:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043330944:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787829421.345591:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043331072:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787829421.345619:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829421.345628:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829421/real 1787829421] req@ffff9181e007df80 x1874663043331072/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829446 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829421.345637:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007df80 x1874663043331072/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829446 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829421.345647:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829421.345650:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829421.345652:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043331072:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787829421.345657:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043331200:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787829421.345674:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829421.345681:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829421/real 1787829421] req@ffff9181e007d500 x1874663043331200/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829446 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829421.345689:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d500 x1874663043331200/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829446 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829421.345694:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829421.345696:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829421.345698:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043331200:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829421.664943:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829422.689043:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829423.712913:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829424.736941:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829425.760891:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829426.720828:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829426.720837:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829426.720841:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829426.720849:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11493 00000100:00080000:2.0:1787829426.720859:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829426.720866:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787829426.720903:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d180 x1874663043331328/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829426.720930:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829426.720935:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829426.720937:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829426.720942:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11493 00000100:00080000:2.0:1787829426.720948:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829426.720953:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829426.720963:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829426.720968:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043331328:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829426.720981:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e300 x1874663043331456/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829426.720995:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829426.720999:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829426.721002:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829426.721021:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11493 00000100:00080000:2.0:1787829426.721026:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000800:00000100:0.0:1787829426.721031:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1787829426.721034:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829426.721042:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829426.721057:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c700 x1874663043331584/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787829426.721065:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829426/real 1787829426] req@ffff9181e007d180 x1874663043331328/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829451 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829426.721084:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d180 x1874663043331328/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829451 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829426.721092:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829426.721095:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829426.721107:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043331328:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787829426.721126:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043331456:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787829426.721144:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829426.721155:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829426/real 1787829426] req@ffff9181e007e300 x1874663043331456/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829451 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829426.721164:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e300 x1874663043331456/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829451 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829426.721170:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829426.721172:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829426.721174:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043331456:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787829426.721178:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043331584:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787829426.721189:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829426.721194:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829426/real 1787829426] req@ffff9181e007c700 x1874663043331584/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829451 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829426.721202:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c700 x1874663043331584/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829451 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829426.721207:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829426.721209:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829426.721211:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043331584:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829426.784879:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829427.808890:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829428.832900:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829429.856860:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829430.880862:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829431.840865:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829431.840875:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829431.840880:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829431.840892:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11498 00000100:00080000:0.0:1787829431.840902:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829431.840910:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829431.840946:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7b80 x1874663043331712/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829431.840971:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829431.840975:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829431.840977:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829431.840981:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11498 00000100:00080000:0.0:1787829431.840986:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829431.840991:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829431.841001:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829431.841013:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043331712:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787829431.841018:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7480 x1874663043331840/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829431.841030:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829431.841033:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829431.841034:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829431.841037:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11498 00000100:00080000:0.0:1787829431.841040:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829431.841043:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829431.841047:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829431.841057:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6680 x1874663043331968/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829431.841070:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829431.841102:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829431/real 1787829431] req@ffff9181c55a7b80 x1874663043331712/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829456 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829431.841114:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7b80 x1874663043331712/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829456 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829431.841122:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829431.841125:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829431.841136:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043331712:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829431.841154:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043331840:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829431.841169:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829431.841177:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829431/real 1787829431] req@ffff9181c55a7480 x1874663043331840/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829456 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829431.841188:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7480 x1874663043331840/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829456 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829431.841194:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829431.841197:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829431.841201:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043331840:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829431.841206:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043331968:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829431.841219:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829431.841225:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829431/real 1787829431] req@ffff9181c55a6680 x1874663043331968/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829456 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829431.841233:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6680 x1874663043331968/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829456 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829431.841237:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829431.841239:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829431.841241:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043331968:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829431.904895:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829432.928858:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829433.952870:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829434.976851:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829436.000919:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829436.960854:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829436.960865:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829436.960869:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829436.960880:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11503 00000100:00080000:2.0:1787829436.960891:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829436.960899:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787829436.960937:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ea00 x1874663043332096/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829436.960967:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829436.960972:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829436.960974:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829436.960979:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11503 00000100:00080000:2.0:1787829436.960984:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829436.960990:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829436.961001:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829436.961006:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043332096:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829436.961020:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f100 x1874663043332224/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829436.961035:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829436.961039:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829436.961041:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829436.961046:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11503 00000100:00080000:2.0:1787829436.961050:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829436.961055:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:0.0:1787829436.961061:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:2.0:1787829436.961062:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829436.961081:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f480 x1874663043332352/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787829436.961095:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829436/real 1787829436] req@ffff9181e007ea00 x1874663043332096/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829461 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829436.961111:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ea00 x1874663043332096/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829461 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829436.961120:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829436.961123:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829436.961134:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043332096:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787829436.961155:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043332224:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787829436.961174:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829436.961184:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829436/real 1787829436] req@ffff9181e007f100 x1874663043332224/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829461 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829436.961193:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f100 x1874663043332224/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829461 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829436.961198:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829436.961200:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829436.961202:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043332224:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787829436.961206:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043332352:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787829436.961216:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829436.961225:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829436/real 1787829436] req@ffff9181e007f480 x1874663043332352/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829461 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829436.961238:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f480 x1874663043332352/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829461 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829436.961247:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829436.961250:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829436.961253:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043332352:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787829437.024855:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829438.048879:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829439.072870:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829440.096872:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829441.120950:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787829441.568866:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-cliir' failed (-108). 10000000:00000400:1.0:1787829441.568874:0:159735:0:(mgc_request.c:1900:mgc_process_log()) MGC192.168.203.147@tcp: IR log lustre-cliir failed, not fatal: rc = -108 10000000:01000000:1.0:1787829441.568880:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:1.0:1787829441.568883:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff9181c5620000 from 52 10000000:01000000:1.0:1787829441.568885:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:1.0:1787829441.568921:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8c5 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829441.568931:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8c5 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829441.568940:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8c5 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787829441.568961:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215db40 going to sleep for 0 seconds 00000100:00100000:1.0:1787829441.568976:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043332480:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787829441.568990:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8c5 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829441.569009:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8c5 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829441.569015:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8c5 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829441.569021:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8c5 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829441.569027:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8c5 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829441.569031:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b5000 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787829441.569033:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8c5 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829441.569038:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8c5 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787829441.569043:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8c5 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787829441.569047:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1787829441.569050:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787829441.569055:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8c5 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829441.569063:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8c5 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787829441.569088:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787829441.569092:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:0.0:1787829441.569149:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829441.569154:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829441.569157:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829441.569162:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11508 00000100:00080000:0.0:1787829441.569170:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829441.569175:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829441.569195:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6d80 x1874663043332608/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829441.569222:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829441.569226:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829441.569228:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829441.569233:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11508 00000100:00080000:0.0:1787829441.569237:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829441.569243:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829441.569252:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829441.569276:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043332608:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787829441.569279:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5880 x1874663043332736/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829441.569291:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829441.569293:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829441.569294:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829441.569298:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11508 00000100:00080000:0.0:1787829441.569301:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829441.569304:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829441.569308:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829441.569316:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7100 x1874663043332864/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829441.569341:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829441.569369:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829441/real 1787829441] req@ffff9181c55a6d80 x1874663043332608/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829466 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829441.569380:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6d80 x1874663043332608/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829466 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829441.569396:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829441.569399:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829441.569408:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043332608:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829441.569423:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043332736:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829441.569440:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829441.569449:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829441/real 1787829441] req@ffff9181c55a5880 x1874663043332736/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829466 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829441.569458:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5880 x1874663043332736/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829466 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829441.569463:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829441.569466:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829441.569468:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043332736:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829441.569472:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043332864:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829441.569482:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829441.569488:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829441/real 1787829441] req@ffff9181c55a7100 x1874663043332864/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829466 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829441.569496:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7100 x1874663043332864/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829466 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829441.569501:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829441.569503:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829441.569505:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043332864:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787829442.144859:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829443.168869:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829444.192918:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829445.216898:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829446.240923:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829446.688880:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829446.688888:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829446.688891:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829446.688899:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11513 00000100:00080000:2.0:1787829446.688908:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829446.688913:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787829446.688942:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f800 x1874663043332992/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829446.688966:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829446.688970:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829446.688972:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829446.688978:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11513 00000100:00080000:2.0:1787829446.688982:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829446.688988:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829446.688998:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829446.689008:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043332992:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829446.689014:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ce00 x1874663043333120/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829446.689027:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829446.689031:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829446.689033:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829446.689037:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11513 00000100:00080000:2.0:1787829446.689041:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829446.689046:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829446.689052:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829446.689065:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d880 x1874663043333248/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787829446.689119:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043333120:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787829446.689156:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043333248:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829447.264875:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829448.288935:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829449.312884:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1787829449.761028:0:2379:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:1.0:1787829449.761066:0:2379:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787829449.761084:0:2379:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1787829449.761128:0:2379:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1787829449.761137:0:2379:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:0.0:1787829449.761190:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829446/real 1787829449] req@ffff9181e007ce00 x1874663043333120/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829471 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829449.761232:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ce00 x1874663043333120/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829471 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829449.761244:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829449.761249:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829449.761254:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043333120:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787829449.761264:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829446/real 1787829449] req@ffff9181e007d880 x1874663043333248/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829471 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829449.761276:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d880 x1874663043333248/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829471 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829449.761284:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829449.761287:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829449.761290:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043333248:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787829449.761298:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829446/real 1787829449] req@ffff9181e007f800 x1874663043332992/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829471 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829449.761315:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f800 x1874663043332992/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829471 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829449.761324:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829449.761327:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829449.761348:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043332992:192.168.203.147@tcp:250:kworker.0 00010000:00010000:2.0:1787829450.336890:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829451.360897:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829451.808934:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829451.808944:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829451.808948:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829451.808959:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11518 00000100:00080000:0.0:1787829451.808970:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829451.808978:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829451.809010:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5f80 x1874663043333376/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829451.809050:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829451.809054:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829451.809056:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829451.809060:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11518 00000100:00080000:0.0:1787829451.809065:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829451.809070:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829451.809081:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829451.809094:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043333376:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787829451.809099:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4380 x1874663043333504/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829451.809114:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829451.809118:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829451.809121:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829451.809126:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11518 00000100:00080000:0.0:1787829451.809130:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829451.809135:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829451.809141:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829451.809153:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4700 x1874663043333632/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829451.809169:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829451.809230:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829451/real 1787829451] req@ffff9181c55a5f80 x1874663043333376/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829476 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829451.809250:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5f80 x1874663043333376/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829476 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829451.809263:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829451.809267:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829451.809289:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043333376:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829451.809319:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043333504:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829451.809346:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829451.809361:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829451/real 1787829451] req@ffff9181c55a4380 x1874663043333504/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829476 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829451.809375:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4380 x1874663043333504/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829476 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829451.809384:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829451.809387:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829451.809390:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043333504:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829451.809397:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043333632:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829451.809417:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829451.809429:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829451/real 1787829451] req@ffff9181c55a4700 x1874663043333632/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829476 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829451.809442:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4700 x1874663043333632/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829476 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829451.809450:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829451.809453:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829451.809456:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043333632:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829452.384882:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829453.408902:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829454.432873:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829455.456889:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829456.480930:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829456.928861:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829456.928871:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829456.928876:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829456.928886:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11523 00000100:00080000:0.0:1787829456.928897:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829456.928904:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829456.928956:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5c00 x1874663043333760/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829456.928982:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829456.928985:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829456.928986:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829456.928990:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11523 00000100:00080000:0.0:1787829456.928992:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829456.928995:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829456.929004:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829456.929014:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5180 x1874663043333888/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829456.929022:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829456.929025:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829456.929027:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829456.929029:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11523 00000100:00080000:0.0:1787829456.929031:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829456.929034:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829456.929037:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829456.929046:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5500 x1874663043334016/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829456.929058:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043333760:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787829456.929138:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829456.929196:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829456/real 1787829456] req@ffff9181c55a5c00 x1874663043333760/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829481 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829456.929218:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5c00 x1874663043333760/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829481 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829456.929230:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829456.929235:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829456.929255:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043333760:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829456.929265:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043333888:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829456.929294:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829456.929312:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829456/real 1787829456] req@ffff9181c55a5180 x1874663043333888/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829481 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829456.929326:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5180 x1874663043333888/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829481 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829456.929335:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829456.929338:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829456.929341:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043333888:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829456.929368:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043334016:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829456.929391:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829456.929403:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829456/real 1787829456] req@ffff9181c55a5500 x1874663043334016/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829481 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829456.929416:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5500 x1874663043334016/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829481 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829456.929424:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829456.929428:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829456.929432:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043334016:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829457.504939:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829458.528973:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829459.552935:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829460.576931:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829461.600915:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829462.048887:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 10000000:01000000:1.0:1787829462.048900:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-client' failed (-108). 00000100:00080000:2.0:1787829462.048901:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829462.048907:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 10000000:00020000:1.0:1787829462.048912:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:1.0:1787829462.048921:0:159735:0:(mgc_request.c:611:do_requeue()) updating log params 00000100:00080000:2.0:1787829462.048923:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11528 10000000:01000000:1.0:1787829462.048926:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff9181c5620000 from 7 10000000:01000000:1.0:1787829462.048932:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00000100:00080000:2.0:1787829462.048939:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829462.048951:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:1.0:1787829462.048990:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8cc lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787829462.048994:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ca80 x1874663043334272/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787829462.049031:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8cc lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829462.049046:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8cc lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787829462.049053:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829462.049062:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00100000:1.0:1787829462.049066:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215db40 going to sleep for 0 seconds 00000100:00080000:2.0:1787829462.049067:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1787829462.049081:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043334144:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00080000:2.0:1787829462.049081:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11528 00000100:00080000:2.0:1787829462.049095:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1787829462.049097:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043334272:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787829462.049103:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:1.0:1787829462.049104:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8cc lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000008:00100000:2.0:1787829462.049133:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:1.0:1787829462.049137:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8cc lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829462.049148:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8cc lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787829462.049153:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007fb80 x1874663043334400/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787829462.049158:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8cc lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829462.049190:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8cc lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000800:00000100:0.0:1787829462.049191:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1787829462.049191:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829462.049206:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829462.049214:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00010000:00010000:1.0:1787829462.049217:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b5000 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787829462.049221:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8cc lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787829462.049222:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11528 00000100:00080000:2.0:1787829462.049231:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00010000:00010000:1.0:1787829462.049235:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8cc lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787829462.049239:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829462/real 1787829462] req@ffff9181e007ca80 x1874663043334272/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829487 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829462.049239:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829462.049253:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829462.049275:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e680 x1874663043334528/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 10000000:00010000:1.0:1787829462.049276:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8cc lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787829462.049291:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ca80 x1874663043334272/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829487 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 10000000:01000000:1.0:1787829462.049320:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:1.0:1787829462.049334:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00000100:00080000:0.0:1787829462.049339:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00010000:00010000:1.0:1787829462.049344:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8cc lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787829462.049349:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00010000:00010000:1.0:1787829462.049359:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8cc lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787829462.049365:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043334272:192.168.203.147@tcp:250:kworker.0 10000000:01000000:1.0:1787829462.049393:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787829462.049397:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00100000:0.0:1787829462.049408:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043334400:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787829462.049442:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1787829462.049456:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1787829462.049461:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829462/real 1787829462] req@ffff9181e007fb80 x1874663043334400/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829487 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829462.049468:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829462.049472:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1787829462.049481:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007fb80 x1874663043334400/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829487 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829462.049481:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11533 00000100:00080000:2.0:1787829462.049487:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829462.049493:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829462.049496:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00080000:0.0:1787829462.049500:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829462.049504:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043334400:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787829462.049509:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043334528:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829462.049516:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007dc00 x1874663043334656/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829462.049529:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829462.049549:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829462/real 1787829462] req@ffff9181e007e680 x1874663043334528/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829487 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829462.049562:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e680 x1874663043334528/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829487 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829462.049567:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829462.049569:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829462.049571:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043334528:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787829462.049581:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043334656:192.168.203.147@tcp:250:kworker.0 00000800:00000100:0.0:1787829462.049593:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829462.049599:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829462/real 1787829462] req@ffff9181e007dc00 x1874663043334656/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829487 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829462.049607:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007dc00 x1874663043334656/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829487 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829462.049612:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829462.049614:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829462.049624:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043334656:192.168.203.147@tcp:250:kworker.0 00010000:00010000:3.0:1787829462.624903:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829463.648935:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829464.672905:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829465.696994:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829466.720887:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829467.168873:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829467.168887:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829467.168894:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829467.168907:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11533 00000100:00080000:0.0:1787829467.168921:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829467.168930:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829467.168981:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7800 x1874663043334784/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829467.169045:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043334784:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787829467.169131:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:0.0:1787829467.169227:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829467.169232:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829467.169234:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829467.169240:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11533 00000100:00080000:0.0:1787829467.169245:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829467.169249:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787829467.169253:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829467/real 1787829467] req@ffff9181c55a7800 x1874663043334784/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829492 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:0.0:1787829467.169261:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829467.169272:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7800 x1874663043334784/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829492 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829467.169278:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4e00 x1874663043334912/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829467.169291:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829467.169293:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829467.169298:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829467.169301:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829467.169301:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00080000:0.0:1787829467.169310:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11533 00000100:00080000:0.0:1787829467.169312:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829467.169315:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829467.169318:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829467.169324:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043334784:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787829467.169329:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5500 x1874663043335040/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829467.169358:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043334912:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829467.169389:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829467.169406:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829467/real 1787829467] req@ffff9181c55a4e00 x1874663043334912/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829492 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829467.169421:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4e00 x1874663043334912/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829492 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829467.169430:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829467.169432:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829467.169435:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043334912:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829467.169439:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043335040:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829467.169460:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829467.169469:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829467/real 1787829467] req@ffff9181c55a5500 x1874663043335040/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829492 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829467.169482:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5500 x1874663043335040/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829492 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829467.169490:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829467.169494:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829467.169498:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043335040:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829467.744931:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829468.768871:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829469.792936:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829470.816939:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829471.840891:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829472.288842:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829472.288849:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829472.288853:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829472.288861:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11538 00000100:00080000:2.0:1787829472.288870:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829472.288876:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787829472.288906:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c380 x1874663043335168/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829472.288926:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829472.288929:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829472.288931:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829472.288934:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11538 00000100:00080000:2.0:1787829472.288936:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829472.288939:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829472.288946:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829472.288955:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c000 x1874663043335296/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829472.288964:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829472.288968:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:0.0:1787829472.288969:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043335168:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787829472.288969:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829472.288976:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11538 00000100:00080000:2.0:1787829472.288981:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829472.288984:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829472.288987:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829472.288997:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007dc00 x1874663043335424/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829472.289059:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829472.289097:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829472/real 1787829472] req@ffff9181e007c380 x1874663043335168/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829497 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829472.289125:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c380 x1874663043335168/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829497 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829472.289137:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829472.289141:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829472.289155:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043335168:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787829472.289181:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043335296:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787829472.289210:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829472.289222:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829472/real 1787829472] req@ffff9181e007c000 x1874663043335296/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829497 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829472.289232:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c000 x1874663043335296/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829497 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829472.289238:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829472.289240:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829472.289242:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043335296:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787829472.289246:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043335424:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787829472.289259:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829472.289266:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829472/real 1787829472] req@ffff9181e007dc00 x1874663043335424/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829497 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829472.289273:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007dc00 x1874663043335424/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829497 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829472.289283:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829472.289284:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829472.289286:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043335424:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829472.864891:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829473.888898:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829474.912910:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829475.936931:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829476.960927:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829477.408857:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829477.408875:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829477.408879:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829477.408890:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11544 00000100:00080000:2.0:1787829477.408901:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829477.408909:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787829477.408946:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e680 x1874663043335552/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829477.408977:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829477.408982:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829477.408984:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829477.408989:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11544 00000100:00080000:2.0:1787829477.408993:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829477.408999:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829477.409021:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829477.409038:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007fb80 x1874663043335680/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787829477.409044:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043335552:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787829477.409058:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829477.409063:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829477.409065:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829477.409071:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11544 00000100:00080000:2.0:1787829477.409075:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829477.409082:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829477.409088:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829477.409102:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ca80 x1874663043335808/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829477.409113:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829477.409151:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829477/real 1787829477] req@ffff9181e007e680 x1874663043335552/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829502 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829477.409172:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e680 x1874663043335552/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829502 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829477.409183:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829477.409187:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829477.409198:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043335552:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787829477.409221:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043335680:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787829477.409252:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829477.409270:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829477/real 1787829477] req@ffff9181e007fb80 x1874663043335680/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829502 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829477.409285:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007fb80 x1874663043335680/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829502 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829477.409294:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829477.409298:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829477.409301:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043335680:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787829477.409309:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043335808:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787829477.409331:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829477.409345:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829477/real 1787829477] req@ffff9181e007ca80 x1874663043335808/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829502 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829477.409359:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ca80 x1874663043335808/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829502 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829477.409367:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829477.409371:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829477.409375:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043335808:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829477.984870:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829479.008906:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829480.032876:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829481.056848:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829482.080860:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829482.528845:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829482.528852:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829482.528855:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829482.528863:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11549 10000000:01000000:1.0:1787829482.528869:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'params' failed (-108). 00000100:00080000:2.0:1787829482.528872:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 10000000:00020000:1.0:1787829482.528874:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:1.0:1787829482.528878:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 00000100:00080000:2.0:1787829482.528881:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 10000000:01000000:1.0:1787829482.528882:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff9181c87b19b8 from 1 10000000:01000000:1.0:1787829482.528886:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00000100:00100000:2.0:1787829482.528911:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f800 x1874663043336064/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787829482.528920:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8d3 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829482.528928:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8d3 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787829482.528928:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829482.528935:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829482.528939:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829482.528942:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11549 00010000:00010000:1.0:1787829482.528943:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8d3 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787829482.528945:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1787829482.528951:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043336064:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787829482.528952:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787829482.528960:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215db40 going to sleep for 0 seconds 00000008:00100000:2.0:1787829482.528962:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829482.528967:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043335936:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00100000:2.0:1787829482.528986:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d880 x1874663043336192/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787829482.528993:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8d3 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787829482.528995:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829482.529003:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829482.529006:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00010000:00010000:1.0:1787829482.529007:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8d3 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787829482.529012:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11549 00010000:00010000:1.0:1787829482.529016:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8d3 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000800:00000100:0.0:1787829482.529017:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1787829482.529018:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829482.529026:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:1.0:1787829482.529033:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8d3 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000008:00100000:2.0:1787829482.529035:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:1.0:1787829482.529044:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8d3 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787829482.529047:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829482/real 1787829482] req@ffff9181e007f800 x1874663043336064/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829507 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:1.0:1787829482.529052:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b5000 already unused, calling callback (ffffffffc1273c20) 00000100:00100000:2.0:1787829482.529054:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ce00 x1874663043336320/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 10000000:00010000:1.0:1787829482.529057:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8d3 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787829482.529076:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f800 x1874663043336064/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829507 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:1.0:1787829482.529077:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8d3 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787829482.529092:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8d3 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787829482.529095:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 10000000:01000000:1.0:1787829482.529098:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00000100:00080000:0.0:1787829482.529101:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00010000:00010000:1.0:1787829482.529105:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00000100:00100000:0.0:1787829482.529106:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043336064:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787829482.529114:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8d3 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787829482.529123:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043336192:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829482.529129:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8d3 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000800:00000100:0.0:1787829482.529138:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829482.529150:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829482/real 1787829482] req@ffff9181e007d880 x1874663043336192/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829507 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 10000000:01000000:1.0:1787829482.529152:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787829482.529157:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00100000:0.0:1787829482.529163:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d880 x1874663043336192/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829507 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829482.529169:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829482.529172:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829482.529174:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043336192:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787829482.529178:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043336320:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787829482.529189:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1787829482.529195:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1787829482.529198:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829482/real 1787829482] req@ffff9181e007ce00 x1874663043336320/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829507 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829482.529202:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829482.529207:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1787829482.529211:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ce00 x1874663043336320/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829507 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829482.529215:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11554 00000100:00080000:0.0:1787829482.529221:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829482.529223:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829482.529225:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00080000:2.0:1787829482.529232:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829482.529233:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043336320:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829482.529252:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f480 x1874663043336448/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787829482.529279:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043336448:192.168.203.147@tcp:250:kworker.0 00000800:00000100:0.0:1787829482.529290:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829482.529297:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829482/real 1787829482] req@ffff9181e007f480 x1874663043336448/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829507 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829482.529304:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f480 x1874663043336448/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829507 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829482.529309:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829482.529311:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829482.529320:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043336448:192.168.203.147@tcp:250:kworker.0 00010000:00010000:2.0:1787829483.104904:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829484.128874:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829485.152895:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829486.176860:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829487.200846:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829487.648847:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829487.648857:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829487.648861:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829487.648870:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11554 00000100:00080000:0.0:1787829487.648880:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829487.648887:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829487.648921:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5180 x1874663043336576/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829487.648957:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829487.648961:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829487.648963:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829487.648968:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11554 00000100:00080000:0.0:1787829487.648971:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829487.648975:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829487.648984:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829487.648997:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043336576:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787829487.648999:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5c00 x1874663043336704/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829487.649013:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829487.649016:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829487.649018:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829487.649022:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11554 00000100:00080000:0.0:1787829487.649026:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829487.649030:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829487.649035:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829487.649049:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4700 x1874663043336832/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829487.649052:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829487.649087:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829487/real 1787829487] req@ffff9181c55a5180 x1874663043336576/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829512 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829487.649103:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5180 x1874663043336576/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829512 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829487.649110:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829487.649112:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829487.649122:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043336576:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829487.649139:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043336704:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829487.649154:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829487.649162:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829487/real 1787829487] req@ffff9181c55a5c00 x1874663043336704/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829512 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829487.649174:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5c00 x1874663043336704/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829512 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829487.649182:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829487.649185:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829487.649189:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043336704:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829487.649194:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043336832:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829487.649214:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829487.649225:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829487/real 1787829487] req@ffff9181c55a4700 x1874663043336832/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829512 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829487.649233:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4700 x1874663043336832/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829512 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829487.649242:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829487.649244:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829487.649246:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043336832:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829488.224848:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829489.248855:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829490.272931:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829491.296951:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829492.320914:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829492.768849:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829492.768856:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829492.768860:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829492.768868:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11559 00000100:00080000:0.0:1787829492.768876:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829492.768882:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829492.768931:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4380 x1874663043336960/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829492.768950:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829492.768953:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829492.768954:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829492.768956:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11559 00000100:00080000:0.0:1787829492.768959:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829492.768961:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829492.768968:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829492.768976:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5f80 x1874663043337088/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829492.768984:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1787829492.768988:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043336960:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787829492.768989:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829492.768992:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829492.768999:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11559 00000100:00080000:0.0:1787829492.769001:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829492.769003:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829492.769006:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829492.769014:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7100 x1874663043337216/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829492.769050:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829492.769075:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829492/real 1787829492] req@ffff9181c55a4380 x1874663043336960/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829517 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829492.769087:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4380 x1874663043336960/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829517 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829492.769093:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829492.769096:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829492.769106:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043336960:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829492.769127:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043337088:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829492.769149:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829492.769161:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829492/real 1787829492] req@ffff9181c55a5f80 x1874663043337088/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829517 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829492.769171:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5f80 x1874663043337088/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829517 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829492.769176:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829492.769178:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829492.769180:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043337088:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829492.769184:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043337216:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829492.769198:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829492.769204:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829492/real 1787829492] req@ffff9181c55a7100 x1874663043337216/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829517 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829492.769211:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7100 x1874663043337216/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829517 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829492.769215:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829492.769216:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829492.769218:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043337216:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829493.344869:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829494.368842:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829495.392901:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829496.416867:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829497.440880:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829497.888814:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829497.888820:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829497.888823:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829497.888830:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11564 00000100:00080000:2.0:1787829497.888838:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829497.888843:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787829497.888880:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f100 x1874663043337344/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829497.888898:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829497.888901:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829497.888902:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829497.888905:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11564 00000100:00080000:2.0:1787829497.888907:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829497.888910:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829497.888916:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829497.888928:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ea00 x1874663043337472/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787829497.888929:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043337344:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787829497.888941:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829497.888943:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829497.888943:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829497.888945:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11564 00000100:00080000:2.0:1787829497.888947:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829497.888950:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829497.888952:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829497.888958:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c700 x1874663043337600/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829497.888990:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829497.889020:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829497/real 1787829497] req@ffff9181e007f100 x1874663043337344/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829522 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829497.889033:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f100 x1874663043337344/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829522 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829497.889040:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829497.889043:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829497.889052:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043337344:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787829497.889068:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043337472:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787829497.889084:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829497.889095:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829497/real 1787829497] req@ffff9181e007ea00 x1874663043337472/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829522 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829497.889105:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ea00 x1874663043337472/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829522 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829497.889110:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829497.889112:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829497.889120:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043337472:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787829497.889124:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043337600:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787829497.889134:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829497.889140:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829497/real 1787829497] req@ffff9181e007c700 x1874663043337600/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829522 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829497.889147:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c700 x1874663043337600/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829522 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829497.889151:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829497.889153:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829497.889155:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043337600:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787829498.464832:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829499.488836:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829500.512862:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829501.536836:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829502.560860:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829503.008849:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829503.008857:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829503.008861:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829503.008869:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11569 10000000:01000000:1.0:1787829503.008873:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-sptlrpc' failed (-108). 10000000:00020000:1.0:1787829503.008877:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00080000:0.0:1787829503.008879:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829503.008888:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829503.008921:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5880 x1874663043337728/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829503.008942:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829503.008944:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829503.008946:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829503.008949:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11569 00000100:00080000:0.0:1787829503.008952:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829503.008956:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829503.008964:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829503.008968:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043337728:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787829503.008981:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6d80 x1874663043337856/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829503.008992:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829503.008995:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829503.008997:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829503.009001:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11569 00000100:00080000:0.0:1787829503.009005:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829503.009008:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829503.009012:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829503.009021:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6680 x1874663043337984/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829503.009025:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829503.009057:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829503/real 1787829503] req@ffff9181c55a5880 x1874663043337728/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829528 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829503.009072:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5880 x1874663043337728/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829528 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829503.009079:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829503.009082:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829503.009084:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043337728:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829503.009098:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043337856:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829503.009112:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829503.009119:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829503/real 1787829503] req@ffff9181c55a6d80 x1874663043337856/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829528 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829503.009126:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6d80 x1874663043337856/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829528 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829503.009131:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829503.009132:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829503.009134:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043337856:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829503.009137:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043337984:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829503.009146:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829503.009152:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829503/real 1787829503] req@ffff9181c55a6680 x1874663043337984/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829528 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829503.009158:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6680 x1874663043337984/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829528 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829503.009172:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829503.009174:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829503.009176:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043337984:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829503.584872:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787829504.224866:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:1.0:1787829504.224872:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff9181c5620000 from 341 10000000:01000000:1.0:1787829504.224875:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:1.0:1787829504.224921:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8da lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829504.224932:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8da lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829504.224944:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8da lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787829504.224964:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215db40 going to sleep for 0 seconds 00000100:00100000:1.0:1787829504.224975:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043338112:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787829504.224990:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8da lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829504.224997:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8da lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829504.225003:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8da lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829504.225010:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8da lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829504.225016:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8da lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829504.225022:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b5000 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787829504.225025:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8da lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829504.225031:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8da lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787829504.225038:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8da lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787829504.225043:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:1.0:1787829504.225046:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787829504.225051:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8da lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829504.225057:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b5000/0xbef42c0221d6f8da lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787829504.225076:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787829504.225079:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:0.0:1787829504.225137:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829504.225141:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829504.225144:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829504.225149:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11574 00000100:00080000:0.0:1787829504.225157:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829504.225161:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829504.225177:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7480 x1874663043338240/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829504.225195:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829504.225197:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829504.225198:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829504.225201:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11574 00000100:00080000:0.0:1787829504.225203:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829504.225205:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829504.225211:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829504.225219:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7b80 x1874663043338368/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829504.225226:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829504.225229:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829504.225231:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1787829504.225231:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043338240:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787829504.225238:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11574 00000100:00080000:0.0:1787829504.225240:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829504.225242:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829504.225245:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829504.225254:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6680 x1874663043338496/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829504.225289:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829504.225319:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829504/real 1787829504] req@ffff9181c55a7480 x1874663043338240/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829529 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829504.225331:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7480 x1874663043338240/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829529 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829504.225337:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829504.225340:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829504.225348:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043338240:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829504.225362:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043338368:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829504.225398:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829504.225412:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829504/real 1787829504] req@ffff9181c55a7b80 x1874663043338368/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829529 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829504.225424:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7b80 x1874663043338368/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829529 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829504.225432:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829504.225435:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829504.225439:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043338368:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829504.225445:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043338496:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829504.225461:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829504.225469:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829504/real 1787829504] req@ffff9181c55a6680 x1874663043338496/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829529 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829504.225477:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6680 x1874663043338496/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829529 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829504.225484:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829504.225487:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829504.225490:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043338496:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829504.608912:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829505.632858:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829506.656903:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829507.680834:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829508.704861:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829509.664836:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829509.664844:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829509.664847:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829509.664855:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11575 00000100:00080000:0.0:1787829509.664863:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829509.664869:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829509.664897:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6d80 x1874663043338624/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829509.664916:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829509.664919:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829509.664920:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829509.664923:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11575 00000100:00080000:0.0:1787829509.664925:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829509.664928:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829509.664934:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829509.664941:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5880 x1874663043338752/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829509.664947:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829509.664949:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829509.664951:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829509.664953:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11575 00000100:00080000:0.0:1787829509.664955:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829509.664957:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829509.664961:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829509.664966:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7100 x1874663043338880/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829509.664973:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043338624:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829509.665055:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043338752:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829509.665097:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043338880:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829509.728860:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829510.752872:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829511.776886:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1787829512.737084:0:2376:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:0.0:1787829512.737139:0:2376:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1787829512.737160:0:2376:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1787829512.737210:0:2376:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1787829512.737220:0:2376:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:2.0:1787829512.737282:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829509/real 1787829512] req@ffff9181c55a7100 x1874663043338880/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829534 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829512.737326:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7100 x1874663043338880/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829534 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829512.737338:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829512.737343:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829512.737348:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043338880:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829512.737356:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829509/real 1787829512] req@ffff9181c55a6d80 x1874663043338624/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829534 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829512.737370:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6d80 x1874663043338624/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829534 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829512.737378:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829512.737382:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829512.737410:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043338624:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829512.737418:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829509/real 1787829512] req@ffff9181c55a5880 x1874663043338752/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829534 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829512.737431:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5880 x1874663043338752/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829534 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829512.737439:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829512.737443:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829512.737446:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043338752:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829512.800934:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829513.824965:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829514.784860:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829514.784871:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829514.784875:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829514.784885:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11581 00000100:00080000:0.0:1787829514.784896:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829514.784912:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829514.784943:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5f80 x1874663043339008/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829514.784969:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829514.784973:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829514.784975:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829514.784979:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11581 00000100:00080000:0.0:1787829514.784982:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829514.784987:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829514.784995:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829514.785000:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043339008:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787829514.785017:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4380 x1874663043339136/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829514.785034:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829514.785038:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829514.785040:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829514.785044:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11581 00000100:00080000:0.0:1787829514.785056:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829514.785060:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:1.0:1787829514.785066:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:0.0:1787829514.785068:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829514.785081:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4700 x1874663043339264/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787829514.785112:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829514/real 1787829514] req@ffff9181c55a5f80 x1874663043339008/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829539 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829514.785129:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5f80 x1874663043339008/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829539 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829514.785138:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829514.785142:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829514.785166:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043339008:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787829514.785188:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043339136:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787829514.785209:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787829514.785221:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829514/real 1787829514] req@ffff9181c55a4380 x1874663043339136/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829539 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829514.785231:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4380 x1874663043339136/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829539 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829514.785239:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829514.785242:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829514.785250:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043339136:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787829514.785256:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043339264:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787829514.785274:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787829514.785284:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829514/real 1787829514] req@ffff9181c55a4700 x1874663043339264/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829539 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829514.785294:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4700 x1874663043339264/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829539 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829514.785302:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829514.785305:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829514.785309:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043339264:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829514.848884:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829515.872884:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829516.896881:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829517.920893:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829518.944881:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829519.904850:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829519.904857:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829519.904860:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829519.904867:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11586 00000100:00080000:2.0:1787829519.904875:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829519.904880:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787829519.904910:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e300 x1874663043339392/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829519.904936:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829519.904939:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829519.904940:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829519.904943:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11586 00000100:00080000:2.0:1787829519.904946:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829519.904949:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829519.904956:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829519.904966:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d180 x1874663043339520/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787829519.904974:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043339392:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787829519.904975:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829519.904981:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829519.904984:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829519.904987:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11586 00000100:00080000:2.0:1787829519.904989:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829519.904992:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829519.904994:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829519.905002:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d500 x1874663043339648/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787829519.905042:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787829519.905075:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829519/real 1787829519] req@ffff9181e007e300 x1874663043339392/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829544 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829519.905089:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e300 x1874663043339392/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829544 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829519.905096:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829519.905098:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829519.905108:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043339392:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787829519.905127:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043339520:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787829519.905145:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787829519.905153:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829519/real 1787829519] req@ffff9181e007d180 x1874663043339520/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829544 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829519.905160:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d180 x1874663043339520/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829544 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829519.905166:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829519.905168:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829519.905171:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043339520:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787829519.905177:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043339648:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787829519.905195:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787829519.905205:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829519/real 1787829519] req@ffff9181e007d500 x1874663043339648/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829544 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829519.905216:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d500 x1874663043339648/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829544 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829519.905222:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829519.905224:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829519.905226:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043339648:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829519.968883:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829520.992864:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829522.016876:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829523.040854:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829524.064857:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:2.0:1787829524.512865:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-cliir' failed (-108). 10000000:00000400:2.0:1787829524.512872:0:159735:0:(mgc_request.c:1900:mgc_process_log()) MGC192.168.203.147@tcp: IR log lustre-cliir failed, not fatal: rc = -108 10000000:01000000:2.0:1787829524.516309:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:2.0:1787829524.516313:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff9181c5620000 from 52 10000000:01000000:2.0:1787829524.516317:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:2.0:1787829524.516360:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1842600/0xbef42c0221d6f8e1 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829524.516368:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181d1842600/0xbef42c0221d6f8e1 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829524.516376:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181d1842600/0xbef42c0221d6f8e1 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787829524.516390:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5c94a80 going to sleep for 0 seconds 00000100:00100000:2.0:1787829524.516399:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007df80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043339776:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:2.0:1787829524.516410:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181d1842600/0xbef42c0221d6f8e1 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829524.516416:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181d1842600/0xbef42c0221d6f8e1 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829524.516427:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1842600/0xbef42c0221d6f8e1 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829524.516431:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181d1842600/0xbef42c0221d6f8e1 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829524.516436:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181d1842600/0xbef42c0221d6f8e1 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829524.516439:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181d1842600 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:2.0:1787829524.516441:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181d1842600/0xbef42c0221d6f8e1 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829524.516446:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181d1842600/0xbef42c0221d6f8e1 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1787829524.516450:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181d1842600/0xbef42c0221d6f8e1 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787829524.516453:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:2.0:1787829524.516455:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787829524.516459:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181d1842600/0xbef42c0221d6f8e1 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829524.516466:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181d1842600/0xbef42c0221d6f8e1 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787829524.516480:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:2.0:1787829524.516482:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:1.0:1787829524.516543:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829524.516550:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829524.516554:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829524.516563:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11591 00000100:00080000:1.0:1787829524.516573:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829524.516581:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787829524.516611:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2a00 x1874663043339904/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829524.516638:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829524.516641:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829524.516642:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829524.516645:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11591 00000100:00080000:1.0:1787829524.516648:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829524.516652:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829524.516660:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829524.516662:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043339904:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787829524.516673:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3480 x1874663043340032/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829524.516695:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829524.516699:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829524.516700:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829524.516704:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11591 00000100:00080000:1.0:1787829524.516707:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829524.516711:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829524.516718:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829524.516728:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2680 x1874663043340160/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829524.516734:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829524.516832:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829524/real 1787829524] req@ffff9181c35b2a00 x1874663043339904/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829549 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829524.516849:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2a00 x1874663043339904/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829549 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829524.516856:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829524.516859:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829524.516868:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043339904:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829524.516883:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043340032:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829524.516906:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829524.516915:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829524/real 1787829524] req@ffff9181c35b3480 x1874663043340032/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829549 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829524.516922:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3480 x1874663043340032/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829549 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829524.516927:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829524.516929:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829524.516931:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043340032:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829524.516935:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043340160:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829524.516945:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829524.516950:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829524/real 1787829524] req@ffff9181c35b2680 x1874663043340160/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829549 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829524.516957:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2680 x1874663043340160/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829549 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829524.516961:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829524.516962:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829524.516964:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043340160:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829525.088900:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829526.112898:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829527.136894:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829528.160886:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829529.184835:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829529.632856:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829529.632863:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829529.632866:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829529.632873:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11596 00000100:00080000:3.0:1787829529.632881:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829529.632887:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829529.632915:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043340288/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829529.632934:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829529.632936:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829529.632938:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829529.632940:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11596 00000100:00080000:3.0:1787829529.632942:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829529.632945:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829529.632951:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829529.632958:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043340416/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829529.632965:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829529.632967:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829529.632969:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829529.632971:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11596 00000100:00080000:3.0:1787829529.632972:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829529.632975:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829529.632979:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829529.632986:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043340288:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829529.632986:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635f80 x1874663043340544/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829529.633046:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829529.633081:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829529/real 1787829529] req@ffff9181c5637100 x1874663043340288/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829554 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829529.633098:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043340288/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829554 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829529.633107:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829529.633111:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829529.633125:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043340288:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829529.633133:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043340416:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829529.633158:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829529.633172:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829529/real 1787829529] req@ffff9181c5637480 x1874663043340416/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829554 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829529.633185:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043340416/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829554 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829529.633192:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829529.633195:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829529.633198:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043340416:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829529.633221:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043340544:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829529.633243:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829529.633254:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829529/real 1787829529] req@ffff9181c5635f80 x1874663043340544/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829554 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829529.633265:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635f80 x1874663043340544/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829554 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829529.633272:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829529.633275:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829529.633278:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043340544:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829530.208880:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829531.232927:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829532.256882:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829533.280861:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829534.304913:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829534.752820:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829534.752827:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829534.752831:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829534.752838:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11601 00000100:00080000:3.0:1787829534.752846:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829534.752851:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829534.752879:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634e00 x1874663043340672/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829534.752899:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829534.752901:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829534.752903:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829534.752905:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11601 00000100:00080000:3.0:1787829534.752907:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829534.752910:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829534.752915:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829534.752925:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635f80 x1874663043340800/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829534.752932:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829534.752934:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829534.752935:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1787829534.752942:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043340672:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787829534.752942:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11601 00000100:00080000:3.0:1787829534.752946:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829534.752951:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829534.752954:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829534.752966:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043340928/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829534.752998:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829534.753025:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829534/real 1787829534] req@ffff9181c5634e00 x1874663043340672/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829559 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829534.753035:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634e00 x1874663043340672/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829559 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829534.753042:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829534.753044:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829534.753057:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043340672:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829534.753079:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043340800:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829534.753104:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829534.753121:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829534/real 1787829534] req@ffff9181c5635f80 x1874663043340800/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829559 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829534.753130:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635f80 x1874663043340800/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829559 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829534.753135:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829534.753137:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829534.753139:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043340800:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829534.753142:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043340928:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829534.753157:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829534.753164:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829534/real 1787829534] req@ffff9181c5637480 x1874663043340928/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829559 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829534.753170:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043340928/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829559 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829534.753174:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829534.753176:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829534.753178:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043340928:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829535.328874:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829536.352860:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829537.376886:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829538.400866:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829539.424874:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829539.872852:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829539.872859:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829539.872862:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829539.872870:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11606 00000100:00080000:0.0:1787829539.872879:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829539.872884:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829539.872912:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5c00 x1874663043341056/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829539.872929:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829539.872932:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829539.872933:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829539.872936:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11606 00000100:00080000:0.0:1787829539.872937:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829539.872940:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829539.872947:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829539.872955:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5180 x1874663043341184/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829539.872962:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829539.872964:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829539.872966:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829539.872968:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11606 00000100:00080000:0.0:1787829539.872970:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829539.872972:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829539.872975:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829539.872979:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5500 x1874663043341312/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829539.872988:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043341056:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787829539.873049:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829539.873082:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829539/real 1787829539] req@ffff9181c55a5c00 x1874663043341056/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829564 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829539.873098:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5c00 x1874663043341056/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829564 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829539.873108:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829539.873112:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829539.873127:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043341056:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829539.873134:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043341184:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829539.873160:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829539.873177:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829539/real 1787829539] req@ffff9181c55a5180 x1874663043341184/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829564 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829539.873188:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5180 x1874663043341184/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829564 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829539.873196:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829539.873199:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829539.873202:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043341184:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829539.873223:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043341312:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829539.873242:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829539.873254:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829539/real 1787829539] req@ffff9181c55a5500 x1874663043341312/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829564 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829539.873265:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5500 x1874663043341312/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829564 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829539.873272:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829539.873275:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829539.873278:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043341312:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829540.448858:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829541.472894:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829542.496869:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829543.520953:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829544.544878:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829544.992892:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 10000000:01000000:2.0:1787829544.992899:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-client' failed (-108). 00000100:00080000:0.0:1787829544.992901:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 10000000:00020000:2.0:1787829544.992907:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00080000:0.0:1787829544.992908:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829544.992919:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11611 00000100:00080000:0.0:1787829544.992928:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829544.992940:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829544.992971:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4e00 x1874663043341440/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829544.992999:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829544.993003:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829544.993006:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829544.993012:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11611 00000100:00080000:0.0:1787829544.993016:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829544.993020:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829544.993030:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829544.993046:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043341440:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787829544.993047:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7800 x1874663043341568/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829544.993065:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829544.993069:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829544.993071:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829544.993076:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11611 00000100:00080000:0.0:1787829544.993081:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829544.993085:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829544.993092:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829544.993104:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5500 x1874663043341696/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787829544.993115:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787829544.993151:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829544/real 1787829544] req@ffff9181c55a4e00 x1874663043341440/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829569 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829544.993164:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4e00 x1874663043341440/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829569 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829544.993172:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829544.993175:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829544.993178:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043341440:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787829544.993195:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043341568:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787829544.993213:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787829544.993221:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829544/real 1787829544] req@ffff9181c55a7800 x1874663043341568/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829569 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829544.993229:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7800 x1874663043341568/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829569 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829544.993235:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829544.993237:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829544.993239:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043341568:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787829544.993243:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043341696:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787829544.993253:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787829544.993259:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829544/real 1787829544] req@ffff9181c55a5500 x1874663043341696/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829569 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829544.993267:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5500 x1874663043341696/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829569 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829544.993272:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829544.993274:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829544.993275:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043341696:192.168.203.147@tcp:38:kworker.0 10000000:01000000:2.0:1787829544.997214:0:159735:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:2.0:1787829544.997217:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff9181c5620000 from 7 10000000:01000000:2.0:1787829544.997220:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00010000:00010000:2.0:1787829544.997246:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1842600/0xbef42c0221d6f8e8 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829544.997255:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181d1842600/0xbef42c0221d6f8e8 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829544.997264:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181d1842600/0xbef42c0221d6f8e8 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787829544.997275:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5c94a80 going to sleep for 0 seconds 00000100:00100000:2.0:1787829544.997280:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007df80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043341824:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:2.0:1787829544.997288:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181d1842600/0xbef42c0221d6f8e8 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829544.997293:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181d1842600/0xbef42c0221d6f8e8 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829544.997298:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1842600/0xbef42c0221d6f8e8 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829544.997306:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181d1842600/0xbef42c0221d6f8e8 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829544.997312:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181d1842600/0xbef42c0221d6f8e8 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829544.997316:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181d1842600 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:2.0:1787829544.997319:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181d1842600/0xbef42c0221d6f8e8 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829544.997325:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181d1842600/0xbef42c0221d6f8e8 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1787829544.997329:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181d1842600/0xbef42c0221d6f8e8 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787829544.997333:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:2.0:1787829544.997335:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787829544.997339:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181d1842600/0xbef42c0221d6f8e8 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829544.997344:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181d1842600/0xbef42c0221d6f8e8 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787829544.997355:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:2.0:1787829544.997357:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:1.0:1787829544.997430:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829544.997434:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829544.997436:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829544.997441:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11616 00000100:00080000:1.0:1787829544.997446:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829544.997449:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787829544.997462:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0e00 x1874663043341952/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829544.997505:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043341952:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787829544.997531:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829544.997561:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829544/real 1787829544] req@ffff9181c35b0e00 x1874663043341952/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829569 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829544.997573:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0e00 x1874663043341952/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829569 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829544.997579:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829544.997582:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829544.997591:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043341952:192.168.203.147@tcp:250:kworker.0 00010000:00010000:2.0:1787829545.568916:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829546.592912:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829547.616909:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829548.640859:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829549.664874:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829550.112841:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829550.112849:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829550.112853:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829550.112861:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11616 00000100:00080000:1.0:1787829550.112871:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829550.112878:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787829550.112911:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0000 x1874663043342080/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829550.112930:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829550.112933:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829550.112935:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829550.112938:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11616 00000100:00080000:1.0:1787829550.112940:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829550.112943:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829550.112951:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829550.112964:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1180 x1874663043342208/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829550.112975:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043342080:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787829550.112979:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829550.112986:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829550.112988:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829550.112993:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11616 00000100:00080000:1.0:1787829550.112997:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829550.113001:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829550.113006:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829550.113017:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3100 x1874663043342336/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829550.113048:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829550.113079:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829550/real 1787829550] req@ffff9181c35b0000 x1874663043342080/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829575 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829550.113093:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0000 x1874663043342080/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829575 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829550.113100:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829550.113104:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829550.113117:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043342080:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829550.113136:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043342208:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829550.113158:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829550.113168:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829550/real 1787829550] req@ffff9181c35b1180 x1874663043342208/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829575 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829550.113176:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1180 x1874663043342208/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829575 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829550.113181:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829550.113184:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829550.113186:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043342208:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829550.113191:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043342336:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829550.113210:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829550.113221:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829550/real 1787829550] req@ffff9181c35b3100 x1874663043342336/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829575 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829550.113234:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3100 x1874663043342336/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829575 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829550.113243:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829550.113247:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829550.113249:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043342336:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829550.688894:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829551.712923:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829552.736915:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829553.760900:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829554.784919:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829555.232851:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829555.232859:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829555.232863:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829555.232871:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11621 00000100:00080000:0.0:1787829555.232880:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829555.232886:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829555.232916:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5180 x1874663043342464/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829555.232943:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829555.232945:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829555.232947:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829555.232950:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11621 00000100:00080000:0.0:1787829555.232952:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829555.232954:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829555.232960:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829555.232968:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5c00 x1874663043342592/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829555.232975:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829555.232978:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829555.232979:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829555.232982:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11621 00000100:00080000:0.0:1787829555.232984:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829555.232987:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829555.232990:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829555.232993:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043342464:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787829555.232999:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4700 x1874663043342720/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829555.233051:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829555.233085:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829555/real 1787829555] req@ffff9181c55a5180 x1874663043342464/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829580 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829555.233098:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5180 x1874663043342464/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829580 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829555.233106:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829555.233109:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829555.233118:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043342464:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829555.233124:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043342592:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829555.233143:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829555.233156:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829555/real 1787829555] req@ffff9181c55a5c00 x1874663043342592/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829580 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829555.233166:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5c00 x1874663043342592/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829580 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829555.233172:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829555.233175:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829555.233177:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043342592:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829555.233194:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043342720:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829555.233205:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829555.233212:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829555/real 1787829555] req@ffff9181c55a4700 x1874663043342720/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829580 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829555.233219:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4700 x1874663043342720/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829580 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829555.233224:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829555.233226:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829555.233228:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043342720:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829555.808889:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829556.832888:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829557.856873:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829558.880876:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829559.904874:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829560.352824:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829560.352833:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829560.352836:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829560.352844:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11626 00000100:00080000:1.0:1787829560.352853:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829560.352859:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787829560.352896:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1500 x1874663043342848/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829560.352916:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829560.352918:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829560.352920:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829560.352922:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11626 00000100:00080000:1.0:1787829560.352925:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829560.352927:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829560.352934:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829560.352942:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0380 x1874663043342976/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829560.352950:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829560.352953:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829560.352954:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829560.352957:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11626 00000100:00080000:1.0:1787829560.352959:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829560.352961:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829560.352964:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829560.352973:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1c00 x1874663043343104/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829560.352976:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043342848:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787829560.353044:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829560.353078:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829560/real 1787829560] req@ffff9181c35b1500 x1874663043342848/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829585 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829560.353097:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1500 x1874663043342848/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829585 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829560.353107:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829560.353112:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829560.353127:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043342848:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829560.353136:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043342976:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829560.353163:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829560.353178:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829560/real 1787829560] req@ffff9181c35b0380 x1874663043342976/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829585 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829560.353191:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0380 x1874663043342976/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829585 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829560.353201:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829560.353204:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829560.353208:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043342976:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829560.353233:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043343104:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829560.353256:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829560.353268:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829560/real 1787829560] req@ffff9181c35b1c00 x1874663043343104/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829585 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829560.353281:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1c00 x1874663043343104/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829585 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829560.353288:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829560.353292:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829560.353295:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043343104:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829560.928900:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829561.952860:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829562.976885:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829564.000869:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829565.024867:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829565.472860:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829565.472869:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829565.472872:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829565.472881:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11632 00000100:00080000:3.0:1787829565.472891:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 10000000:01000000:1.0:1787829565.472893:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'params' failed (-108). 00000100:00080000:3.0:1787829565.472899:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 10000000:00020000:1.0:1787829565.472901:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:1.0:1787829565.472908:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:1.0:1787829565.472912:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff9181c87b19b8 from 1 10000000:01000000:1.0:1787829565.472916:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00000100:00100000:3.0:1787829565.472929:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043343232/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829565.472952:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:1.0:1787829565.472954:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6f8ef lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787829565.472955:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829565.472959:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829565.472967:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11632 00000100:00080000:3.0:1787829565.472973:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00010000:00010000:1.0:1787829565.472975:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6f8ef lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787829565.472979:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829565.472989:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:1.0:1787829565.472996:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6f8ef lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787829565.473003:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043343232:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829565.473004:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043343488/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787829565.473024:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215db40 going to sleep for 0 seconds 00000100:00080000:3.0:1787829565.473027:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829565.473031:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829565.473033:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1787829565.473037:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043343360:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00080000:3.0:1787829565.473041:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11632 00010000:00010000:1.0:1787829565.473056:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6f8ef lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787829565.473062:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00010000:00010000:1.0:1787829565.473070:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6f8ef lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787829565.473071:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:2.0:1787829565.473078:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:3.0:1787829565.473083:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:1.0:1787829565.473088:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6f8ef lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787829565.473100:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043343616/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787829565.473101:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6f8ef lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787829565.473118:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829565/real 1787829565] req@ffff9181c5637100 x1874663043343232/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829590 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:1.0:1787829565.473125:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6f8ef lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787829565.473138:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043343232/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829590 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:1.0:1787829565.473148:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b5400 already unused, calling callback (ffffffffc1273c20) 00000100:00080000:2.0:1787829565.473151:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 10000000:00010000:1.0:1787829565.473155:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6f8ef lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787829565.473156:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829565.473163:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043343232:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787829565.473179:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6f8ef lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787829565.473186:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043343488:192.168.203.147@tcp:38:kworker.0 10000000:00010000:1.0:1787829565.473195:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6f8ef lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787829565.473202:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1787829565.473207:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00000800:00000100:2.0:1787829565.473210:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00010000:00010000:1.0:1787829565.473218:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6f8ef lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787829565.473223:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829565/real 1787829565] req@ffff9181c5635500 x1874663043343488/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829590 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829565.473240:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043343488/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829590 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:1.0:1787829565.473242:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6f8ef lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787829565.473248:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829565.473255:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829565.473258:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043343488:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829565.473264:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043343616:192.168.203.147@tcp:38:kworker.0 10000000:01000000:1.0:1787829565.473272:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787829565.473276:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000800:00000100:2.0:1787829565.473278:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829565.473289:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829565/real 1787829565] req@ffff9181c5634380 x1874663043343616/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829590 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829565.473302:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043343616/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829590 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829565.473311:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829565.473314:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829565.473318:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043343616:192.168.203.147@tcp:38:kworker.0 00000100:00080000:3.0:1787829565.473332:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829565.473337:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829565.473339:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829565.473343:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11637 00000100:00080000:3.0:1787829565.473347:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829565.473352:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829565.473366:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043343744/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829565.473422:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043343744:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787829565.473442:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829565.473450:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829565/real 1787829565] req@ffff9181c5636d80 x1874663043343744/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829590 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829565.473458:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043343744/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829590 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829565.473468:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829565.473470:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829565.473481:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043343744:192.168.203.147@tcp:250:kworker.0 00010000:00010000:2.0:1787829566.048879:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829567.072861:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829568.096883:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829569.120882:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829570.144875:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829570.592862:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829570.592871:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829570.592875:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829570.592883:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11637 00000100:00080000:3.0:1787829570.592893:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829570.592899:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829570.592931:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043343872/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829570.592954:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829570.592956:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829570.592958:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829570.592961:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11637 00000100:00080000:3.0:1787829570.592963:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829570.592965:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829570.592973:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829570.592981:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043344000/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829570.592995:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043343872:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787829570.593000:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829570.593003:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829570.593005:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829570.593008:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11637 00000100:00080000:3.0:1787829570.593010:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829570.593013:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829570.593016:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829570.593026:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043344128/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829570.593059:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829570.593091:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829570/real 1787829570] req@ffff9181c5637800 x1874663043343872/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829595 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829570.593104:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043343872/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829595 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829570.593112:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829570.593115:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829570.593125:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043343872:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829570.593144:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043344000:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829570.593161:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829570.593172:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829570/real 1787829570] req@ffff9181c5636a00 x1874663043344000/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829595 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829570.593181:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043344000/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829595 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829570.593192:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829570.593196:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829570.593201:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043344000:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829570.593208:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043344128:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829570.593234:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829570.593247:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829570/real 1787829570] req@ffff9181c5635c00 x1874663043344128/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829595 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829570.593260:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043344128/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829595 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829570.593266:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829570.593268:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829570.593271:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043344128:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787829571.168949:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829572.192944:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829573.216919:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829574.240985:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829575.264895:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829575.712884:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829575.712891:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829575.712895:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829575.712903:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11642 00000100:00080000:0.0:1787829575.712913:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829575.712919:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829575.712951:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4380 x1874663043344256/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829575.712973:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829575.712976:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829575.712978:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829575.712980:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11642 00000100:00080000:0.0:1787829575.712983:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829575.712985:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829575.712992:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829575.713001:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5f80 x1874663043344384/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829575.713009:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829575.713012:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829575.713013:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829575.713015:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11642 00000100:00080000:0.0:1787829575.713017:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1787829575.713022:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043344256:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787829575.713024:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829575.713032:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829575.713041:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5880 x1874663043344512/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829575.713103:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043344384:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829575.713133:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043344512:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829576.288896:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829577.312907:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829578.336925:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1787829578.784981:0:2377:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:0.0:1787829578.785018:0:2377:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1787829578.785032:0:2377:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1787829578.785071:0:2377:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1787829578.785080:0:2377:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:2.0:1787829578.785123:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829575/real 1787829578] req@ffff9181c55a5880 x1874663043344512/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829600 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829578.785158:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5880 x1874663043344512/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829600 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829578.785171:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829578.785177:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829578.785182:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043344512:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829578.785190:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829575/real 1787829578] req@ffff9181c55a4380 x1874663043344256/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829600 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829578.785203:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4380 x1874663043344256/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829600 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829578.785210:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829578.785214:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829578.785234:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043344256:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829578.785242:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829575/real 1787829578] req@ffff9181c55a5f80 x1874663043344384/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829600 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829578.785256:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5f80 x1874663043344384/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829600 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829578.785265:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829578.785269:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829578.785273:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043344384:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829579.360885:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829580.384935:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829580.832892:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829580.832899:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829580.832902:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829580.832910:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11647 00000100:00080000:0.0:1787829580.832926:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829580.832932:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829580.832960:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6d80 x1874663043344640/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829580.832981:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829580.832984:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829580.832985:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829580.832988:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11647 00000100:00080000:0.0:1787829580.832990:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829580.832993:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829580.833001:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829580.833015:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7100 x1874663043344768/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829580.833027:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829580.833031:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829580.833033:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1787829580.833038:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043344640:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787829580.833041:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11647 00000100:00080000:0.0:1787829580.833050:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829580.833055:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829580.833062:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829580.833082:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6680 x1874663043344896/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829580.833117:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829580.833155:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829580/real 1787829580] req@ffff9181c55a6d80 x1874663043344640/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829605 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829580.833175:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6d80 x1874663043344640/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829605 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829580.833186:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829580.833190:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829580.833207:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043344640:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829580.833234:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043344768:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829580.833265:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829580.833282:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829580/real 1787829580] req@ffff9181c55a7100 x1874663043344768/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829605 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829580.833314:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7100 x1874663043344768/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829605 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829580.833323:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829580.833326:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829580.833329:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043344768:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829580.833336:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043344896:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829580.833360:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829580.833372:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829580/real 1787829580] req@ffff9181c55a6680 x1874663043344896/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829605 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829580.833390:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6680 x1874663043344896/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829605 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829580.833398:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829580.833400:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829580.833404:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043344896:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829581.408866:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829582.432927:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829583.456918:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829584.480887:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829585.504858:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829585.952844:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 10000000:01000000:1.0:1787829585.952845:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-sptlrpc' failed (-108). 10000000:00020000:1.0:1787829585.952854:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00080000:3.0:1787829585.952857:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829585.952863:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829585.952873:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11652 00000100:00080000:3.0:1787829585.952884:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829585.952891:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829585.952925:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637b80 x1874663043345024/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829585.952952:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829585.952956:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829585.952958:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829585.952962:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11652 00000100:00080000:3.0:1787829585.952973:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829585.952978:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829585.952986:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829585.952999:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043345152/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829585.953007:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043345024:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787829585.953013:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829585.953017:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829585.953019:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829585.953024:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11652 00000100:00080000:3.0:1787829585.953028:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829585.953032:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829585.953037:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829585.953048:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634a80 x1874663043345280/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829585.953080:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829585.953117:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829585/real 1787829585] req@ffff9181c5637b80 x1874663043345024/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829610 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829585.953136:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637b80 x1874663043345024/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829610 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829585.953147:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829585.953152:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829585.953156:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043345024:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829585.953179:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043345152:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829585.953205:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829585.953219:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829585/real 1787829585] req@ffff9181c5634000 x1874663043345152/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829610 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829585.953233:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043345152/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829610 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829585.953243:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829585.953247:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829585.953264:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043345152:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829585.953272:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043345280:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829585.953295:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829585.953308:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829585/real 1787829585] req@ffff9181c5634a80 x1874663043345280/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829610 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829585.953321:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634a80 x1874663043345280/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829610 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829585.953331:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829585.953343:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829585.953347:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043345280:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829586.528871:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787829587.296880:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:1.0:1787829587.296887:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff9181c5620000 from 341 10000000:01000000:1.0:1787829587.296891:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:1.0:1787829587.296938:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6f8f6 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829587.296951:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6f8f6 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829587.296968:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6f8f6 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787829587.296986:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215d180 going to sleep for 0 seconds 00000100:00100000:1.0:1787829587.296996:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043345408:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787829587.297009:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6f8f6 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829587.297015:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6f8f6 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829587.297020:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6f8f6 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829587.297025:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6f8f6 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829587.297030:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6f8f6 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829587.297034:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b5400 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787829587.297036:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6f8f6 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829587.297041:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6f8f6 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787829587.297047:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6f8f6 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787829587.297051:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:1.0:1787829587.297053:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787829587.297057:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6f8f6 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829587.297061:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b5400/0xbef42c0221d6f8f6 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787829587.297078:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787829587.297080:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:3.0:1787829587.297144:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829587.297151:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829587.297155:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829587.297161:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11657 00000100:00080000:3.0:1787829587.297171:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829587.297177:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829587.297201:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634a80 x1874663043345536/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829587.297227:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829587.297231:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829587.297233:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829587.297238:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11657 00000100:00080000:3.0:1787829587.297257:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829587.297263:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829587.297273:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829587.297275:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043345536:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829587.297292:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043345664/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829587.297306:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829587.297310:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829587.297312:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829587.297317:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11657 00000100:00080000:3.0:1787829587.297330:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829587.297336:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:2.0:1787829587.297341:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:3.0:1787829587.297345:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829587.297363:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637b80 x1874663043345792/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829587.297384:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829587/real 1787829587] req@ffff9181c5634a80 x1874663043345536/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829612 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829587.297403:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634a80 x1874663043345536/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829612 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829587.297414:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829587.297419:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829587.297433:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043345536:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829587.297455:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043345664:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829587.297480:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829587.297492:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829587/real 1787829587] req@ffff9181c5634000 x1874663043345664/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829612 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829587.297501:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043345664/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829612 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829587.297507:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829587.297509:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829587.297512:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043345664:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829587.297516:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043345792:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829587.297529:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829587.297535:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829587/real 1787829587] req@ffff9181c5637b80 x1874663043345792/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829612 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829587.297543:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637b80 x1874663043345792/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829612 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829587.297548:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829587.297550:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829587.297553:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043345792:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829587.552860:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829588.576929:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829589.600862:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829590.624901:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829591.648897:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829592.608835:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829592.608845:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829592.608848:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829592.608858:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11659 00000100:00080000:3.0:1787829592.608867:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829592.608874:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829592.608909:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043345920/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829592.608934:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829592.608937:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829592.608939:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829592.608944:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11659 00000100:00080000:3.0:1787829592.608948:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829592.608953:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829592.608962:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829592.608964:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043345920:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829592.608977:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043346048/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829592.608989:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829592.608993:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829592.609003:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829592.609010:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11659 00000100:00080000:3.0:1787829592.609014:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000800:00000100:2.0:1787829592.609017:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1787829592.609023:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829592.609029:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829592.609042:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043346176/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829592.609049:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829592/real 1787829592] req@ffff9181c5635c00 x1874663043345920/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829617 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829592.609066:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043345920/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829617 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829592.609073:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829592.609076:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829592.609088:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043345920:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829592.609109:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043346048:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829592.609125:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829592.609134:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829592/real 1787829592] req@ffff9181c5636a00 x1874663043346048/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829617 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829592.609142:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043346048/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829617 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829592.609147:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829592.609149:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829592.609151:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043346048:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829592.609154:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043346176:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829592.609163:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829592.609168:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829592/real 1787829592] req@ffff9181c5637800 x1874663043346176/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829617 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829592.609179:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043346176/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829617 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829592.609183:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829592.609185:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829592.609187:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043346176:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829592.672878:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829593.696867:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829594.720858:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829595.744871:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829596.768936:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829597.728922:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829597.728936:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829597.728941:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829597.728960:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11664 00000100:00080000:0.0:1787829597.728973:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829597.728983:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829597.729038:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7b80 x1874663043346304/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829597.729073:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829597.729077:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829597.729079:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829597.729085:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11664 00000100:00080000:0.0:1787829597.729090:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829597.729095:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829597.729107:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829597.729117:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043346304:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787829597.729127:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7480 x1874663043346432/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829597.729143:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829597.729145:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829597.729147:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829597.729151:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11664 00000100:00080000:0.0:1787829597.729155:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829597.729158:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829597.729162:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829597.729188:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6680 x1874663043346560/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829597.729419:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829597.729507:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829597/real 1787829597] req@ffff9181c55a7b80 x1874663043346304/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829622 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829597.729531:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7b80 x1874663043346304/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829622 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829597.729545:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829597.729551:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829597.729566:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043346304:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829597.729596:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043346432:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829597.729624:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829597.729640:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829597/real 1787829597] req@ffff9181c55a7480 x1874663043346432/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829622 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829597.729655:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7480 x1874663043346432/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829622 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829597.729665:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829597.729670:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829597.729674:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043346432:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829597.729681:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043346560:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829597.729704:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829597.729717:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829597/real 1787829597] req@ffff9181c55a6680 x1874663043346560/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829622 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829597.729731:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6680 x1874663043346560/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829622 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829597.729740:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829597.729772:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829597.729777:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043346560:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829597.793040:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829598.816878:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829599.840876:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829600.864872:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829601.888871:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829602.848869:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829602.848879:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829602.848884:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829602.848896:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11669 00000100:00080000:0.0:1787829602.848907:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829602.848915:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829602.848963:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7100 x1874663043346688/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829602.848989:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829602.848992:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829602.848994:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829602.848997:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11669 00000100:00080000:0.0:1787829602.848999:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829602.849002:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829602.849012:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829602.849021:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6d80 x1874663043346816/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829602.849030:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829602.849032:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829602.849034:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829602.849037:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11669 00000100:00080000:0.0:1787829602.849038:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829602.849041:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829602.849045:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829602.849056:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5f80 x1874663043346944/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829602.849056:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043346688:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1787829602.849206:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829602.849247:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829602/real 1787829602] req@ffff9181c55a7100 x1874663043346688/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829627 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829602.849263:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7100 x1874663043346688/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829627 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829602.849272:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829602.849276:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829602.849286:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043346688:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829602.849293:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043346816:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829602.849325:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829602.849335:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829602/real 1787829602] req@ffff9181c55a6d80 x1874663043346816/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829627 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829602.849344:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6d80 x1874663043346816/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829627 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829602.849350:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829602.849352:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829602.849355:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043346816:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829602.849376:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043346944:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829602.849389:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829602.849396:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829602/real 1787829602] req@ffff9181c55a5f80 x1874663043346944/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829627 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829602.849406:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5f80 x1874663043346944/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829627 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829602.849415:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829602.849418:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829602.849422:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043346944:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829602.912875:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829603.936901:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829604.960857:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829605.984896:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829607.008860:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787829607.456848:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-cliir' failed (-108). 10000000:00000400:1.0:1787829607.456855:0:159735:0:(mgc_request.c:1900:mgc_process_log()) MGC192.168.203.147@tcp: IR log lustre-cliir failed, not fatal: rc = -108 10000000:01000000:1.0:1787829607.456861:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:1.0:1787829607.456864:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff9181c5620000 from 52 10000000:01000000:1.0:1787829607.456866:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:1.0:1787829607.456912:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4600/0xbef42c0221d6f8fd lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829607.456926:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b4600/0xbef42c0221d6f8fd lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829607.456935:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b4600/0xbef42c0221d6f8fd lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787829607.456951:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215d180 going to sleep for 0 seconds 00000100:00100000:1.0:1787829607.456960:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043347072:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787829607.456970:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4600/0xbef42c0221d6f8fd lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829607.456975:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b4600/0xbef42c0221d6f8fd lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829607.456980:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4600/0xbef42c0221d6f8fd lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829607.456985:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b4600/0xbef42c0221d6f8fd lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829607.456989:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b4600/0xbef42c0221d6f8fd lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829607.456994:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4600 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787829607.456996:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4600/0xbef42c0221d6f8fd lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829607.457001:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b4600/0xbef42c0221d6f8fd lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787829607.457007:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4600/0xbef42c0221d6f8fd lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787829607.457011:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1787829607.457013:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787829607.457018:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b4600/0xbef42c0221d6f8fd lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829607.457022:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b4600/0xbef42c0221d6f8fd lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787829607.457037:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787829607.457039:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:0.0:1787829607.457117:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829607.457124:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829607.457127:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829607.457134:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11674 00000100:00080000:0.0:1787829607.457144:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829607.457150:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829607.457175:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4380 x1874663043347200/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829607.457199:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829607.457203:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829607.457204:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829607.457209:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11674 00000100:00080000:0.0:1787829607.457212:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829607.457216:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829607.457224:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829607.457231:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043347200:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787829607.457237:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5880 x1874663043347328/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829607.457246:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829607.457248:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829607.457250:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829607.457253:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11674 00000100:00080000:0.0:1787829607.457255:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829607.457258:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829607.457261:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829607.457269:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4700 x1874663043347456/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829607.457291:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829607.457318:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829607/real 1787829607] req@ffff9181c55a4380 x1874663043347200/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829632 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829607.457337:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4380 x1874663043347200/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829632 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829607.457344:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829607.457347:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829607.457363:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043347200:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829607.457379:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043347328:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829607.457395:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829607.457404:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829607/real 1787829607] req@ffff9181c55a5880 x1874663043347328/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829632 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829607.457412:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5880 x1874663043347328/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829632 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829607.457417:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829607.457419:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829607.457422:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043347328:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829607.457425:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043347456:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829607.457435:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829607.457441:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829607/real 1787829607] req@ffff9181c55a4700 x1874663043347456/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829632 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829607.457448:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4700 x1874663043347456/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829632 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829607.457458:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829607.457460:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829607.457462:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043347456:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829608.032909:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829609.056907:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829610.080907:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829611.104870:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829612.128849:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829612.576829:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829612.576838:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829612.576841:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829612.576849:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11679 00000100:00080000:0.0:1787829612.576858:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829612.576864:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829612.576896:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5c00 x1874663043347584/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829612.576921:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829612.576924:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829612.576926:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829612.576928:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11679 00000100:00080000:0.0:1787829612.576930:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829612.576933:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829612.576939:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829612.576948:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5180 x1874663043347712/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829612.576954:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829612.576957:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829612.576958:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829612.576961:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11679 00000100:00080000:0.0:1787829612.576964:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787829612.576966:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043347584:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787829612.576969:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829612.576977:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829612.576984:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5500 x1874663043347840/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829612.577028:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829612.577057:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829612/real 1787829612] req@ffff9181c55a5c00 x1874663043347584/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829637 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829612.577088:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5c00 x1874663043347584/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829637 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829612.577097:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829612.577100:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829612.577110:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043347584:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829612.577116:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043347712:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829612.577135:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829612.577145:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829612/real 1787829612] req@ffff9181c55a5180 x1874663043347712/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829637 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829612.577154:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5180 x1874663043347712/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829637 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829612.577161:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829612.577164:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829612.577166:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043347712:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829612.577184:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043347840:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829612.577195:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829612.577202:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829612/real 1787829612] req@ffff9181c55a5500 x1874663043347840/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829637 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829612.577209:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5500 x1874663043347840/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829637 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829612.577215:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829612.577216:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829612.577218:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043347840:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829613.152890:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829614.176883:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829615.200913:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829616.224892:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829617.248905:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829617.696895:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829617.696903:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829617.696906:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829617.696915:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11684 00000100:00080000:1.0:1787829617.696925:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829617.696930:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787829617.696963:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1c00 x1874663043347968/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829617.696986:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829617.696990:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829617.696992:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829617.696995:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11684 00000100:00080000:1.0:1787829617.696997:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829617.697000:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829617.697007:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829617.697016:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0380 x1874663043348096/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829617.697024:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829617.697028:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:3.0:1787829617.697040:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043347968:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787829617.697048:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829617.697053:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11684 00000100:00080000:1.0:1787829617.697055:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829617.697058:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829617.697063:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829617.697073:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1500 x1874663043348224/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829617.697109:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829617.697138:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829617/real 1787829617] req@ffff9181c35b1c00 x1874663043347968/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829642 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829617.697151:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1c00 x1874663043347968/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829642 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829617.697158:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829617.697161:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829617.697172:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043347968:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829617.697192:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043348096:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829617.697210:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829617.697219:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829617/real 1787829617] req@ffff9181c35b0380 x1874663043348096/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829642 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829617.697228:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0380 x1874663043348096/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829642 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829617.697234:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829617.697236:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829617.697238:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043348096:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829617.697242:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043348224:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829617.697252:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829617.697257:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829617/real 1787829617] req@ffff9181c35b1500 x1874663043348224/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829642 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829617.697265:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1500 x1874663043348224/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829642 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829617.697270:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829617.697272:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829617.697274:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043348224:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829618.272889:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829619.296893:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829620.320880:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829621.344895:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829622.368845:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829622.816815:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829622.816823:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829622.816826:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829622.816834:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11689 00000100:00080000:1.0:1787829622.816843:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829622.816849:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787829622.816884:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3100 x1874663043348352/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829622.816912:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829622.816915:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829622.816916:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829622.816920:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11689 00000100:00080000:1.0:1787829622.816923:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829622.816927:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829622.816935:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829622.816948:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043348352:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787829622.816949:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1180 x1874663043348480/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829622.816963:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829622.816965:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829622.816967:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829622.816970:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11689 00000100:00080000:1.0:1787829622.816973:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829622.816975:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829622.816978:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829622.816987:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0000 x1874663043348608/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829622.817023:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829622.817052:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829622/real 1787829622] req@ffff9181c35b3100 x1874663043348352/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829647 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829622.817065:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3100 x1874663043348352/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829647 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829622.817078:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829622.817081:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829622.817092:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043348352:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829622.817112:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043348480:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829622.817137:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829622.817146:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829622/real 1787829622] req@ffff9181c35b1180 x1874663043348480/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829647 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829622.817154:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1180 x1874663043348480/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829647 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829622.817160:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829622.817162:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829622.817164:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043348480:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829622.817168:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043348608:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829622.817179:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829622.817185:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829622/real 1787829622] req@ffff9181c35b0000 x1874663043348608/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829647 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829622.817192:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0000 x1874663043348608/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829647 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829622.817198:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829622.817200:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829622.817201:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043348608:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829623.392866:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829624.416884:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829625.440890:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829626.464917:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829627.488887:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829627.936901:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 10000000:01000000:1.0:1787829627.936905:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-client' failed (-108). 00000100:00080000:0.0:1787829627.936913:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 10000000:00020000:1.0:1787829627.936916:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00080000:0.0:1787829627.936918:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 10000000:01000000:1.0:1787829627.936926:0:159735:0:(mgc_request.c:611:do_requeue()) updating log params 00000100:00080000:0.0:1787829627.936929:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11694 10000000:01000000:1.0:1787829627.936932:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff9181c5620000 from 7 10000000:01000000:1.0:1787829627.936939:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00000100:00080000:0.0:1787829627.936943:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829627.936951:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829627.937011:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7800 x1874663043348736/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787829627.937026:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f904 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787829627.937031:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829627.937037:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829627.937039:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00010000:00010000:1.0:1787829627.937039:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f904 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787829627.937048:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11694 00000100:00080000:0.0:1787829627.937053:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00010000:00010000:1.0:1787829627.937055:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f904 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787829627.937059:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829627.937089:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829627.937090:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043348736:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787829627.937092:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215d180 going to sleep for 0 seconds 00000100:00100000:1.0:1787829627.937103:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043348864:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00100000:0.0:1787829627.937109:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4e00 x1874663043348992/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787829627.937118:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f904 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787829627.937121:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829627.937145:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829627.937148:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00010000:00010000:1.0:1787829627.937149:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f904 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787829627.937155:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11694 00010000:00010000:1.0:1787829627.937159:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f904 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787829627.937161:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829627.937169:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:1.0:1787829627.937169:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f904 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000800:00000100:3.0:1787829627.937171:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:0.0:1787829627.937179:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:1.0:1787829627.937181:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f904 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829627.937191:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b4400 already unused, calling callback (ffffffffc1273c20) 00000100:00100000:0.0:1787829627.937196:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5500 x1874663043349120/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 10000000:00010000:1.0:1787829627.937196:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f904 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787829627.937223:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829627/real 1787829627] req@ffff9181c55a7800 x1874663043348736/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829652 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:1.0:1787829627.937240:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f904 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787829627.937262:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f904 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787829627.937270:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00000100:00100000:3.0:1787829627.937270:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7800 x1874663043348736/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829652 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:1.0:1787829627.937278:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787829627.937286:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f904 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787829627.937289:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00010000:00010000:1.0:1787829627.937299:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b4400/0xbef42c0221d6f904 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787829627.937299:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829627.937307:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043348736:192.168.203.147@tcp:250:kworker.0 10000000:01000000:1.0:1787829627.937352:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00100000:3.0:1787829627.937353:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043348992:192.168.203.147@tcp:38:kworker.0 00000100:00080000:1.0:1787829627.937358:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000800:00000100:3.0:1787829627.937373:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829627.937387:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829627/real 1787829627] req@ffff9181c55a4e00 x1874663043348992/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829652 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829627.937394:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829627.937399:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829627.937401:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787829627.937405:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4e00 x1874663043348992/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829652 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829627.937408:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11699 00000100:00080000:0.0:1787829627.937414:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829627.937423:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00080000:3.0:1787829627.937425:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829627.937432:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829627.937438:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043348992:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787829627.937441:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5180 x1874663043349248/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829627.937449:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043349120:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829627.937481:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829627.937494:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829627/real 1787829627] req@ffff9181c55a5500 x1874663043349120/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829652 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829627.937508:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5500 x1874663043349120/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829652 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829627.937515:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829627.937518:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829627.937520:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043349120:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829627.937531:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043349248:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1787829627.937545:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829627.937552:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829627/real 1787829627] req@ffff9181c55a5180 x1874663043349248/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829652 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829627.937560:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5180 x1874663043349248/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829652 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829627.937565:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829627.937567:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829627.937578:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043349248:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787829628.512863:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829629.536900:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829630.560927:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829631.584840:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829632.608851:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829633.056882:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829633.056891:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829633.056894:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829633.056903:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11699 00000100:00080000:1.0:1787829633.056912:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829633.056918:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787829633.056975:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0e00 x1874663043349376/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829633.056996:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829633.056999:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829633.057001:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829633.057004:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11699 00000100:00080000:1.0:1787829633.057007:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829633.057010:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829633.057028:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829633.057037:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2680 x1874663043349504/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829633.057045:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829633.057047:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829633.057049:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829633.057052:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11699 00000100:00080000:1.0:1787829633.057054:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829633.057056:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829633.057060:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829633.057062:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043349376:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787829633.057069:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3480 x1874663043349632/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829633.057121:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829633.057151:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829633/real 1787829633] req@ffff9181c35b0e00 x1874663043349376/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829658 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829633.057165:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0e00 x1874663043349376/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829658 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829633.057177:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829633.057180:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829633.057193:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043349376:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829633.057198:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043349504:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829633.057216:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829633.057247:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829633/real 1787829633] req@ffff9181c35b2680 x1874663043349504/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829658 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829633.057262:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2680 x1874663043349504/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829658 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829633.057272:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829633.057276:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829633.057280:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043349504:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829633.057308:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043349632:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829633.057341:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829633.057352:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829633/real 1787829633] req@ffff9181c35b3480 x1874663043349632/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829658 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829633.057361:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3480 x1874663043349632/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829658 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829633.057368:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829633.057371:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829633.057374:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043349632:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829633.632861:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829634.656902:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829635.680843:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829636.704852:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829637.728875:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829638.176845:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829638.176857:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829638.176861:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829638.176873:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11704 00000100:00080000:1.0:1787829638.176883:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829638.176907:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787829638.176952:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2a00 x1874663043349760/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829638.176980:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829638.176984:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829638.176986:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829638.176995:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11704 00000100:00080000:1.0:1787829638.176999:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829638.177004:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829638.177013:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829638.177027:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2d80 x1874663043349888/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829638.177038:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787829638.177042:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043349760:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787829638.177043:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829638.177047:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829638.177054:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11704 00000100:00080000:1.0:1787829638.177059:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829638.177064:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829638.177070:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829638.177085:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1f80 x1874663043350016/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829638.177109:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829638.177139:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829638/real 1787829638] req@ffff9181c35b2a00 x1874663043349760/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829663 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829638.177152:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2a00 x1874663043349760/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829663 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829638.177160:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829638.177163:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829638.177175:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043349760:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829638.177196:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043349888:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829638.177213:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829638.177223:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829638/real 1787829638] req@ffff9181c35b2d80 x1874663043349888/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829663 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829638.177234:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2d80 x1874663043349888/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829663 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829638.177242:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829638.177246:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829638.177249:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043349888:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829638.177256:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043350016:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829638.177276:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829638.177286:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829638/real 1787829638] req@ffff9181c35b1f80 x1874663043350016/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829663 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829638.177295:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1f80 x1874663043350016/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829663 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829638.177300:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829638.177302:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829638.177304:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043350016:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829638.752934:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829639.776873:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829640.800936:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829641.824879:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829642.848889:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829643.296877:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829643.296885:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829643.296888:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829643.296896:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11709 00000100:00080000:2.0:1787829643.296906:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829643.296912:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787829643.296943:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007df80 x1874663043350144/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829643.296965:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829643.296967:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829643.296969:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829643.296972:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11709 00000100:00080000:2.0:1787829643.296974:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829643.296977:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829643.296984:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829643.296992:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ed80 x1874663043350272/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829643.297000:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829643.297003:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829643.297004:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829643.297008:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11709 00000100:00080000:2.0:1787829643.297012:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787829643.297018:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043350144:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787829643.297020:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829643.297030:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829643.297056:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d500 x1874663043350400/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829643.297115:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043350272:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829643.297153:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043350400:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829643.872884:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829644.896914:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829645.920849:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1787829646.368958:0:2378:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:2.0:1787829646.369000:0:2378:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1787829646.369018:0:2378:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:2.0:1787829646.369052:0:2378:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:2.0:1787829646.369063:0:2378:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:3.0:1787829646.369098:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829643/real 1787829646] req@ffff9181e007d500 x1874663043350400/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829668 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829646.369127:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d500 x1874663043350400/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829668 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829646.369140:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829646.369145:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829646.369150:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043350400:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829646.369159:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829643/real 1787829646] req@ffff9181e007df80 x1874663043350144/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829668 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829646.369174:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007df80 x1874663043350144/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829668 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829646.369190:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829646.369194:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829646.369219:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043350144:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829646.369227:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829643/real 1787829646] req@ffff9181e007ed80 x1874663043350272/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829668 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829646.369241:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ed80 x1874663043350272/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829668 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829646.369251:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829646.369255:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829646.369259:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043350272:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829646.944911:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829647.968865:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:3.0:1787829647.968871:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'params' failed (-108). 10000000:00020000:3.0:1787829647.968877:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:3.0:1787829647.968882:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:3.0:1787829647.968884:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff9181c87b19b8 from 1 10000000:01000000:3.0:1787829647.968887:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00010000:00010000:3.0:1787829647.968923:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f90b lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787829647.968932:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f90b lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787829647.968942:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f90b lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787829647.968958:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d225f0c0 going to sleep for 0 seconds 00000100:00100000:3.0:1787829647.968969:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043350528:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:3.0:1787829647.968982:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f90b lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787829647.968991:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f90b lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787829647.969000:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f90b lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787829647.969008:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f90b lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787829647.969015:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f90b lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787829647.969027:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3480000 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:3.0:1787829647.969030:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f90b lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787829647.969036:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f90b lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1787829647.969041:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f90b lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787829647.969045:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:3.0:1787829647.969047:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787829647.969052:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f90b lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787829647.969061:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f90b lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787829647.969077:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:3.0:1787829647.969080:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:2.0:1787829647.969118:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829647.969125:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829647.969129:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829647.969135:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11715 00000100:00080000:2.0:1787829647.969143:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829647.969148:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787829647.969166:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d180 x1874663043350656/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829647.969185:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829647.969188:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829647.969189:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829647.969192:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11715 00000100:00080000:2.0:1787829647.969195:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829647.969198:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829647.969206:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829647.969208:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043350656:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829647.969220:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e300 x1874663043350784/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829647.969232:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829647.969236:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829647.969238:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829647.969242:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11715 00000100:00080000:2.0:1787829647.969255:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829647.969260:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:3.0:1787829647.969265:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:2.0:1787829647.969266:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829647.969279:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c700 x1874663043350912/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829647.969295:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829647/real 1787829647] req@ffff9181e007d180 x1874663043350656/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829672 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829647.969309:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d180 x1874663043350656/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829672 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829647.969316:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829647.969319:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829647.969328:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043350656:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829647.969344:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043350784:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829647.969369:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829647.969382:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829647/real 1787829647] req@ffff9181e007e300 x1874663043350784/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829672 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829647.969397:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e300 x1874663043350784/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829672 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829647.969406:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829647.969408:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829647.969411:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043350784:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829647.969416:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043350912:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829647.969431:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829647.969439:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829647/real 1787829647] req@ffff9181e007c700 x1874663043350912/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829672 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829647.969446:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c700 x1874663043350912/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829672 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829647.969452:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829647.969454:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829647.969456:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043350912:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829648.992908:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829650.016896:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829651.040901:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829652.064912:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829653.024863:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829653.024871:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829653.024875:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829653.024883:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11719 00000100:00080000:3.0:1787829653.024892:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829653.024898:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829653.024929:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043351040/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829653.024951:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829653.024954:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829653.024955:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829653.024959:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11719 00000100:00080000:3.0:1787829653.024962:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829653.024966:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829653.024974:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829653.024984:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043351040:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829653.024985:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043351168/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829653.024998:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829653.025001:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829653.025002:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829653.025005:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11719 00000100:00080000:3.0:1787829653.025007:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829653.025010:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829653.025013:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829653.025021:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043351296/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829653.025048:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829653.025078:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829653/real 1787829653] req@ffff9181c5636d80 x1874663043351040/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829678 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829653.025091:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043351040/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829678 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829653.025099:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829653.025102:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829653.025113:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043351040:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829653.025134:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043351168:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829653.025151:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829653.025159:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829653/real 1787829653] req@ffff9181c5634380 x1874663043351168/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829678 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829653.025167:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043351168/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829678 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829653.025172:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829653.025174:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829653.025177:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043351168:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829653.025180:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043351296:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829653.025190:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829653.025195:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829653/real 1787829653] req@ffff9181c5635500 x1874663043351296/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829678 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829653.025202:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043351296/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829678 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829653.025207:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829653.025209:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829653.025211:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043351296:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829653.088944:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829654.112845:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829655.136881:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829656.160906:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829657.184972:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829658.144853:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829658.144862:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829658.144866:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829658.144874:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11724 00000100:00080000:1.0:1787829658.144884:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829658.144891:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787829658.144923:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0a80 x1874663043351424/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829658.144955:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829658.144958:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829658.144959:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829658.144963:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11724 00000100:00080000:1.0:1787829658.144968:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829658.144972:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829658.144980:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829658.145009:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043351424:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787829658.145011:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3800 x1874663043351552/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829658.145033:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829658.145036:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829658.145037:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829658.145042:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11724 00000100:00080000:1.0:1787829658.145044:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829658.145048:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829658.145067:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829658.145077:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3b80 x1874663043351680/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829658.145126:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829658.145170:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829658/real 1787829658] req@ffff9181c35b0a80 x1874663043351424/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829683 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829658.145191:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0a80 x1874663043351424/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829683 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829658.145202:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829658.145207:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829658.145226:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043351424:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829658.145259:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043351552:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829658.145284:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829658.145294:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829658/real 1787829658] req@ffff9181c35b3800 x1874663043351552/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829683 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829658.145303:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3800 x1874663043351552/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829683 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829658.145309:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829658.145311:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829658.145314:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043351552:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829658.145318:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043351680:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829658.145332:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829658.145343:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829658/real 1787829658] req@ffff9181c35b3b80 x1874663043351680/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829683 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829658.145355:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3b80 x1874663043351680/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829683 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829658.145371:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829658.145375:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829658.145379:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043351680:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829658.208922:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829659.232892:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829660.256908:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829661.280874:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829662.304884:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829663.264851:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829663.264859:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829663.264862:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829663.264871:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11729 00000100:00080000:1.0:1787829663.264880:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829663.264887:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787829663.264922:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0700 x1874663043351808/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829663.264951:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829663.264956:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829663.264958:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829663.264963:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11729 00000100:00080000:1.0:1787829663.264967:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829663.264972:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829663.264981:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829663.264999:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3b80 x1874663043351936/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829663.265004:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043351808:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787829663.265021:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829663.265025:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829663.265027:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829663.265032:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11729 00000100:00080000:1.0:1787829663.265037:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829663.265042:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829663.265048:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829663.265071:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3800 x1874663043352064/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829663.265073:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829663.265111:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829663/real 1787829663] req@ffff9181c35b0700 x1874663043351808/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829688 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829663.265124:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0700 x1874663043351808/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829688 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829663.265131:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829663.265134:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829663.265146:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043351808:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829663.265164:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043351936:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829663.265190:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829663.265206:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829663/real 1787829663] req@ffff9181c35b3b80 x1874663043351936/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829688 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829663.265220:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3b80 x1874663043351936/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829688 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829663.265227:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829663.265229:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829663.265231:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043351936:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829663.265236:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043352064:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829663.265251:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829663.265259:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829663/real 1787829663] req@ffff9181c35b3800 x1874663043352064/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829688 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829663.265272:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3800 x1874663043352064/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829688 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829663.265280:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829663.265283:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829663.265287:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043352064:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829663.328889:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829664.352856:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829665.376889:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829666.400907:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829667.424959:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829668.384854:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829668.384864:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829668.384867:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829668.384877:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11735 10000000:01000000:3.0:1787829668.384878:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-sptlrpc' failed (-108). 10000000:00020000:3.0:1787829668.384884:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00080000:0.0:1787829668.384889:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829668.384896:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829668.384924:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5c00 x1874663043352192/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829668.384946:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829668.384950:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829668.384952:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829668.384957:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11735 00000100:00080000:0.0:1787829668.384962:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829668.384967:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829668.384977:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829668.384986:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043352192:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787829668.384993:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4700 x1874663043352320/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829668.385014:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829668.385017:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829668.385020:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829668.385024:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11735 00000100:00080000:0.0:1787829668.385029:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829668.385034:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829668.385039:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829668.385053:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5880 x1874663043352448/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829668.385059:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829668.385100:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829668/real 1787829668] req@ffff9181c55a5c00 x1874663043352192/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829693 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829668.385119:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5c00 x1874663043352192/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829693 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829668.385130:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829668.385134:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829668.385139:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043352192:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829668.385159:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043352320:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829668.385186:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829668.385198:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829668/real 1787829668] req@ffff9181c55a4700 x1874663043352320/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829693 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829668.385211:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4700 x1874663043352320/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829693 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829668.385220:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829668.385223:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829668.385226:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043352320:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829668.385232:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043352448:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829668.385251:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829668.385262:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829668/real 1787829668] req@ffff9181c55a5880 x1874663043352448/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829693 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829668.385273:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5880 x1874663043352448/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829693 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829668.385281:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829668.385284:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829668.385287:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043352448:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829668.448886:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829669.472898:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:3.0:1787829670.304843:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:3.0:1787829670.304861:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff9181c5620000 from 341 10000000:01000000:3.0:1787829670.304866:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:3.0:1787829670.304927:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f912 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787829670.304937:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f912 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787829670.304946:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f912 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787829670.304964:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d225f0c0 going to sleep for 0 seconds 00000100:00100000:3.0:1787829670.304973:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043352576:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:3.0:1787829670.304997:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f912 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787829670.305002:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f912 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787829670.305007:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f912 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787829670.305013:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f912 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787829670.305021:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f912 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787829670.305027:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3480000 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:3.0:1787829670.305031:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f912 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787829670.305040:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f912 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1787829670.305059:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f912 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787829670.305065:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:3.0:1787829670.305069:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787829670.305074:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f912 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787829670.305080:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f912 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787829670.305098:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:3.0:1787829670.305100:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:2.0:1787829670.305182:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829670.305189:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829670.305192:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829670.305199:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11740 00000100:00080000:2.0:1787829670.305210:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829670.305216:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787829670.305251:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ea00 x1874663043352704/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829670.305277:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829670.305281:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829670.305283:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829670.305289:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11740 00000100:00080000:2.0:1787829670.305294:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787829670.305299:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043352704:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787829670.305300:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829670.305313:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829670.305329:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f100 x1874663043352832/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829670.305339:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829670.305343:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829670.305345:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829670.305349:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11740 00000100:00080000:2.0:1787829670.305353:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829670.305359:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:3.0:1787829670.305361:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000008:00100000:2.0:1787829670.305366:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829670.305379:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f480 x1874663043352960/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829670.305411:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829670/real 1787829670] req@ffff9181e007ea00 x1874663043352704/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829695 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829670.305438:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ea00 x1874663043352704/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829695 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829670.305445:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829670.305448:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829670.305457:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043352704:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829670.305474:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043352832:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829670.305506:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829670.305515:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829670/real 1787829670] req@ffff9181e007f100 x1874663043352832/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829695 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829670.305528:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f100 x1874663043352832/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829695 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829670.305536:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829670.305541:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829670.305544:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043352832:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829670.305551:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043352960:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829670.305573:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829670.305583:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829670/real 1787829670] req@ffff9181e007f480 x1874663043352960/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829695 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829670.305592:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f480 x1874663043352960/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829695 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829670.305598:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829670.305600:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829670.305602:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043352960:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829670.496910:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829671.520932:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829672.544860:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829673.568886:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829674.592904:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829675.552892:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829675.552901:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829675.552903:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829675.552912:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11742 00000100:00080000:0.0:1787829675.552921:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829675.552927:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829675.552959:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4380 x1874663043353088/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829675.552983:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829675.552986:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829675.552988:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829675.552991:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11742 00000100:00080000:0.0:1787829675.552993:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829675.552995:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829675.553003:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829675.553011:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5f80 x1874663043353216/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829675.553017:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829675.553020:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829675.553021:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829675.553023:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11742 00000100:00080000:0.0:1787829675.553025:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829675.553027:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829675.553030:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829675.553035:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6d80 x1874663043353344/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829675.553065:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043353088:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1787829675.553121:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829675.553155:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829675/real 1787829675] req@ffff9181c55a4380 x1874663043353088/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829700 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829675.553173:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4380 x1874663043353088/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829700 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829675.553185:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829675.553190:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829675.553205:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043353088:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829675.553212:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043353216:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829675.553234:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829675.553245:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829675/real 1787829675] req@ffff9181c55a5f80 x1874663043353216/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829700 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829675.553253:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5f80 x1874663043353216/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829700 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829675.553267:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829675.553269:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829675.553271:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043353216:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829675.553275:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043353344:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829675.553287:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829675.553294:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829675/real 1787829675] req@ffff9181c55a6d80 x1874663043353344/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829700 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829675.553301:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6d80 x1874663043353344/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829700 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829675.553306:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829675.553308:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829675.553310:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043353344:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829675.616900:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829676.640912:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829677.664893:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829678.688900:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829679.712905:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829680.672914:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829680.672922:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829680.672926:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829680.672935:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11747 00000100:00080000:2.0:1787829680.672944:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829680.672949:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787829680.672983:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ce00 x1874663043353472/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829680.673007:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829680.673009:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829680.673011:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829680.673013:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11747 00000100:00080000:2.0:1787829680.673015:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829680.673018:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829680.673025:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829680.673035:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d880 x1874663043353600/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829680.673042:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829680.673044:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829680.673045:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829680.673048:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11747 00000100:00080000:2.0:1787829680.673051:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829680.673056:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829680.673058:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043353472:192.168.203.147@tcp:250:kworker.0 00000008:00100000:2.0:1787829680.673060:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829680.673071:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f800 x1874663043353728/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829680.673128:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829680.673160:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829680/real 1787829680] req@ffff9181e007ce00 x1874663043353472/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829705 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829680.673179:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ce00 x1874663043353472/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829705 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829680.673187:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829680.673190:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829680.673202:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043353472:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829680.673207:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043353600:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829680.673227:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829680.673237:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829680/real 1787829680] req@ffff9181e007d880 x1874663043353600/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829705 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829680.673244:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d880 x1874663043353600/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829705 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829680.673250:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829680.673252:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829680.673254:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043353600:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829680.673272:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043353728:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829680.673282:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829680.673289:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829680/real 1787829680] req@ffff9181e007f800 x1874663043353728/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829705 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829680.673296:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f800 x1874663043353728/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829705 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829680.673306:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829680.673308:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829680.673310:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043353728:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829680.736885:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829681.760908:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829682.784846:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829683.808851:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829684.832875:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829685.792853:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829685.792864:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829685.792869:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829685.792879:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11752 00000100:00080000:1.0:1787829685.792890:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829685.792905:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787829685.792937:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0a80 x1874663043353856/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829685.792957:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829685.792960:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829685.792965:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829685.792968:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11752 00000100:00080000:1.0:1787829685.792971:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829685.792974:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829685.792981:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829685.792990:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1f80 x1874663043353984/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829685.792998:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829685.793001:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829685.793003:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829685.793005:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11752 00000100:00080000:1.0:1787829685.793008:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829685.793012:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829685.793018:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043353856:192.168.203.147@tcp:250:kworker.0 00000008:00100000:1.0:1787829685.793020:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829685.793037:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2d80 x1874663043354112/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829685.793090:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829685.793119:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829685/real 1787829685] req@ffff9181c35b0a80 x1874663043353856/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829710 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829685.793139:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0a80 x1874663043353856/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829710 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829685.793147:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829685.793150:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829685.793159:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043353856:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829685.793166:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043353984:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829685.793192:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829685.793208:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829685/real 1787829685] req@ffff9181c35b1f80 x1874663043353984/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829710 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829685.793221:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1f80 x1874663043353984/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829710 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829685.793227:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829685.793230:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829685.793232:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043353984:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829685.793258:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043354112:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829685.793274:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829685.793281:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829685/real 1787829685] req@ffff9181c35b2d80 x1874663043354112/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829710 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829685.793289:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2d80 x1874663043354112/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829710 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829685.793294:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829685.793296:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829685.793298:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043354112:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829685.856901:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829686.880912:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829687.904968:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829688.928979:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829689.952913:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:2.0:1787829690.400825:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-cliir' failed (-108). 10000000:00000400:2.0:1787829690.400832:0:159735:0:(mgc_request.c:1900:mgc_process_log()) MGC192.168.203.147@tcp: IR log lustre-cliir failed, not fatal: rc = -108 10000000:01000000:2.0:1787829690.400838:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:2.0:1787829690.400841:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff9181c5620000 from 52 10000000:01000000:2.0:1787829690.400843:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:2.0:1787829690.400879:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f919 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829690.400888:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f919 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829690.400897:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f919 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787829690.400914:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5c94a80 going to sleep for 0 seconds 00000100:00100000:2.0:1787829690.400923:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043354240:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:2.0:1787829690.400932:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f919 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829690.400938:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f919 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829690.400943:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f919 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829690.400948:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f919 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829690.400953:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f919 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829690.400958:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181d1842c00 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:2.0:1787829690.400960:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f919 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829690.400965:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f919 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1787829690.400970:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f919 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787829690.400973:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:2.0:1787829690.400975:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787829690.400980:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f919 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829690.400984:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f919 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787829690.401002:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:2.0:1787829690.401004:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:1.0:1787829690.401069:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829690.401076:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829690.401079:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829690.401086:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11757 00000100:00080000:1.0:1787829690.401096:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829690.401102:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787829690.401123:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2a00 x1874663043354368/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829690.401148:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829690.401152:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829690.401154:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829690.401159:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11757 00000100:00080000:1.0:1787829690.401163:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829690.401168:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829690.401178:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829690.401192:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043354368:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787829690.401195:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3480 x1874663043354496/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829690.401211:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829690.401214:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829690.401216:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829690.401220:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11757 00000100:00080000:1.0:1787829690.401224:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829690.401229:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829690.401234:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829690.401247:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2680 x1874663043354624/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829690.401263:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829690.401296:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829690/real 1787829690] req@ffff9181c35b2a00 x1874663043354368/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829715 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829690.401312:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2a00 x1874663043354368/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829715 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829690.401332:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829690.401336:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829690.401347:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043354368:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829690.401368:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043354496:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829690.401398:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829690.401410:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829690/real 1787829690] req@ffff9181c35b3480 x1874663043354496/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829715 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829690.401423:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3480 x1874663043354496/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829715 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829690.401432:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829690.401435:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829690.401438:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043354496:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829690.401444:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043354624:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829690.401462:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829690.401472:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829690/real 1787829690] req@ffff9181c35b2680 x1874663043354624/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829715 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829690.401483:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2680 x1874663043354624/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829715 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829690.401491:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829690.401494:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829690.401498:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043354624:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829690.976998:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829692.000891:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829693.024921:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829694.048881:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829695.072894:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829695.520888:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829695.520898:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829695.520902:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829695.520911:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11762 00000100:00080000:1.0:1787829695.520921:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829695.520929:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787829695.520979:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0e00 x1874663043354752/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829695.521000:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829695.521003:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829695.521005:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829695.521010:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11762 00000100:00080000:1.0:1787829695.521013:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829695.521019:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829695.521028:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829695.521043:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0000 x1874663043354880/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829695.521055:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043354752:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787829695.521057:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829695.521063:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829695.521065:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829695.521070:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11762 00000100:00080000:1.0:1787829695.521074:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829695.521077:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829695.521081:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829695.521092:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1180 x1874663043355008/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829695.521114:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829695.521143:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829695/real 1787829695] req@ffff9181c35b0e00 x1874663043354752/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829720 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829695.521156:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0e00 x1874663043354752/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829720 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829695.521169:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829695.521172:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829695.521184:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043354752:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829695.521202:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043354880:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829695.521221:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829695.521253:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829695/real 1787829695] req@ffff9181c35b0000 x1874663043354880/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829720 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829695.521269:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0000 x1874663043354880/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829720 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829695.521279:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829695.521283:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829695.521287:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043354880:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829695.521292:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043355008:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829695.521308:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829695.521315:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829695/real 1787829695] req@ffff9181c35b1180 x1874663043355008/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829720 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829695.521323:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1180 x1874663043355008/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829720 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829695.521328:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829695.521330:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829695.521332:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043355008:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829696.096889:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829697.120849:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829698.144882:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829699.168867:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829700.192870:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829700.640862:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829700.640872:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829700.640875:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829700.640885:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11767 00000100:00080000:0.0:1787829700.640895:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829700.640902:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829700.640938:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7100 x1874663043355136/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829700.640979:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829700.640983:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829700.640985:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829700.640990:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11767 00000100:00080000:0.0:1787829700.640993:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829700.640997:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829700.641006:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829700.641019:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6680 x1874663043355264/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829700.641048:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829700.641051:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829700.641053:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787829700.641060:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043355136:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787829700.641062:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11767 00000100:00080000:0.0:1787829700.641069:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829700.641074:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829700.641078:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829700.641092:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7480 x1874663043355392/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829700.641134:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829700.641171:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829700/real 1787829700] req@ffff9181c55a7100 x1874663043355136/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829725 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829700.641189:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7100 x1874663043355136/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829725 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829700.641226:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829700.641230:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829700.641243:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043355136:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829700.641284:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043355264:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829700.641309:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829700.641320:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829700/real 1787829700] req@ffff9181c55a6680 x1874663043355264/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829725 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829700.641329:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6680 x1874663043355264/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829725 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829700.641343:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829700.641345:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829700.641348:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043355264:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829700.641352:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043355392:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829700.641364:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829700.641371:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829700/real 1787829700] req@ffff9181c55a7480 x1874663043355392/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829725 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829700.641380:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7480 x1874663043355392/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829725 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829700.641386:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829700.641388:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829700.641390:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043355392:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829701.216893:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829702.240877:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829703.264865:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829704.288865:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829705.312902:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829705.760837:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829705.760845:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829705.760849:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829705.760857:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11772 00000100:00080000:2.0:1787829705.760867:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829705.760872:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787829705.760901:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ca80 x1874663043355520/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829705.760921:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829705.760924:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829705.760926:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829705.760929:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11772 00000100:00080000:2.0:1787829705.760931:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829705.760934:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829705.760941:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829705.760950:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007fb80 x1874663043355648/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829705.760958:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829705.760960:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829705.760961:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787829705.760966:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043355520:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787829705.760967:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11772 00000100:00080000:2.0:1787829705.760973:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829705.760976:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829705.760980:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829705.760990:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e680 x1874663043355776/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829705.761033:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829705.761064:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829705/real 1787829705] req@ffff9181e007ca80 x1874663043355520/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829730 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829705.761078:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ca80 x1874663043355520/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829730 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829705.761085:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829705.761088:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829705.761103:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043355520:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829705.761124:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043355648:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829705.761140:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829705.761149:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829705/real 1787829705] req@ffff9181e007fb80 x1874663043355648/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829730 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829705.761157:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007fb80 x1874663043355648/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829730 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829705.761167:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829705.761170:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829705.761172:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043355648:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829705.761176:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043355776:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829705.761186:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829705.761192:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829705/real 1787829705] req@ffff9181e007e680 x1874663043355776/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829730 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829705.761199:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e680 x1874663043355776/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829730 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829705.761204:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829705.761206:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829705.761208:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043355776:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829706.336907:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829707.360928:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829708.384875:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829709.408922:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829710.432921:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829710.880899:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829710.880907:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829710.880910:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829710.880919:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11777 10000000:01000000:2.0:1787829710.880924:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-client' failed (-108). 00000100:00080000:0.0:1787829710.880929:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 10000000:00020000:2.0:1787829710.880930:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:2.0:1787829710.880936:0:159735:0:(mgc_request.c:611:do_requeue()) updating log params 00000100:00080000:0.0:1787829710.880939:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 10000000:01000000:2.0:1787829710.880941:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff9181c5620000 from 7 10000000:01000000:2.0:1787829710.880947:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00000100:00100000:0.0:1787829710.880970:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7b80 x1874663043355904/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:2.0:1787829710.880986:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f920 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787829710.880997:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829710.881005:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829710.881008:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829710.881013:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11777 00010000:00010000:2.0:1787829710.881014:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f920 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787829710.881016:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829710.881023:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829710.881033:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:2.0:1787829710.881036:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f920 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787829710.881046:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7480 x1874663043356160/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829710.881057:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5c94a80 going to sleep for 0 seconds 00000100:00080000:0.0:1787829710.881058:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787829710.881058:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043355904:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787829710.881069:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:2.0:1787829710.881070:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043356032:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00080000:0.0:1787829710.881072:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829710.881080:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11777 00000100:00080000:0.0:1787829710.881084:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829710.881087:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:2.0:1787829710.881090:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f920 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000008:00100000:0.0:1787829710.881091:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:2.0:1787829710.881103:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f920 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787829710.881104:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6680 x1874663043356288/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:2.0:1787829710.881126:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f920 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829710.881136:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f920 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829710.881146:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f920 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829710.881154:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181d1842c00 already unused, calling callback (ffffffffc1273c20) 00000100:00100000:3.0:1787829710.881156:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043356160:192.168.203.147@tcp:38:kworker.0 10000000:00010000:2.0:1787829710.881160:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f920 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829710.881171:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f920 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1787829710.881180:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f920 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787829710.881182:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043356288:192.168.203.147@tcp:38:kworker.0 10000000:01000000:2.0:1787829710.881193:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:2.0:1787829710.881199:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787829710.881207:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f920 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829710.881215:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f920 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787829710.881235:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:2.0:1787829710.881239:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:CONNECTING 00010000:00010000:2.0:1787829711.456918:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829712.480906:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829713.504902:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1787829713.953023:0:2379:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:1.0:1787829713.953070:0:2379:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787829713.953087:0:2379:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1787829713.953125:0:2379:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1787829713.953134:0:2379:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:3.0:1787829713.953181:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829710/real 1787829713] req@ffff9181c55a7480 x1874663043356160/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829735 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829713.953214:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7480 x1874663043356160/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829735 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829713.953246:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829713.953251:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829713.953257:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043356160:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829713.953265:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829710/real 1787829713] req@ffff9181c55a6680 x1874663043356288/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829735 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829713.953279:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6680 x1874663043356288/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829735 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829713.953288:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829713.953291:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829713.953295:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043356288:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829713.953303:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829710/real 1787829713] req@ffff9181c55a7b80 x1874663043355904/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829735 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829713.953317:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7b80 x1874663043355904/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829735 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829713.953325:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829713.953350:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829713.953382:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043355904:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787829713.953951:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829713.953959:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829713.953963:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829713.953969:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11782 00000100:00080000:1.0:1787829713.953977:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829713.953984:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787829713.954018:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3100 x1874663043356416/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829713.954059:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829713.954063:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829713.954065:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829713.954070:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11782 00000100:00080000:1.0:1787829713.954074:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829713.954078:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829713.954088:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829713.954101:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1500 x1874663043356544/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829713.954112:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829713.954115:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829713.954117:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829713.954120:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11782 00000100:00080000:1.0:1787829713.954124:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829713.954128:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829713.954133:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829713.954143:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0380 x1874663043356672/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829713.954334:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043356416:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787829713.954409:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829713.954439:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829713/real 1787829713] req@ffff9181c35b3100 x1874663043356416/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829738 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829713.954455:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3100 x1874663043356416/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829738 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829713.954464:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829713.954467:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829713.954482:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043356416:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829713.954490:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043356544:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829713.954524:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829713.954539:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829713/real 1787829713] req@ffff9181c35b1500 x1874663043356544/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829738 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829713.954552:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1500 x1874663043356544/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829738 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829713.954560:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829713.954563:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829713.954566:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043356544:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829713.954573:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043356672:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829713.954607:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829713.954619:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829713/real 1787829713] req@ffff9181c35b0380 x1874663043356672/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829738 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829713.954632:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0380 x1874663043356672/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829738 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829713.954641:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829713.954645:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829713.954648:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043356672:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829714.528903:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829715.552922:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829716.576918:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829717.600877:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829718.624878:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829719.072852:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829719.072863:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829719.072867:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829719.072877:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11785 00000100:00080000:1.0:1787829719.072889:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829719.072895:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787829719.072927:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1c00 x1874663043356800/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829719.072950:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829719.072953:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829719.072954:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829719.072957:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11785 00000100:00080000:1.0:1787829719.072959:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829719.072962:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829719.072969:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829719.072979:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0380 x1874663043356928/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829719.072988:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1787829719.072991:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043356800:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787829719.072992:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829719.072994:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829719.073003:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11785 00000100:00080000:1.0:1787829719.073005:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829719.073008:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829719.073011:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829719.073020:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1500 x1874663043357056/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829719.073060:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829719.073096:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829719/real 1787829719] req@ffff9181c35b1c00 x1874663043356800/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829744 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829719.073110:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1c00 x1874663043356800/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829744 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829719.073119:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829719.073123:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829719.073138:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043356800:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829719.073163:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043356928:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829719.073187:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829719.073197:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829719/real 1787829719] req@ffff9181c35b0380 x1874663043356928/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829744 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829719.073206:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0380 x1874663043356928/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829744 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829719.073211:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829719.073214:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829719.073216:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043356928:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829719.073220:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043357056:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829719.073232:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829719.073238:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829719/real 1787829719] req@ffff9181c35b1500 x1874663043357056/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829744 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829719.073245:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1500 x1874663043357056/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829744 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829719.073255:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829719.073257:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829719.073259:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043357056:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829719.648879:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829720.672942:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829721.696885:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829722.720945:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829723.744890:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829724.192831:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829724.192839:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829724.192842:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829724.192850:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11790 00000100:00080000:1.0:1787829724.192860:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829724.192865:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787829724.192897:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3100 x1874663043357184/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829724.192919:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829724.192922:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829724.192923:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829724.192926:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11790 00000100:00080000:1.0:1787829724.192929:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829724.192931:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829724.192937:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829724.192948:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1180 x1874663043357312/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829724.192956:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829724.192958:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829724.192960:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829724.192962:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11790 00000100:00080000:1.0:1787829724.192965:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829724.192967:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829724.192971:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829724.192976:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0000 x1874663043357440/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829724.192983:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043357184:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787829724.193059:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829724.193097:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829724/real 1787829724] req@ffff9181c35b3100 x1874663043357184/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829749 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829724.193115:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3100 x1874663043357184/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829749 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829724.193136:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829724.193141:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829724.193158:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043357184:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829724.193174:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043357312:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829724.193211:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829724.193225:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829724/real 1787829724] req@ffff9181c35b1180 x1874663043357312/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829749 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829724.193246:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1180 x1874663043357312/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829749 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829724.193267:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829724.193272:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829724.193276:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043357312:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829724.193318:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043357440:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829724.193339:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829724.193360:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829724/real 1787829724] req@ffff9181c35b0000 x1874663043357440/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829749 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829724.193373:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0000 x1874663043357440/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829749 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829724.193382:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829724.193385:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829724.193388:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043357440:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829724.768881:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829725.792883:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829726.816887:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829727.840894:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829728.864884:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829729.312869:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829729.312877:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829729.312881:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829729.312889:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11795 00000100:00080000:0.0:1787829729.312898:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829729.312905:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829729.312943:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7100 x1874663043357568/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829729.312972:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829729.312976:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829729.312978:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829729.312983:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11795 00000100:00080000:0.0:1787829729.312987:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829729.312991:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829729.313001:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829729.313015:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6d80 x1874663043357696/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829729.313023:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829729.313026:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829729.313028:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829729.313031:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11795 00000100:00080000:0.0:1787829729.313034:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1787829729.313037:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043357568:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787829729.313040:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829729.313046:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829729.313054:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5f80 x1874663043357824/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829729.313111:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829729.313150:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829729/real 1787829729] req@ffff9181c55a7100 x1874663043357568/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829754 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829729.313170:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7100 x1874663043357568/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829754 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829729.313183:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829729.313188:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829729.313204:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043357568:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829729.313213:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043357696:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829729.313243:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829729.313277:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829729/real 1787829729] req@ffff9181c55a6d80 x1874663043357696/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829754 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829729.313293:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6d80 x1874663043357696/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829754 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829729.313303:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829729.313306:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829729.313310:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043357696:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829729.313336:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043357824:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829729.313359:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829729.313373:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829729/real 1787829729] req@ffff9181c55a5f80 x1874663043357824/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829754 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829729.313387:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5f80 x1874663043357824/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829754 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829729.313397:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829729.313400:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829729.313404:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043357824:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829729.888909:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829730.912878:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:2.0:1787829730.912921:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'params' failed (-108). 10000000:00020000:2.0:1787829730.912927:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:2.0:1787829730.912931:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:2.0:1787829730.912932:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff9181c87b19b8 from 1 10000000:01000000:2.0:1787829730.912935:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00010000:00010000:2.0:1787829730.912972:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f927 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829730.912981:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f927 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829730.912991:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f927 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787829730.913005:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5c94a80 going to sleep for 0 seconds 00000100:00100000:2.0:1787829730.913016:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043357952:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:2.0:1787829730.913025:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f927 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829730.913031:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f927 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829730.913036:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f927 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829730.913041:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f927 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829730.913045:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f927 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829730.913054:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181d1842c00 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:2.0:1787829730.913057:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f927 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829730.913062:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f927 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1787829730.913066:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f927 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787829730.913070:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:2.0:1787829730.913072:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787829730.913077:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f927 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829730.913082:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f927 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787829730.913096:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:2.0:1787829730.913098:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:1.0:1787829730.913155:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829730.913161:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829730.913165:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829730.913171:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11801 00000100:00080000:1.0:1787829730.913180:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829730.913187:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787829730.913209:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0e00 x1874663043358080/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829730.913239:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829730.913257:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829730.913260:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829730.913265:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11801 00000100:00080000:1.0:1787829730.913269:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829730.913274:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829730.913284:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829730.913284:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043358080:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787829730.913303:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2680 x1874663043358208/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829730.913316:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829730.913320:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829730.913322:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829730.913327:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11801 00000100:00080000:1.0:1787829730.913331:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829730.913335:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829730.913340:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000800:00000100:2.0:1787829730.913346:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787829730.913353:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3480 x1874663043358336/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829730.913380:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829730/real 1787829730] req@ffff9181c35b0e00 x1874663043358080/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829755 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829730.913393:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0e00 x1874663043358080/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829755 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829730.913401:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829730.913405:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829730.913416:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043358080:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829730.913431:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043358208:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829730.913450:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829730.913459:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829730/real 1787829730] req@ffff9181c35b2680 x1874663043358208/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829755 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829730.913467:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2680 x1874663043358208/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829755 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829730.913473:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829730.913475:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829730.913477:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043358208:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829730.913481:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043358336:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829730.913491:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829730.913496:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829730/real 1787829730] req@ffff9181c35b3480 x1874663043358336/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829755 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829730.913503:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3480 x1874663043358336/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829755 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829730.913509:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829730.913510:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829730.913512:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043358336:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829731.936897:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829732.960944:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829733.984894:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829735.008905:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829735.968852:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829735.968863:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829735.968867:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829735.968877:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11802 00000100:00080000:0.0:1787829735.968887:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829735.968893:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829735.968927:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4380 x1874663043358464/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829735.968955:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829735.968959:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829735.968961:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829735.968966:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11802 00000100:00080000:0.0:1787829735.968970:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829735.968975:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829735.968984:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829735.968998:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5880 x1874663043358592/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829735.969008:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829735.969012:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829735.969014:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829735.969021:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11802 00000100:00100000:2.0:1787829735.969022:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043358464:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787829735.969026:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829735.969033:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829735.969038:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829735.969054:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4700 x1874663043358720/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829735.969081:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829735.969111:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829735/real 1787829735] req@ffff9181c55a4380 x1874663043358464/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829760 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829735.969125:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4380 x1874663043358464/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829760 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829735.969133:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829735.969135:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829735.969148:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043358464:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829735.969153:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043358592:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829735.969168:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829735.969175:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829735/real 1787829735] req@ffff9181c55a5880 x1874663043358592/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829760 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829735.969183:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5880 x1874663043358592/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829760 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829735.969189:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829735.969191:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829735.969193:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043358592:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829735.969230:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043358720:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829735.969244:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829735.969251:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829735/real 1787829735] req@ffff9181c55a4700 x1874663043358720/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829760 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829735.969260:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4700 x1874663043358720/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829760 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829735.969265:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829735.969267:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829735.969269:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043358720:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829736.032897:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829737.056977:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829738.080925:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829739.104855:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829740.128918:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829741.088885:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829741.088896:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829741.088902:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829741.088912:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11807 00000100:00080000:2.0:1787829741.088922:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829741.088931:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787829741.088979:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007dc00 x1874663043358848/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829741.089013:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829741.089027:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829741.089030:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829741.089035:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11807 00000100:00080000:2.0:1787829741.089040:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829741.089045:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829741.089055:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829741.089066:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043358848:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829741.089076:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c000 x1874663043358976/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829741.089103:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829741.089107:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829741.089110:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829741.089116:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11807 00000100:00080000:2.0:1787829741.089121:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829741.089127:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829741.089134:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000800:00000100:0.0:1787829741.089150:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829741.089153:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c380 x1874663043359104/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787829741.089225:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829741/real 1787829741] req@ffff9181e007dc00 x1874663043358848/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829766 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829741.089245:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007dc00 x1874663043358848/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829766 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829741.089256:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829741.089261:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829741.089276:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043358848:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787829741.089299:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043358976:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787829741.089336:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829741.089351:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829741/real 1787829741] req@ffff9181e007c000 x1874663043358976/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829766 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829741.089365:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c000 x1874663043358976/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829766 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829741.089373:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829741.089377:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829741.089381:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043358976:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787829741.089388:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043359104:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787829741.089411:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829741.089426:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829741/real 1787829741] req@ffff9181e007c380 x1874663043359104/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829766 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829741.089440:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c380 x1874663043359104/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829766 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829741.089450:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829741.089454:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829741.089459:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043359104:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829741.152879:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829742.176878:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829743.200914:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829744.224932:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829745.248884:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829746.208833:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829746.208841:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829746.208845:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829746.208852:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11812 00000100:00080000:1.0:1787829746.208861:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829746.208867:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787829746.208910:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2a00 x1874663043359232/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829746.208933:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829746.208936:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829746.208937:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829746.208940:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11812 00000100:00080000:1.0:1787829746.208942:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829746.208944:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829746.208951:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829746.208960:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2d80 x1874663043359360/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829746.208968:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829746.208971:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829746.208972:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829746.208974:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11812 00000100:00080000:1.0:1787829746.208976:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829746.208979:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829746.208983:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829746.208989:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1f80 x1874663043359488/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787829746.208998:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043359232:192.168.203.147@tcp:250:kworker.0 00000800:00000100:0.0:1787829746.209066:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829746.209106:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829746/real 1787829746] req@ffff9181c35b2a00 x1874663043359232/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829771 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829746.209120:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2a00 x1874663043359232/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829771 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829746.209128:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829746.209131:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829746.209170:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043359232:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787829746.209176:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043359360:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787829746.209198:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829746.209211:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829746/real 1787829746] req@ffff9181c35b2d80 x1874663043359360/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829771 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829746.209220:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2d80 x1874663043359360/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829771 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829746.209241:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829746.209244:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829746.209247:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043359360:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787829746.209274:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043359488:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787829746.209296:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829746.209309:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829746/real 1787829746] req@ffff9181c35b1f80 x1874663043359488/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829771 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829746.209329:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1f80 x1874663043359488/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829771 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829746.209337:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829746.209341:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829746.209344:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043359488:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829746.272891:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829747.296872:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829748.320890:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829749.344890:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829750.368915:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829751.328867:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829751.328876:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829751.328879:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829751.328888:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11817 10000000:01000000:2.0:1787829751.328892:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-sptlrpc' failed (-108). 10000000:00020000:2.0:1787829751.328908:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00080000:0.0:1787829751.328911:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829751.328920:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829751.328955:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5c00 x1874663043359616/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829751.328982:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829751.328986:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829751.328988:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829751.328994:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11817 00000100:00080000:0.0:1787829751.328998:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829751.329003:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787829751.329010:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043359616:192.168.203.147@tcp:250:kworker.0 00000008:00100000:0.0:1787829751.329012:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829751.329028:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5180 x1874663043359744/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829751.329039:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829751.329043:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829751.329045:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829751.329050:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11817 00000100:00080000:0.0:1787829751.329055:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829751.329059:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829751.329065:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000800:00000100:2.0:1787829751.329074:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829751.329080:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5500 x1874663043359872/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829751.329126:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829751/real 1787829751] req@ffff9181c55a5c00 x1874663043359616/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829776 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829751.329141:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5c00 x1874663043359616/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829776 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829751.329157:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829751.329160:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829751.329163:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043359616:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829751.329181:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043359744:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829751.329199:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829751.329207:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829751/real 1787829751] req@ffff9181c55a5180 x1874663043359744/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829776 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829751.329215:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5180 x1874663043359744/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829776 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829751.329220:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829751.329223:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829751.329225:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043359744:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829751.329228:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043359872:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829751.329240:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829751.329245:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829751/real 1787829751] req@ffff9181c55a5500 x1874663043359872/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829776 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829751.329252:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5500 x1874663043359872/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829776 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829751.329257:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829751.329259:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829751.329262:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043359872:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829751.392901:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829752.416920:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:2.0:1787829752.864854:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:2.0:1787829752.864862:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff9181c5620000 from 341 10000000:01000000:2.0:1787829752.864869:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:2.0:1787829752.864931:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f92e lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829752.864952:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f92e lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829752.864967:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f92e lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787829752.864990:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c5c94a80 going to sleep for 0 seconds 00000100:00100000:2.0:1787829752.865002:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c380 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043360000:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:2.0:1787829752.865017:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f92e lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829752.865027:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f92e lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829752.865034:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f92e lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829752.865043:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f92e lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829752.865052:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f92e lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829752.865059:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181d1842c00 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:2.0:1787829752.865064:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f92e lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829752.865073:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f92e lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1787829752.865081:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f92e lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787829752.865088:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:2.0:1787829752.865107:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787829752.865114:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f92e lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829752.865122:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181d1842c00/0xbef42c0221d6f92e lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787829752.865142:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:2.0:1787829752.865146:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:1.0:1787829752.865199:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829752.865204:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829752.865207:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829752.865212:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11823 00000100:00080000:1.0:1787829752.865220:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829752.865225:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787829752.865242:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0a80 x1874663043360128/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829752.865261:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829752.865263:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829752.865265:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829752.865268:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11823 00000100:00080000:1.0:1787829752.865270:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829752.865272:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829752.865279:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829752.865287:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3800 x1874663043360256/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829752.865294:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829752.865297:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829752.865298:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829752.865304:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11823 00000100:00100000:2.0:1787829752.865305:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043360128:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787829752.865309:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829752.865315:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829752.865320:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829752.865334:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3b80 x1874663043360384/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829752.865365:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829752.865394:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829752/real 1787829752] req@ffff9181c35b0a80 x1874663043360128/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829777 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829752.865406:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0a80 x1874663043360128/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829777 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829752.865413:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829752.865416:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829752.865431:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043360128:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829752.865438:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043360256:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829752.865455:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829752.865476:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829752/real 1787829752] req@ffff9181c35b3800 x1874663043360256/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829777 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829752.865484:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3800 x1874663043360256/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829777 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829752.865490:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829752.865492:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829752.865495:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043360256:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829752.865510:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043360384:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829752.865522:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829752.865528:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829752/real 1787829752] req@ffff9181c35b3b80 x1874663043360384/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829777 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829752.865535:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3b80 x1874663043360384/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829777 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829752.865542:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829752.865545:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829752.865548:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043360384:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829753.440923:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829754.464899:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829755.488898:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829756.512862:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829757.536910:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829757.984816:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829757.984824:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829757.984827:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829757.984835:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11824 00000100:00080000:1.0:1787829757.984844:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829757.984850:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787829757.984880:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0700 x1874663043360512/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829757.984901:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829757.984903:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829757.984905:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829757.984908:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11824 00000100:00080000:1.0:1787829757.984910:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829757.984913:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829757.984919:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829757.984929:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3b80 x1874663043360640/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829757.984935:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829757.984938:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829757.984939:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829757.984941:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11824 00000100:00080000:1.0:1787829757.984943:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829757.984945:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829757.984948:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829757.984954:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3800 x1874663043360768/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829757.984979:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043360512:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787829757.985036:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829757.985079:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829757/real 1787829757] req@ffff9181c35b0700 x1874663043360512/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829782 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829757.985095:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0700 x1874663043360512/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829782 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829757.985102:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829757.985105:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829757.985117:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043360512:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829757.985123:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043360640:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829757.985142:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829757.985153:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829757/real 1787829757] req@ffff9181c35b3b80 x1874663043360640/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829782 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829757.985162:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3b80 x1874663043360640/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829782 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829757.985168:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829757.985171:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829757.985173:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043360640:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829757.985176:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043360768:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829757.985187:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829757.985192:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829757/real 1787829757] req@ffff9181c35b3800 x1874663043360768/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829782 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829757.985204:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3800 x1874663043360768/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829782 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829757.985210:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829757.985211:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829757.985213:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043360768:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829758.560867:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829759.584911:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829760.608895:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829761.632896:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829762.656896:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829763.104866:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829763.104880:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829763.104885:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829763.104898:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11829 00000100:00080000:2.0:1787829763.104913:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829763.104922:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787829763.104972:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c380 x1874663043360896/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829763.105007:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829763.105012:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829763.105014:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829763.105033:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11829 00000100:00080000:2.0:1787829763.105039:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829763.105045:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829763.105057:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829763.105063:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043360896:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829763.105077:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c000 x1874663043361024/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829763.105091:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829763.105095:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829763.105097:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829763.105102:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11829 00000100:00080000:2.0:1787829763.105106:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829763.105111:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829763.105117:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829763.105132:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007dc00 x1874663043361152/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829763.105145:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829763.105183:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829763/real 1787829763] req@ffff9181e007c380 x1874663043360896/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829788 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829763.105201:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c380 x1874663043360896/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829788 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829763.105215:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829763.105220:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829763.105235:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043360896:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787829763.105265:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043361024:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787829763.105288:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829763.105299:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829763/real 1787829763] req@ffff9181e007c000 x1874663043361024/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829788 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829763.105308:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c000 x1874663043361024/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829788 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829763.105314:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829763.105316:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829763.105319:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043361024:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787829763.105322:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043361152:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787829763.105340:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829763.105348:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829763/real 1787829763] req@ffff9181e007dc00 x1874663043361152/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829788 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829763.105356:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007dc00 x1874663043361152/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829788 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829763.105361:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829763.105363:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829763.105365:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043361152:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829763.680951:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829764.704872:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829765.728900:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829766.752909:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829767.776967:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829768.224846:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829768.224856:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829768.224859:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829768.224867:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11834 00000100:00080000:2.0:1787829768.224876:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829768.224882:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787829768.224913:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e680 x1874663043361280/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829768.224937:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829768.224940:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829768.224941:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829768.224945:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11834 00000100:00080000:2.0:1787829768.224948:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829768.224950:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829768.224958:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829768.224967:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007fb80 x1874663043361408/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829768.224975:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829768.224978:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829768.224979:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1787829768.224994:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043361280:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787829768.224998:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11834 00000100:00080000:2.0:1787829768.225005:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829768.225008:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829768.225012:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829768.225022:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ca80 x1874663043361536/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1787829768.225055:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829768.225085:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829768/real 1787829768] req@ffff9181e007e680 x1874663043361280/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829793 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829768.225100:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e680 x1874663043361280/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829793 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829768.225108:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829768.225111:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829768.225122:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043361280:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787829768.225141:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043361408:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787829768.225158:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829768.225169:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829768/real 1787829768] req@ffff9181e007fb80 x1874663043361408/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829793 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829768.225179:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007fb80 x1874663043361408/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829793 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829768.225185:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829768.225188:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829768.225190:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043361408:192.168.203.147@tcp:38:kworker.0 00000100:00100000:0.0:1787829768.225194:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043361536:192.168.203.147@tcp:38:kworker.0 00000800:00000100:0.0:1787829768.225204:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829768.225210:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829768/real 1787829768] req@ffff9181e007ca80 x1874663043361536/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829793 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829768.225218:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ca80 x1874663043361536/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829793 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829768.225223:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1787829768.225225:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:0.0:1787829768.225227:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043361536:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829768.800848:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829769.824872:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829770.848933:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829771.872971:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829772.896900:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829773.344894:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829773.344902:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829773.344909:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 10000000:01000000:3.0:1787829773.344910:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-cliir' failed (-108). 10000000:00000400:3.0:1787829773.344925:0:159735:0:(mgc_request.c:1900:mgc_process_log()) MGC192.168.203.147@tcp: IR log lustre-cliir failed, not fatal: rc = -108 00000100:00080000:0.0:1787829773.344930:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11839 10000000:01000000:3.0:1787829773.344940:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 00000100:00080000:0.0:1787829773.344941:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 10000000:01000000:3.0:1787829773.344959:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff9181c5620000 from 52 00000100:00080000:0.0:1787829773.344967:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 10000000:01000000:3.0:1787829773.344969:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00000100:00100000:0.0:1787829773.345010:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4e00 x1874663043361792/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:3.0:1787829773.345030:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f935 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787829773.345039:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829773.345052:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00010000:00010000:3.0:1787829773.345053:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f935 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787829773.345056:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829773.345073:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11839 00000100:00080000:0.0:1787829773.345080:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00010000:00010000:3.0:1787829773.345081:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f935 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787829773.345089:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787829773.345103:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043361792:192.168.203.147@tcp:250:kworker.0 00000008:00100000:0.0:1787829773.345108:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829773.345115:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d225f0c0 going to sleep for 0 seconds 00000100:00100000:3.0:1787829773.345127:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043361664:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00100000:0.0:1787829773.345136:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7800 x1874663043361920/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:3.0:1787829773.345148:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f935 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787829773.345177:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f935 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787829773.345183:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:3.0:1787829773.345188:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f935 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787829773.345191:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829773.345198:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829773.345210:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11839 00010000:00010000:3.0:1787829773.345212:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f935 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787829773.345218:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00010000:00010000:3.0:1787829773.345228:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f935 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787829773.345229:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787829773.345240:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043361920:192.168.203.147@tcp:38:kworker.0 00000008:00100000:0.0:1787829773.345249:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:3.0:1787829773.345251:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181c3480000 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:3.0:1787829773.345259:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f935 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787829773.345276:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5500 x1874663043362048/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:3.0:1787829773.345280:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f935 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1787829773.345301:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f935 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787829773.345306:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043362048:192.168.203.147@tcp:38:kworker.0 10000000:01000000:3.0:1787829773.345316:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:3.0:1787829773.345323:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1787829773.345331:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f935 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1787829773.345340:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181c3480000/0xbef42c0221d6f935 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1787829773.345370:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:3.0:1787829773.345374:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:CONNECTING 00010000:00010000:1.0:1787829773.920913:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829774.944900:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829775.968899:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1787829776.417141:0:2376:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:1.0:1787829776.417195:0:2376:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1787829776.417213:0:2376:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1787829776.417264:0:2376:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:1.0:1787829776.417274:0:2376:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:2.0:1787829776.417392:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829773/real 1787829776] req@ffff9181c55a5500 x1874663043362048/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829798 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829776.417433:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5500 x1874663043362048/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829798 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829776.417445:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829776.417451:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829776.417455:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043362048:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829776.417465:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829773/real 1787829776] req@ffff9181c55a7800 x1874663043361920/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829798 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829776.417478:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7800 x1874663043361920/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829798 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829776.417485:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829776.417489:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829776.417492:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043361920:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829776.417500:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829773/real 1787829776] req@ffff9181c55a4e00 x1874663043361792/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829798 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829776.417513:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4e00 x1874663043361792/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829798 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829776.417521:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829776.417557:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829776.417584:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043361792:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787829776.417721:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829776.417727:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829776.417730:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829776.417737:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11845 00000100:00080000:3.0:1787829776.417784:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829776.417869:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829776.417923:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043362176/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829776.417959:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829776.417963:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829776.417965:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829776.417971:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11845 00000100:00080000:3.0:1787829776.417975:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829776.417980:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829776.417993:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829776.418006:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043362304/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829776.418015:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829776.418018:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829776.418020:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829776.418024:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11845 00000100:00080000:3.0:1787829776.418027:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829776.418031:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829776.418035:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829776.418044:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635f80 x1874663043362432/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829776.418132:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043362176:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787829776.418222:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829776.418258:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829776/real 1787829776] req@ffff9181c5637100 x1874663043362176/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829801 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829776.418269:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043362176/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829801 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829776.418276:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829776.418279:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829776.418282:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043362176:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829776.418288:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043362304:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829776.418307:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829776.418316:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829776/real 1787829776] req@ffff9181c5637480 x1874663043362304/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829801 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829776.418325:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043362304/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829801 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829776.418332:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829776.418335:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829776.418337:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043362304:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829776.418341:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043362432:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829776.418354:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829776.418361:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829776/real 1787829776] req@ffff9181c5635f80 x1874663043362432/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829801 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829776.418376:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635f80 x1874663043362432/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829801 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829776.418383:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829776.418386:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829776.418388:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043362432:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829776.992885:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829778.016871:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829779.040869:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829780.064872:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829781.088879:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829781.536838:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829781.536846:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829781.536849:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829781.536858:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11848 00000100:00080000:3.0:1787829781.536867:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829781.536873:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829781.536918:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634e00 x1874663043362560/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829781.536957:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829781.536963:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829781.536964:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829781.536968:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11848 00000100:00080000:3.0:1787829781.536972:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829781.536976:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829781.536984:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829781.536996:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043362560:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829781.536998:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635f80 x1874663043362688/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829781.537010:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829781.537013:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829781.537015:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829781.537019:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11848 00000100:00080000:3.0:1787829781.537022:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829781.537026:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829781.537030:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829781.537039:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043362816/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829781.537048:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829781.537079:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829781/real 1787829781] req@ffff9181c5634e00 x1874663043362560/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829806 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829781.537101:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634e00 x1874663043362560/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829806 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829781.537112:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829781.537116:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829781.537128:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043362560:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829781.537147:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043362688:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829781.537166:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829781.537177:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829781/real 1787829781] req@ffff9181c5635f80 x1874663043362688/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829806 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829781.537188:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635f80 x1874663043362688/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829806 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829781.537194:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829781.537196:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829781.537197:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043362688:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829781.537201:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043362816:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829781.537210:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829781.537215:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829781/real 1787829781] req@ffff9181c5637480 x1874663043362816/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829806 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829781.537223:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043362816/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829806 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829781.537227:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829781.537228:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829781.537230:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043362816:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829782.112848:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829783.136904:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829784.160852:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829785.184854:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829786.208854:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829786.656907:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829786.656918:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829786.656922:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829786.656933:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11853 00000100:00080000:3.0:1787829786.656946:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829786.656954:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829786.656990:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043362944/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829786.657014:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829786.657016:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829786.657018:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829786.657021:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11853 00000100:00080000:3.0:1787829786.657024:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829786.657026:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829786.657034:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829786.657042:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043363072/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829786.657050:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829786.657054:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:2.0:1787829786.657056:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043362944:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787829786.657056:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829786.657064:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11853 00000100:00080000:3.0:1787829786.657067:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829786.657069:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829786.657072:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829786.657080:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043363200/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829786.657114:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829786.657145:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829786/real 1787829786] req@ffff9181c5637100 x1874663043362944/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829811 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829786.657158:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043362944/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829811 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829786.657167:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829786.657170:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829786.657182:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043362944:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829786.657202:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043363072:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829786.657220:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829786.657232:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829786/real 1787829786] req@ffff9181c5635500 x1874663043363072/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829811 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829786.657240:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043363072/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829811 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829786.657248:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829786.657252:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829786.657256:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043363072:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829786.657262:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043363200:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829786.657283:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829786.657296:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829786/real 1787829786] req@ffff9181c5634380 x1874663043363200/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829811 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829786.657313:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043363200/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829811 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829786.657319:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829786.657321:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829786.657323:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043363200:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829787.232905:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829788.256924:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829789.280931:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829790.304895:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829791.328914:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829791.776885:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829791.776893:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829791.776896:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829791.776905:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11858 00000100:00080000:3.0:1787829791.776915:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829791.776921:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829791.776954:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043363328/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829791.776977:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829791.776980:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829791.776982:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829791.776985:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11858 00000100:00080000:3.0:1787829791.776987:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829791.776994:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829791.777001:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829791.777011:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043363456/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829791.777019:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829791.777021:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829791.777025:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1787829791.777027:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043363328:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787829791.777029:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11858 00000100:00080000:3.0:1787829791.777034:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829791.777040:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829791.777043:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829791.777052:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043363584/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829791.777091:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829791.777123:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829791/real 1787829791] req@ffff9181c5636d80 x1874663043363328/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829816 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829791.777136:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043363328/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829816 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829791.777143:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829791.777146:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829791.777165:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043363328:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829791.777186:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043363456:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829791.777206:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829791.777215:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829791/real 1787829791] req@ffff9181c5637800 x1874663043363456/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829816 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829791.777224:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043363456/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829816 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829791.777229:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829791.777232:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829791.777234:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043363456:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829791.777237:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043363584:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829791.777249:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829791.777255:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829791/real 1787829791] req@ffff9181c5636a00 x1874663043363584/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829816 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829791.777262:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043363584/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829816 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829791.777267:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829791.777269:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829791.777271:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043363584:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829792.352914:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787829793.376950:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-client' failed (-108). 00010000:00010000:2.0:1787829793.376950:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:00020000:1.0:1787829793.376959:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:1.0:1787829793.376963:0:159735:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:1.0:1787829793.376966:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff9181c5620000 from 7 10000000:01000000:1.0:1787829793.376969:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00010000:00010000:1.0:1787829793.377026:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f93c lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829793.377038:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f93c lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829793.377052:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f93c lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787829793.377072:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215d180 going to sleep for 0 seconds 00000100:00100000:1.0:1787829793.377081:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043363712:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787829793.377091:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f93c lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829793.377098:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f93c lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829793.377103:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f93c lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829793.377112:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f93c lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829793.377118:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f93c lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829793.377122:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b5c00 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787829793.377125:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f93c lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829793.377130:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f93c lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787829793.377134:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f93c lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787829793.377138:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:1.0:1787829793.377141:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787829793.377146:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f93c lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829793.377150:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b5c00/0xbef42c0221d6f93c lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787829793.377166:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787829793.377169:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:3.0:1787829793.377213:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829793.377218:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829793.377221:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829793.377226:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11863 00000100:00080000:3.0:1787829793.377233:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829793.377238:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829793.377256:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043363840/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829793.377279:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829793.377282:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829793.377283:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829793.377287:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11863 00000100:00080000:3.0:1787829793.377290:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829793.377294:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829793.377301:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829793.377308:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043363840:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829793.377313:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637b80 x1874663043363968/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829793.377322:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829793.377324:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829793.377326:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829793.377328:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11863 00000100:00080000:3.0:1787829793.377330:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829793.377332:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829793.377337:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829793.377345:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043364096/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829793.377375:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829793.377403:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829793/real 1787829793] req@ffff9181c5635c00 x1874663043363840/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829818 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829793.377416:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043363840/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829818 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829793.377423:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829793.377426:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829793.377436:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043363840:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829793.377451:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043363968:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829793.377470:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829793.377479:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829793/real 1787829793] req@ffff9181c5637b80 x1874663043363968/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829818 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829793.377494:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637b80 x1874663043363968/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829818 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829793.377502:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829793.377506:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829793.377509:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043363968:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829793.377516:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043364096:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829793.377538:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829793.377549:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829793/real 1787829793] req@ffff9181c5634000 x1874663043364096/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829818 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829793.377557:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043364096/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829818 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829793.377563:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829793.377565:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829793.377567:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043364096:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829794.400915:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829795.424908:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829796.448955:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829797.472918:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829798.432909:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829798.432918:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829798.432923:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829798.432932:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11865 00000100:00080000:1.0:1787829798.432943:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829798.432951:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787829798.432990:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0a80 x1874663043364224/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829798.433029:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829798.433051:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829798.433053:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829798.433060:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11865 00000100:00080000:1.0:1787829798.433064:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829798.433069:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829798.433080:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829798.433086:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043364224:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787829798.433098:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1f80 x1874663043364352/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829798.433128:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829798.433132:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829798.433134:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829798.433140:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11865 00000100:00080000:1.0:1787829798.433146:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829798.433151:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829798.433157:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829798.433184:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2d80 x1874663043364480/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829798.433185:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829798.433229:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829798/real 1787829798] req@ffff9181c35b0a80 x1874663043364224/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829823 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829798.433249:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0a80 x1874663043364224/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829823 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829798.433491:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829798.433498:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829798.433556:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043364224:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829798.433593:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043364352:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829798.433621:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829798.433636:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829798/real 1787829798] req@ffff9181c35b1f80 x1874663043364352/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829823 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829798.433649:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1f80 x1874663043364352/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829823 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829798.433658:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829798.433661:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829798.433665:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043364352:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829798.433671:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043364480:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829798.433691:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829798.433703:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829798/real 1787829798] req@ffff9181c35b2d80 x1874663043364480/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829823 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829798.433715:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2d80 x1874663043364480/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829823 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829798.433723:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829798.433726:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829798.433729:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043364480:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829798.496928:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829799.520898:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829800.544991:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829801.568874:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829802.592948:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829803.552914:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829803.552924:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829803.552929:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829803.552940:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11870 00000100:00080000:3.0:1787829803.552952:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829803.552959:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829803.553004:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634a80 x1874663043364608/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829803.553034:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829803.553037:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829803.553039:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829803.553042:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11870 00000100:00080000:3.0:1787829803.553044:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829803.553047:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829803.553056:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829803.553069:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634000 x1874663043364736/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829803.553076:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829803.553079:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829803.553080:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829803.553083:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11870 00000100:00080000:3.0:1787829803.553085:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829803.553088:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829803.553093:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829803.553102:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637b80 x1874663043364864/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829803.553111:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043364608:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787829803.553188:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829803.553223:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829803/real 1787829803] req@ffff9181c5634a80 x1874663043364608/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829828 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829803.553243:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634a80 x1874663043364608/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829828 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829803.553252:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829803.553256:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829803.553274:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043364608:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829803.553281:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043364736:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829803.553300:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829803.553311:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829803/real 1787829803] req@ffff9181c5634000 x1874663043364736/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829828 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829803.553321:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634000 x1874663043364736/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829828 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829803.553328:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829803.553332:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829803.553335:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043364736:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829803.553355:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043364864:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829803.553368:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829803.553375:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829803/real 1787829803] req@ffff9181c5637b80 x1874663043364864/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829828 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829803.553383:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637b80 x1874663043364864/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829828 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829803.553393:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829803.553396:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829803.553398:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043364864:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829803.616857:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829804.640894:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829805.664850:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829806.688897:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829807.712889:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829808.672891:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829808.672912:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829808.672917:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829808.672930:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11875 00000100:00080000:1.0:1787829808.672943:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829808.672951:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787829808.673004:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2a00 x1874663043364992/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829808.673032:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829808.673035:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829808.673037:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829808.673040:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11875 00000100:00080000:1.0:1787829808.673043:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829808.673046:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829808.673055:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829808.673066:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3480 x1874663043365120/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829808.673074:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829808.673083:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829808.673086:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829808.673090:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11875 00000100:00080000:1.0:1787829808.673095:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829808.673099:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829808.673104:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829808.673113:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b2680 x1874663043365248/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829808.673134:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043364992:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787829808.673257:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829808.673325:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829808/real 1787829808] req@ffff9181c35b2a00 x1874663043364992/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829833 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829808.673358:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2a00 x1874663043364992/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829833 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829808.673373:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829808.673380:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829808.673408:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043364992:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829808.673426:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043365120:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829808.673483:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829808.673515:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829808/real 1787829808] req@ffff9181c35b3480 x1874663043365120/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829833 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829808.673542:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3480 x1874663043365120/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829833 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829808.673574:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829808.673582:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829808.673589:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043365120:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829808.673629:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043365248:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829808.673661:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829808.673676:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829808/real 1787829808] req@ffff9181c35b2680 x1874663043365248/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829833 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829808.673692:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b2680 x1874663043365248/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829833 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829808.673727:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829808.673735:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829808.673740:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043365248:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829808.736987:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829809.760931:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829810.784973:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829811.808944:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829812.832908:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829813.792887:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829813.792898:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829813.792903:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 10000000:01000000:0.0:1787829813.792910:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'params' failed (-108). 00000100:00080000:2.0:1787829813.792915:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11880 10000000:00020000:0.0:1787829813.792916:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:0.0:1787829813.792923:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:0.0:1787829813.792926:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff9181c87b19b8 from 1 00000100:00080000:2.0:1787829813.792928:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 10000000:01000000:0.0:1787829813.792930:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00000100:00080000:2.0:1787829813.792939:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787829813.792972:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f800 x1874663043365376/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:0.0:1787829813.792996:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512c00/0xbef42c0221d6f943 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787829813.793012:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:0.0:1787829813.793020:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e0512c00/0xbef42c0221d6f943 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787829813.793021:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829813.793024:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829813.793032:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11880 00000100:00080000:2.0:1787829813.793037:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829813.793040:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:0.0:1787829813.793045:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e0512c00/0xbef42c0221d6f943 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000008:00100000:2.0:1787829813.793050:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829813.793063:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d880 x1874663043365632/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1787829813.793066:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181c7749e40 going to sleep for 0 seconds 00000100:00080000:2.0:1787829813.793076:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787829813.793077:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043365376:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787829813.793080:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043365504:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00080000:2.0:1787829813.793084:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829813.793090:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829813.793095:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11880 00000100:00080000:2.0:1787829813.793098:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829813.793101:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:0.0:1787829813.793103:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e0512c00/0xbef42c0221d6f943 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000008:00100000:2.0:1787829813.793104:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829813.793116:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ce00 x1874663043365760/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:0.0:1787829813.793119:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e0512c00/0xbef42c0221d6f943 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787829813.793137:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e0512c00/0xbef42c0221d6f943 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787829813.793147:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e0512c00/0xbef42c0221d6f943 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787829813.793155:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e0512c00/0xbef42c0221d6f943 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787829813.793160:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e0512c00 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:0.0:1787829813.793163:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512c00/0xbef42c0221d6f943 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787829813.793170:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e0512c00/0xbef42c0221d6f943 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000800:00000100:3.0:1787829813.793177:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 10000000:00010000:0.0:1787829813.793180:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e0512c00/0xbef42c0221d6f943 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1787829813.793186:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:0.0:1787829813.793190:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1787829813.793197:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e0512c00/0xbef42c0221d6f943 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1787829813.793203:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e0512c00/0xbef42c0221d6f943 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787829813.793228:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829813/real 1787829813] req@ffff9181e007f800 x1874663043365376/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829838 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 10000000:01000000:0.0:1787829813.793236:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:0.0:1787829813.793242:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:CONNECTING 00000100:00100000:3.0:1787829813.793245:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f800 x1874663043365376/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829838 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829813.793255:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829813.793267:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829813.793274:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043365376:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829813.793292:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043365632:192.168.203.147@tcp:38:kworker.0 00000100:00080000:2.0:1787829813.793301:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829813.793307:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829813.793310:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000800:00000100:3.0:1787829813.793313:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1787829813.793319:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11885 00000100:00080000:2.0:1787829813.793327:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829813.793331:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829813.793335:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829813/real 1787829813] req@ffff9181e007d880 x1874663043365632/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829838 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829813.793345:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f480 x1874663043365888/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829813.793348:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d880 x1874663043365632/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829838 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829813.793372:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829813.793376:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829813.793380:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043365632:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829813.793385:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043365760:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829813.793400:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829813.793407:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829813/real 1787829813] req@ffff9181e007ce00 x1874663043365760/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829838 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829813.793415:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ce00 x1874663043365760/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829838 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829813.793420:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829813.793422:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829813.793424:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043365760:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829813.793433:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043365888:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1787829813.793442:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829813.793448:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829813/real 1787829813] req@ffff9181e007f480 x1874663043365888/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829838 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829813.793455:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f480 x1874663043365888/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829838 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829813.793461:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829813.793463:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829813.793471:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043365888:192.168.203.147@tcp:250:kworker.0 00010000:00010000:2.0:1787829813.856942:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829814.880954:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829815.904958:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829816.928955:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829817.952921:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829818.912885:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829818.912896:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829818.912900:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829818.912913:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11885 00000100:00080000:2.0:1787829818.912926:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829818.912936:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787829818.912995:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f100 x1874663043366016/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829818.913026:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829818.913030:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829818.913031:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829818.913037:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11885 00000100:00080000:2.0:1787829818.913040:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829818.913045:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829818.913056:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829818.913070:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ea00 x1874663043366144/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829818.913081:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829818.913085:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829818.913088:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829818.913092:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11885 00000100:00080000:2.0:1787829818.913097:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829818.913104:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829818.913110:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829818.913117:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043366016:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829818.913126:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c700 x1874663043366272/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829818.913206:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829818.913246:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829818/real 1787829818] req@ffff9181e007f100 x1874663043366016/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829843 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829818.913260:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f100 x1874663043366016/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829843 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829818.913269:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829818.913272:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829818.913290:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043366016:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829818.913299:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043366144:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829818.913327:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829818.913341:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829818/real 1787829818] req@ffff9181e007ea00 x1874663043366144/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829843 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829818.913353:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ea00 x1874663043366144/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829843 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829818.913362:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829818.913367:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829818.913371:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043366144:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829818.913400:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043366272:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829818.913419:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829818.913428:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829818/real 1787829818] req@ffff9181e007c700 x1874663043366272/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829843 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829818.913436:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c700 x1874663043366272/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829843 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829818.913441:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829818.913444:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829818.913446:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043366272:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829818.976886:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829820.000933:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829821.024881:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829822.048872:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829823.072865:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829824.032842:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829824.032850:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829824.032854:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829824.032861:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11890 00000100:00080000:2.0:1787829824.032870:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829824.032875:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787829824.032903:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e300 x1874663043366400/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829824.032923:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829824.032925:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829824.032927:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829824.032929:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11890 00000100:00080000:2.0:1787829824.032931:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829824.032934:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829824.032940:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829824.032946:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d180 x1874663043366528/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829824.032953:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829824.032955:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829824.032956:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829824.032959:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11890 00000100:00080000:2.0:1787829824.032961:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829824.032964:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829824.032967:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829824.032972:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ed80 x1874663043366656/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829824.032980:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043366400:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1787829824.033037:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829824.033070:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829824/real 1787829824] req@ffff9181e007e300 x1874663043366400/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829849 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829824.033082:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e300 x1874663043366400/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829849 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829824.033089:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829824.033092:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829824.033104:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043366400:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829824.033115:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043366528:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829824.033138:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829824.033154:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829824/real 1787829824] req@ffff9181e007d180 x1874663043366528/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829849 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829824.033165:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d180 x1874663043366528/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829849 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829824.033169:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829824.033171:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829824.033173:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043366528:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829824.033191:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043366656:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829824.033203:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829824.033209:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829824/real 1787829824] req@ffff9181e007ed80 x1874663043366656/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829849 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829824.033216:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ed80 x1874663043366656/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829849 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829824.033220:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829824.033222:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829824.033223:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043366656:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787829824.096866:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829825.120881:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829826.144904:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829827.168868:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829828.192933:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829829.152856:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829829.152863:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829829.152866:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829829.152874:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11895 00000100:00080000:2.0:1787829829.152882:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829829.152887:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787829829.152931:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007df80 x1874663043366784/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829829.152956:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829829.152959:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829829.152961:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829829.152965:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11895 00000100:00080000:2.0:1787829829.152968:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829829.152971:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829829.152980:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829829.152990:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d500 x1874663043366912/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829829.152999:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829829.153002:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829829.153003:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829829.153006:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11895 00000100:00080000:2.0:1787829829.153008:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829829.153010:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829829.153014:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829829.153020:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ed80 x1874663043367040/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829829.153032:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043366784:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1787829829.153117:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829829.153155:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829829/real 1787829829] req@ffff9181e007df80 x1874663043366784/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829854 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829829.153169:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007df80 x1874663043366784/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829854 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829829.153179:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829829.153183:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829829.153207:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043366784:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829829.153216:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043366912:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829829.153246:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829829.153261:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829829/real 1787829829] req@ffff9181e007d500 x1874663043366912/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829854 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829829.153269:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d500 x1874663043366912/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829854 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829829.153275:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829829.153277:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829829.153279:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043366912:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829829.153300:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043367040:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829829.153317:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829829.153327:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829829/real 1787829829] req@ffff9181e007ed80 x1874663043367040/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829854 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829829.153339:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ed80 x1874663043367040/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829854 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829829.153346:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829829.153349:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829829.153352:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043367040:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829829.216923:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829830.240901:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829831.264841:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829832.288849:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829833.312851:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829834.272808:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829834.272816:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829834.272819:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829834.272828:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11900 00000100:00080000:3.0:1787829834.272836:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829834.272841:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 10000000:01000000:0.0:1787829834.272860:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-sptlrpc' failed (-108). 10000000:00020000:0.0:1787829834.272866:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00100000:3.0:1787829834.272867:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043367168/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829834.272886:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829834.272889:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829834.272890:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829834.272894:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11900 00000100:00080000:3.0:1787829834.272897:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829834.272899:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829834.272906:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829834.272914:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043367296/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829834.272923:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043367168:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787829834.272923:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829834.272928:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829834.272931:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829834.272935:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11900 00000100:00080000:3.0:1787829834.272937:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829834.272940:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829834.272943:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829834.272951:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043367424/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829834.273003:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829834.273039:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829834/real 1787829834] req@ffff9181c5635c00 x1874663043367168/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829859 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829834.273059:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043367168/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829859 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829834.273071:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829834.273075:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829834.273087:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043367168:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829834.273111:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043367296:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829834.273137:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829834.273150:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829834/real 1787829834] req@ffff9181c5636a00 x1874663043367296/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829859 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829834.273161:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043367296/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829859 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829834.273169:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829834.273172:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829834.273174:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043367296:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829834.273179:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043367424:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829834.273198:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829834.273209:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829834/real 1787829834] req@ffff9181c5637800 x1874663043367424/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829859 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829834.273223:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043367424/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829859 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829834.273232:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829834.273236:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829834.273239:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043367424:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829834.336838:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829835.360849:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787829836.064895:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:1.0:1787829836.064904:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff9181c5620000 from 341 10000000:01000000:1.0:1787829836.064909:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:1.0:1787829836.064965:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f94a lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829836.064975:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f94a lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829836.064986:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f94a lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787829836.065011:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215d180 going to sleep for 0 seconds 00000100:00100000:1.0:1787829836.065024:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043367552:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787829836.065039:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f94a lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829836.065044:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f94a lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829836.065049:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f94a lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829836.065054:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f94a lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829836.065058:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f94a lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829836.065062:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b5800 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787829836.065065:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f94a lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829836.065070:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f94a lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787829836.065076:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f94a lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787829836.065082:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:1.0:1787829836.065085:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787829836.065093:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f94a lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829836.065100:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f94a lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787829836.065123:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787829836.065127:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:2.0:1787829836.065199:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829836.065208:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829836.065212:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829836.065221:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11906 00000100:00080000:2.0:1787829836.065231:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829836.065238:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787829836.065268:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d180 x1874663043367680/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829836.065293:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829836.065295:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829836.065296:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829836.065299:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11906 00000100:00080000:2.0:1787829836.065302:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829836.065304:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829836.065311:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829836.065320:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e300 x1874663043367808/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829836.065327:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829836.065329:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829836.065330:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829836.065332:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11906 00000100:00080000:2.0:1787829836.065333:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829836.065335:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829836.065338:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829836.065343:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c700 x1874663043367936/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829836.065354:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043367680:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1787829836.065425:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829836.065458:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829836/real 1787829836] req@ffff9181e007d180 x1874663043367680/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829861 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829836.065471:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d180 x1874663043367680/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829861 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829836.065479:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829836.065482:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829836.065492:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043367680:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829836.065497:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043367808:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829836.065526:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829836.065537:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829836/real 1787829836] req@ffff9181e007e300 x1874663043367808/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829861 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829836.065545:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e300 x1874663043367808/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829861 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829836.065550:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829836.065552:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829836.065555:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043367808:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829836.065572:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043367936:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829836.065585:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829836.065591:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829836/real 1787829836] req@ffff9181e007c700 x1874663043367936/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829861 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829836.065603:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c700 x1874663043367936/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829861 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829836.065610:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829836.065613:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829836.065616:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043367936:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829836.384894:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829837.408895:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829838.432885:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829839.456904:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829840.480854:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829841.440875:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829841.440884:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829841.440888:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829841.440896:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11907 00000100:00080000:3.0:1787829841.440905:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829841.440911:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829841.440942:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043368064/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829841.440967:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829841.440970:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829841.440972:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829841.440975:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11907 00000100:00080000:3.0:1787829841.440978:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829841.440982:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829841.440989:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829841.441003:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043368192/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829841.441007:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043368064:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787829841.441025:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829841.441029:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829841.441031:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829841.441036:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11907 00000100:00080000:3.0:1787829841.441041:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829841.441046:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829841.441052:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829841.441068:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043368320/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829841.441122:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043368192:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829841.441157:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043368320:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829841.504900:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829842.528861:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829843.552905:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1787829844.513032:0:2377:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:0.0:1787829844.513078:0:2377:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1787829844.513096:0:2377:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1787829844.513153:0:2377:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1787829844.513168:0:2377:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:2.0:1787829844.513195:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829841/real 1787829844] req@ffff9181c5634380 x1874663043368192/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829866 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829844.513224:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043368192/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829866 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829844.513237:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829844.513243:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829844.513248:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043368192:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829844.513257:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829841/real 1787829844] req@ffff9181c5635500 x1874663043368320/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829866 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829844.513270:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043368320/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829866 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829844.513279:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829844.513283:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829844.513287:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043368320:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829844.513295:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829841/real 1787829844] req@ffff9181c5636d80 x1874663043368064/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829866 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829844.513308:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043368064/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829866 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829844.513316:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829844.513320:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829844.513348:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043368064:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787829844.576848:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829845.600905:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829846.560868:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829846.560878:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829846.560882:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829846.560892:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11913 00000100:00080000:2.0:1787829846.560902:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829846.560910:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787829846.560943:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ea00 x1874663043368448/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829846.560975:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829846.560979:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829846.560981:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829846.560985:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11913 00000100:00080000:2.0:1787829846.560988:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829846.560992:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829846.561000:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829846.561013:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f100 x1874663043368576/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829846.561025:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787829846.561026:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043368448:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787829846.561032:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829846.561037:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829846.561041:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11913 00000100:00080000:2.0:1787829846.561044:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829846.561049:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829846.561054:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829846.561066:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f480 x1874663043368704/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829846.561102:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829846.561138:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829846/real 1787829846] req@ffff9181e007ea00 x1874663043368448/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829871 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829846.561157:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ea00 x1874663043368448/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829871 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829846.561168:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829846.561173:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829846.561197:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043368448:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829846.561223:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043368576:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829846.561248:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829846.561263:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829846/real 1787829846] req@ffff9181e007f100 x1874663043368576/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829871 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829846.561278:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f100 x1874663043368576/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829871 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829846.561288:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829846.561292:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829846.561296:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043368576:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829846.561303:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043368704:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829846.561323:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829846.561335:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829846/real 1787829846] req@ffff9181e007f480 x1874663043368704/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829871 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829846.561349:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f480 x1874663043368704/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829871 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829846.561358:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829846.561362:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829846.561365:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043368704:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829846.624893:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829847.648889:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829848.672924:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829849.696957:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829850.720906:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829851.680853:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829851.680866:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829851.680871:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829851.680883:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11918 00000100:00080000:3.0:1787829851.680897:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829851.680905:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829851.681085:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043368832/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829851.681127:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829851.681133:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829851.681136:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829851.681144:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11918 00000100:00080000:3.0:1787829851.681153:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829851.681160:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829851.681316:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829851.681336:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043368960/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829851.681347:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829851.681351:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829851.681353:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829851.681357:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11918 00000100:00080000:3.0:1787829851.681363:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829851.681368:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829851.681372:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829851.681384:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635f80 x1874663043369088/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829851.681416:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043368832:192.168.203.147@tcp:250:kworker.0 00000800:00000100:2.0:1787829851.681472:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829851.681502:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829851/real 1787829851] req@ffff9181c5637100 x1874663043368832/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829876 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829851.681516:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043368832/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829876 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829851.681523:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829851.681526:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829851.681541:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043368832:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829851.681560:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043368960:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829851.681590:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829851.681603:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829851/real 1787829851] req@ffff9181c5637480 x1874663043368960/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829876 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829851.681613:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043368960/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829876 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829851.681619:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829851.681622:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829851.681627:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043368960:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829851.681646:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043369088:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829851.681659:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829851.681665:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829851/real 1787829851] req@ffff9181c5635f80 x1874663043369088/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829876 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829851.681673:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635f80 x1874663043369088/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829876 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829851.681678:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829851.681681:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829851.681683:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043369088:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829851.744932:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829852.768865:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829853.792917:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829854.816929:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829855.840909:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787829856.288879:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-cliir' failed (-108). 10000000:00000400:1.0:1787829856.288888:0:159735:0:(mgc_request.c:1900:mgc_process_log()) MGC192.168.203.147@tcp: IR log lustre-cliir failed, not fatal: rc = -108 10000000:01000000:1.0:1787829856.288896:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:1.0:1787829856.288899:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff9181c5620000 from 52 10000000:01000000:1.0:1787829856.288903:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:1.0:1787829856.288946:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f951 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829856.288958:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f951 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829856.288970:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f951 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787829856.288989:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215d180 going to sleep for 0 seconds 00000100:00100000:1.0:1787829856.289001:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043369216:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787829856.289013:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f951 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829856.289021:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f951 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829856.289029:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f951 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829856.289037:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f951 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829856.289050:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f951 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829856.289057:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b5800 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787829856.289061:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f951 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829856.289069:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f951 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787829856.289076:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f951 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787829856.289083:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1787829856.289086:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787829856.289092:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f951 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829856.289099:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f951 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787829856.289132:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787829856.289135:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:2.0:1787829856.289208:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829856.289213:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829856.289216:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829856.289222:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11923 00000100:00080000:2.0:1787829856.289230:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829856.289235:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787829856.289260:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ce00 x1874663043369344/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829856.289286:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829856.289289:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829856.289291:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829856.289294:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11923 00000100:00080000:2.0:1787829856.289297:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829856.289301:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829856.289309:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829856.289319:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d880 x1874663043369472/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829856.289327:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1787829856.289329:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043369344:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787829856.289331:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829856.289335:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829856.289340:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11923 00000100:00080000:2.0:1787829856.289342:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829856.289344:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829856.289348:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829856.289356:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f800 x1874663043369600/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829856.289401:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829856.289426:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829856/real 1787829856] req@ffff9181e007ce00 x1874663043369344/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829881 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829856.289443:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ce00 x1874663043369344/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829881 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829856.289455:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829856.289459:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829856.289470:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043369344:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829856.289484:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043369472:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829856.289509:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829856.289522:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829856/real 1787829856] req@ffff9181e007d880 x1874663043369472/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829881 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829856.289535:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d880 x1874663043369472/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829881 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829856.289543:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829856.289546:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829856.289550:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043369472:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829856.289556:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043369600:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829856.289577:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829856.289585:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829856/real 1787829856] req@ffff9181e007f800 x1874663043369600/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829881 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829856.289594:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f800 x1874663043369600/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829881 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829856.289604:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829856.289606:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829856.289608:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043369600:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829856.864874:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829857.888902:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829858.912914:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829859.936890:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829860.960908:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829861.408887:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829861.408904:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829861.408908:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829861.408918:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11928 00000100:00080000:3.0:1787829861.408929:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829861.408936:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829861.408972:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634e00 x1874663043369728/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829861.408999:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829861.409003:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829861.409005:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829861.409009:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11928 00000100:00080000:3.0:1787829861.409014:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829861.409019:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829861.409036:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829861.409042:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043369728:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829861.409055:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635f80 x1874663043369856/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829861.409067:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829861.409071:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829861.409073:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829861.409084:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11928 00000100:00080000:3.0:1787829861.409095:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829861.409099:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829861.409105:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000800:00000100:2.0:1787829861.409106:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829861.409119:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637480 x1874663043369984/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1787829861.409151:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829861/real 1787829861] req@ffff9181c5634e00 x1874663043369728/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829886 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829861.409185:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634e00 x1874663043369728/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829886 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829861.409196:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829861.409202:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829861.409219:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043369728:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829861.409249:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043369856:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829861.409278:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829861.409292:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829861/real 1787829861] req@ffff9181c5635f80 x1874663043369856/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829886 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829861.409306:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635f80 x1874663043369856/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829886 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829861.409315:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829861.409319:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829861.409323:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043369856:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829861.409330:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043369984:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829861.409367:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829861.409380:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829861/real 1787829861] req@ffff9181c5637480 x1874663043369984/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829886 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829861.409393:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637480 x1874663043369984/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829886 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829861.409401:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829861.409405:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829861.409409:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043369984:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829861.984868:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829863.008897:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829864.032888:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829865.056903:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829866.080896:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829866.528873:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829866.528883:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829866.528888:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829866.528899:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11933 00000100:00080000:2.0:1787829866.528910:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829866.528916:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787829866.528954:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ca80 x1874663043370112/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829866.528981:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829866.528984:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829866.528986:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829866.528998:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11933 00000100:00080000:2.0:1787829866.529002:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829866.529007:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829866.529017:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829866.529023:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043370112:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829866.529032:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007fb80 x1874663043370240/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829866.529045:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829866.529048:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829866.529050:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829866.529054:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11933 00000100:00080000:2.0:1787829866.529058:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829866.529062:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829866.529067:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829866.529079:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e680 x1874663043370368/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829866.529102:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829866.529134:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829866/real 1787829866] req@ffff9181e007ca80 x1874663043370112/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829891 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829866.529148:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ca80 x1874663043370112/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829891 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829866.529155:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829866.529159:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829866.529170:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043370112:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829866.529190:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043370240:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829866.529209:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829866.529221:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829866/real 1787829866] req@ffff9181e007fb80 x1874663043370240/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829891 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829866.529229:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007fb80 x1874663043370240/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829891 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829866.529240:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829866.529242:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829866.529244:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043370240:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829866.529248:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043370368:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829866.529260:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829866.529266:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829866/real 1787829866] req@ffff9181e007e680 x1874663043370368/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829891 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829866.529273:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e680 x1874663043370368/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829891 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829866.529278:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829866.529280:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829866.529282:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043370368:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829867.104891:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829868.128900:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829869.152905:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829870.176889:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829871.200872:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829871.648857:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829871.648865:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829871.648868:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829871.648876:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11938 00000100:00080000:0.0:1787829871.648885:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829871.648891:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829871.648923:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5180 x1874663043370496/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829871.648944:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829871.648947:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829871.648948:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829871.648951:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11938 00000100:00080000:0.0:1787829871.648953:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829871.648956:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829871.648963:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829871.648971:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5c00 x1874663043370624/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829871.648979:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829871.648982:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829871.648983:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829871.648986:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11938 00000100:00080000:0.0:1787829871.648988:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829871.648990:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829871.648993:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829871.649000:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4700 x1874663043370752/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829871.649007:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043370496:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1787829871.649078:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829871.649117:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829871/real 1787829871] req@ffff9181c55a5180 x1874663043370496/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829896 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829871.649136:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5180 x1874663043370496/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829896 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829871.649147:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829871.649152:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829871.649168:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043370496:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829871.649177:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043370624:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829871.649207:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829871.649223:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829871/real 1787829871] req@ffff9181c55a5c00 x1874663043370624/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829896 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829871.649236:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5c00 x1874663043370624/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829896 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829871.649245:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829871.649249:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829871.649252:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043370624:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829871.649291:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043370752:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829871.649314:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829871.649326:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829871/real 1787829871] req@ffff9181c55a4700 x1874663043370752/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829896 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829871.649339:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4700 x1874663043370752/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829896 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829871.649347:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829871.649351:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829871.649354:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043370752:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829872.224893:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829873.248913:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829874.272893:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829875.296904:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829876.320870:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829876.768852:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829876.768862:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829876.768866:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829876.768876:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11943 00000100:00080000:0.0:1787829876.768887:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 10000000:01000000:1.0:1787829876.768890:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-client' failed (-108). 00000100:00080000:0.0:1787829876.768893:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 10000000:00020000:1.0:1787829876.768896:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:1.0:1787829876.768900:0:159735:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:1.0:1787829876.768902:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff9181c5620000 from 7 10000000:01000000:1.0:1787829876.768905:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00000100:00100000:0.0:1787829876.768923:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5880 x1874663043370880/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787829876.768937:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f958 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829876.768947:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f958 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787829876.768950:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829876.768954:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829876.768955:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00010000:00010000:1.0:1787829876.768959:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f958 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787829876.768961:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11943 00000100:00080000:0.0:1787829876.768965:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829876.768970:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787829876.768978:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215d180 going to sleep for 0 seconds 00000008:00100000:0.0:1787829876.768979:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829876.768986:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043371008:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00100000:0.0:1787829876.768991:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4380 x1874663043371136/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829876.769001:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043370880:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787829876.769002:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f958 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1787829876.769003:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829876.769013:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829876.769016:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829876.769021:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11943 00000100:00080000:0.0:1787829876.769024:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829876.769029:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:1.0:1787829876.769030:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f958 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000008:00100000:0.0:1787829876.769035:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:1.0:1787829876.769038:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f958 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829876.769046:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f958 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1787829876.769048:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5f80 x1874663043371264/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787829876.769056:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f958 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829876.769063:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b5800 already unused, calling callback (ffffffffc1273c20) 00000800:00000100:3.0:1787829876.769068:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 10000000:00010000:1.0:1787829876.769070:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f958 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829876.769081:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f958 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787829876.769093:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f958 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787829876.769100:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829876/real 1787829876] req@ffff9181c55a5880 x1874663043370880/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829901 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 10000000:01000000:1.0:1787829876.769102:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:1.0:1787829876.769111:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787829876.769119:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f958 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829876.769126:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f958 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787829876.769129:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5880 x1874663043370880/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829901 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829876.769155:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829876.769161:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829876.769166:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043370880:192.168.203.147@tcp:250:kworker.0 10000000:01000000:1.0:1787829876.769167:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787829876.769171:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00100000:3.0:1787829876.769193:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043371136:192.168.203.147@tcp:38:kworker.0 00000100:00080000:0.0:1787829876.769212:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829876.769216:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829876.769218:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829876.769222:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11948 00000100:00080000:0.0:1787829876.769238:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000800:00000100:3.0:1787829876.769243:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:0.0:1787829876.769245:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829876.769265:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829876/real 1787829876] req@ffff9181c55a4380 x1874663043371136/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829901 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787829876.769266:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6d80 x1874663043371392/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829876.769288:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4380 x1874663043371136/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829901 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829876.769311:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829876.769315:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829876.769318:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043371136:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829876.769324:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043371264:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829876.769343:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829876.769352:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829876/real 1787829876] req@ffff9181c55a5f80 x1874663043371264/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829901 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829876.769360:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5f80 x1874663043371264/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829901 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829876.769371:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829876.769373:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829876.769375:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043371264:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829876.769385:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043371392:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1787829876.769395:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829876.769401:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829876/real 1787829876] req@ffff9181c55a6d80 x1874663043371392/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829901 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829876.769409:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6d80 x1874663043371392/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829901 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829876.769417:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829876.769420:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829876.769433:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043371392:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787829877.344897:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829878.368903:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829879.392918:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829880.416898:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829881.440988:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829881.888943:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829881.888953:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829881.888957:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829881.888965:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11948 00000100:00080000:2.0:1787829881.888987:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829881.889003:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787829881.889052:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007dc00 x1874663043371520/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829881.889090:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829881.889094:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829881.889096:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829881.889101:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11948 00000100:00080000:2.0:1787829881.889104:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829881.889114:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829881.889124:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829881.889136:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043371520:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829881.889138:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c000 x1874663043371648/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829881.889153:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829881.889155:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829881.889156:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829881.889159:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11948 00000100:00080000:2.0:1787829881.889161:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829881.889164:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829881.889168:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829881.889178:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c380 x1874663043371776/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829881.889228:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829881.889265:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829881/real 1787829881] req@ffff9181e007dc00 x1874663043371520/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829906 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829881.889285:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007dc00 x1874663043371520/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829906 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829881.889297:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829881.889301:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829881.889316:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043371520:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829881.889344:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043371648:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829881.889374:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829881.889389:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829881/real 1787829881] req@ffff9181e007c000 x1874663043371648/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829906 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829881.889404:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c000 x1874663043371648/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829906 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829881.889413:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829881.889417:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829881.889420:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043371648:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829881.889427:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043371776:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829881.889449:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829881.889461:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829881/real 1787829881] req@ffff9181e007c380 x1874663043371776/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829906 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829881.889474:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c380 x1874663043371776/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829906 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829881.889483:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829881.889487:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829881.889491:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043371776:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829882.464900:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829883.488882:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829884.512913:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829885.536926:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829886.560918:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829887.008852:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829887.008860:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829887.008864:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829887.008872:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11953 00000100:00080000:2.0:1787829887.008882:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829887.008888:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787829887.008927:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c380 x1874663043371904/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829887.008955:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829887.008959:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829887.008961:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829887.008966:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11953 00000100:00080000:2.0:1787829887.008970:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829887.008974:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829887.008983:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829887.008998:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c000 x1874663043372032/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829887.009007:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829887.009010:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829887.009011:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829887.009020:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11953 00000100:00100000:3.0:1787829887.009022:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043371904:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787829887.009023:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829887.009030:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829887.009036:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829887.009046:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007dc00 x1874663043372160/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829887.009094:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829887.009130:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829887/real 1787829887] req@ffff9181e007c380 x1874663043371904/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829912 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829887.009151:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c380 x1874663043371904/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829912 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829887.009175:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829887.009179:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829887.009195:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043371904:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829887.009204:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043372032:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829887.009233:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829887.009249:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829887/real 1787829887] req@ffff9181e007c000 x1874663043372032/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829912 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829887.009263:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c000 x1874663043372032/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829912 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829887.009271:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829887.009275:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829887.009279:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043372032:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829887.009303:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043372160:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829887.009324:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829887.009335:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829887/real 1787829887] req@ffff9181e007dc00 x1874663043372160/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829912 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829887.009349:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007dc00 x1874663043372160/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829912 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829887.009372:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829887.009376:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829887.009380:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043372160:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829887.584914:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829888.608906:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829889.632894:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829890.656895:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829891.680924:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1787829892.128880:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829892.128888:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829892.128891:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829892.128899:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11958 00000100:00080000:3.0:1787829892.128908:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829892.128914:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829892.128946:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637100 x1874663043372288/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829892.128968:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829892.128971:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829892.128972:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829892.128975:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11958 00000100:00080000:3.0:1787829892.128977:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829892.128980:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829892.128986:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829892.128995:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636d80 x1874663043372416/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1787829892.129003:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829892.129006:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829892.129008:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829892.129013:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11958 00000100:00080000:3.0:1787829892.129016:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1787829892.129017:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043372288:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787829892.129021:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829892.129028:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829892.129037:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635500 x1874663043372544/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1787829892.129091:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829892.129142:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829892/real 1787829892] req@ffff9181c5637100 x1874663043372288/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829917 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829892.129161:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637100 x1874663043372288/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829917 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829892.129172:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829892.129176:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829892.129193:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043372288:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829892.129202:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043372416:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829892.129230:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829892.129250:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829892/real 1787829892] req@ffff9181c5636d80 x1874663043372416/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829917 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829892.129266:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636d80 x1874663043372416/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829917 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829892.129276:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829892.129280:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829892.129284:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043372416:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829892.129309:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043372544:192.168.203.147@tcp:38:kworker.0 00000800:00000100:2.0:1787829892.129330:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829892.129342:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829892/real 1787829892] req@ffff9181c5635500 x1874663043372544/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829917 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829892.129356:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635500 x1874663043372544/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829917 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829892.129365:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1787829892.129369:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:2.0:1787829892.129372:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043372544:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829892.704906:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829893.728980:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829894.752907:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829895.776944:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829896.800898:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829897.248888:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829897.248899:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829897.248903:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829897.248912:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11963 10000000:01000000:1.0:1787829897.248914:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'params' failed (-108). 10000000:00020000:1.0:1787829897.248920:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00080000:2.0:1787829897.248924:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 10000000:01000000:1.0:1787829897.248926:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:1.0:1787829897.248930:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff9181c87b19b8 from 1 00000100:00080000:2.0:1787829897.248935:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 10000000:01000000:1.0:1787829897.248936:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00000100:00100000:2.0:1787829897.248973:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e680 x1874663043372800/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787829897.248983:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f95f lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829897.248993:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f95f lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787829897.248997:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829897.249002:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829897.249004:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00010000:00010000:1.0:1787829897.249005:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f95f lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787829897.249009:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11963 00000100:00080000:2.0:1787829897.249016:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829897.249023:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787829897.249027:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215dc00 going to sleep for 0 seconds 00000008:00100000:2.0:1787829897.249032:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829897.249036:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043372672:192.168.203.147@tcp:101:ll_cfg_requeue.0 00000100:00100000:3.0:1787829897.249042:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043372800:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829897.249048:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007fb80 x1874663043372928/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:1.0:1787829897.249051:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f95f lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829897.249068:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f95f lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787829897.249075:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00010000:00010000:1.0:1787829897.249076:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f95f lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787829897.249101:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00010000:00010000:1.0:1787829897.249102:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f95f lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787829897.249107:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00010000:00010000:1.0:1787829897.249111:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f95f lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787829897.249116:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11963 00010000:00010000:1.0:1787829897.249119:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b5800 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787829897.249124:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f95f lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787829897.249126:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000800:00000100:3.0:1787829897.249130:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1787829897.249136:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00010000:00010000:1.0:1787829897.249140:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f95f lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000008:00100000:2.0:1787829897.249142:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 10000000:00010000:1.0:1787829897.249155:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f95f lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787829897.249158:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ca80 x1874663043373056/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829897.249171:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829897/real 1787829897] req@ffff9181e007e680 x1874663043372800/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829922 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 10000000:01000000:1.0:1787829897.249174:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1787829897.249184:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787829897.249195:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f95f lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829897.249202:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f95f lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787829897.249205:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e680 x1874663043372800/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829922 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829897.249221:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 10000000:01000000:1.0:1787829897.249223:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787829897.249227:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:3.0:1787829897.249228:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829897.249236:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043372800:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829897.249261:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043372928:192.168.203.147@tcp:38:kworker.0 00000100:00080000:2.0:1787829897.249268:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829897.249274:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829897.249276:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829897.249282:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11968 00000100:00080000:2.0:1787829897.249287:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000800:00000100:3.0:1787829897.249293:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1787829897.249295:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829897.249312:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829897/real 1787829897] req@ffff9181e007fb80 x1874663043372928/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829922 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1787829897.249317:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f800 x1874663043373184/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829897.249349:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007fb80 x1874663043372928/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829922 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829897.249366:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829897.249370:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829897.249374:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043372928:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829897.249381:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043373056:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829897.249403:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829897.249415:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829897/real 1787829897] req@ffff9181e007ca80 x1874663043373056/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829922 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829897.249429:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ca80 x1874663043373056/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829922 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829897.249438:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829897.249442:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829897.249446:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043373056:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829897.249462:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043373184:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1787829897.249489:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829897.249502:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829897/real 1787829897] req@ffff9181e007f800 x1874663043373184/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829922 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829897.249516:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f800 x1874663043373184/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829922 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829897.249524:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829897.249528:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829897.249541:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043373184:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787829897.824929:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829898.848849:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829899.872902:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829900.896853:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829901.920872:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829902.368853:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829902.368860:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829902.368864:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829902.368872:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11968 00000100:00080000:0.0:1787829902.368883:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829902.368890:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829902.368923:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7100 x1874663043373312/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829902.368942:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829902.368945:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829902.368946:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829902.368949:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11968 00000100:00080000:0.0:1787829902.368952:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829902.368955:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829902.368961:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829902.368970:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7b80 x1874663043373440/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829902.368977:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829902.368979:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:3.0:1787829902.368979:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043373312:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787829902.368983:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829902.368987:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11968 00000100:00080000:0.0:1787829902.368989:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829902.368991:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829902.368995:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829902.369003:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6680 x1874663043373568/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829902.369033:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829902.369070:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829902/real 1787829902] req@ffff9181c55a7100 x1874663043373312/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829927 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829902.369083:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7100 x1874663043373312/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829927 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829902.369090:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829902.369093:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829902.369102:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043373312:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829902.369118:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043373440:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829902.369136:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829902.369146:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829902/real 1787829902] req@ffff9181c55a7b80 x1874663043373440/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829927 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829902.369153:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7b80 x1874663043373440/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829927 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829902.369158:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829902.369160:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829902.369162:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043373440:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829902.369165:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043373568:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829902.369174:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829902.369180:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829902/real 1787829902] req@ffff9181c55a6680 x1874663043373568/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829927 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829902.369186:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6680 x1874663043373568/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829927 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829902.369191:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829902.369192:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829902.369194:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043373568:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829902.944847:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1787829903.968907:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829904.992892:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829906.016856:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829907.040872:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829907.488869:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829907.488879:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829907.488882:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829907.488892:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11974 00000100:00080000:2.0:1787829907.488901:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829907.488907:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787829907.488938:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d880 x1874663043373696/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829907.488959:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829907.488962:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829907.488964:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829907.488967:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11974 00000100:00080000:2.0:1787829907.488970:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829907.488974:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829907.488981:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829907.488996:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043373696:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829907.488997:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ce00 x1874663043373824/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829907.489011:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829907.489025:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829907.489026:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829907.489030:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11974 00000100:00080000:2.0:1787829907.489033:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829907.489037:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829907.489041:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829907.489051:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f480 x1874663043373952/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829907.489099:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043373824:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829907.489119:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043373952:192.168.203.147@tcp:38:kworker.0 00010000:00010000:3.0:1787829908.064914:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829909.088932:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829910.112880:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1787829910.561014:0:2378:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:0.0:1787829910.561055:0:2378:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1787829910.561071:0:2378:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1787829910.561112:0:2378:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1787829910.561122:0:2378:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:3.0:1787829910.561188:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829907/real 1787829910] req@ffff9181e007ce00 x1874663043373824/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829932 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829910.561210:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ce00 x1874663043373824/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829932 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829910.561223:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829910.561227:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829910.561232:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043373824:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829910.561239:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829907/real 1787829910] req@ffff9181e007f480 x1874663043373952/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829932 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829910.561257:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f480 x1874663043373952/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829932 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829910.561264:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829910.561267:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829910.561270:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043373952:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829910.561276:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829907/real 1787829910] req@ffff9181e007d880 x1874663043373696/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829932 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829910.561285:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d880 x1874663043373696/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829932 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829910.561293:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829910.561296:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829910.561318:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043373696:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787829911.136908:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829912.160956:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829912.608842:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829912.608852:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829912.608855:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829912.608865:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11979 00000100:00080000:2.0:1787829912.608874:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829912.608881:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787829912.608911:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f100 x1874663043374080/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829912.608944:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829912.608948:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829912.608949:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829912.608954:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11979 00000100:00080000:2.0:1787829912.608957:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829912.608961:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829912.608968:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829912.608989:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ea00 x1874663043374208/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829912.608999:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829912.609002:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829912.609003:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829912.609007:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11979 00000100:00080000:2.0:1787829912.609010:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829912.609013:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829912.609019:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829912.609028:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c700 x1874663043374336/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829912.609123:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043374080:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1787829912.609178:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829912.609207:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829912/real 1787829912] req@ffff9181e007f100 x1874663043374080/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829937 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829912.609227:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f100 x1874663043374080/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829937 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829912.609237:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829912.609240:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829912.609262:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043374080:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829912.609271:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043374208:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829912.609294:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829912.609306:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829912/real 1787829912] req@ffff9181e007ea00 x1874663043374208/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829937 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829912.609318:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ea00 x1874663043374208/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829937 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829912.609325:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829912.609329:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829912.609332:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043374208:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829912.609338:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043374336:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829912.609355:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829912.609364:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829912/real 1787829912] req@ffff9181e007c700 x1874663043374336/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829937 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829912.609374:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c700 x1874663043374336/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829937 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829912.609381:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829912.609384:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829912.609387:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043374336:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829913.184859:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829914.208937:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829915.232876:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829916.256983:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829917.280877:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787829917.728872:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-sptlrpc' failed (-108). 10000000:00020000:1.0:1787829917.728878:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00080000:0.0:1787829917.729008:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829917.729013:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829917.729016:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829917.729022:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11984 00000100:00080000:0.0:1787829917.729030:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829917.729034:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829917.729059:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7480 x1874663043374464/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829917.729078:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829917.729082:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829917.729084:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829917.729088:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11984 00000100:00080000:0.0:1787829917.729091:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829917.729103:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829917.729113:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829917.729113:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043374464:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787829917.729132:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6680 x1874663043374592/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829917.729140:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829917.729142:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829917.729143:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829917.729146:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11984 00000100:00080000:0.0:1787829917.729149:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829917.729151:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829917.729155:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829917.729163:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7b80 x1874663043374720/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829917.729167:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829917.729195:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829917/real 1787829917] req@ffff9181c55a7480 x1874663043374464/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829942 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829917.729207:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7480 x1874663043374464/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829942 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829917.729214:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829917.729217:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829917.729219:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043374464:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829917.729233:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043374592:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829917.729246:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829917.729253:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829917/real 1787829917] req@ffff9181c55a6680 x1874663043374592/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829942 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829917.729260:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6680 x1874663043374592/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829942 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829917.729265:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829917.729267:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829917.729269:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043374592:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829917.729272:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043374720:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829917.729280:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829917.729285:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829917/real 1787829917] req@ffff9181c55a7b80 x1874663043374720/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829942 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829917.729291:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7b80 x1874663043374720/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829942 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829917.729300:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829917.729301:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829917.729303:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043374720:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829918.304878:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829919.328879:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787829919.456837:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:1.0:1787829919.456843:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff9181c5620000 from 341 10000000:01000000:1.0:1787829919.456847:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:1.0:1787829919.456886:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f966 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829919.456894:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f966 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829919.456902:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f966 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787829919.456917:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215dc00 going to sleep for 0 seconds 00000100:00100000:1.0:1787829919.456925:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043374848:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787829919.456953:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f966 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829919.456959:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f966 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829919.456963:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f966 lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829919.456968:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f966 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829919.456972:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f966 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829919.456976:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b5800 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787829919.456980:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f966 lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829919.456985:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f966 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787829919.456989:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f966 lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787829919.456992:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:1.0:1787829919.456994:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787829919.456998:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f966 lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829919.457002:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f966 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787829919.457017:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787829919.457019:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:0.0:1787829919.457072:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829919.457076:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829919.457079:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829919.457084:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11989 00000100:00080000:0.0:1787829919.457091:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829919.457095:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829919.457110:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7100 x1874663043374976/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829919.457132:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829919.457136:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829919.457138:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829919.457142:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11989 00000100:00080000:0.0:1787829919.457146:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829919.457151:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829919.457160:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829919.457173:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6d80 x1874663043375104/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829919.457175:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043374976:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787829919.457184:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829919.457187:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829919.457188:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829919.457191:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11989 00000100:00080000:0.0:1787829919.457194:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829919.457196:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829919.457199:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829919.457208:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5f80 x1874663043375232/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829919.457230:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829919.457260:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829919/real 1787829919] req@ffff9181c55a7100 x1874663043374976/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829944 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829919.457271:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7100 x1874663043374976/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829944 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829919.457278:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829919.457280:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829919.457288:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043374976:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829919.457303:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043375104:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829919.457317:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829919.457325:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829919/real 1787829919] req@ffff9181c55a6d80 x1874663043375104/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829944 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829919.457337:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6d80 x1874663043375104/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829944 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829919.457345:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829919.457348:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829919.457351:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043375104:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829919.457357:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043375232:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829919.457375:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829919.457383:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829919/real 1787829919] req@ffff9181c55a5f80 x1874663043375232/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829944 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829919.457390:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5f80 x1874663043375232/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829944 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829919.457394:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829919.457396:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829919.457398:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043375232:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829920.352869:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829921.376892:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829922.400835:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829923.424887:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829924.448867:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829924.896829:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829924.896836:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829924.896839:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829924.896847:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11991 00000100:00080000:0.0:1787829924.896855:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829924.896860:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829924.896890:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4380 x1874663043375360/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829924.896923:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829924.896926:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829924.896928:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829924.896931:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11991 00000100:00080000:0.0:1787829924.896936:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829924.896939:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829924.896947:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829924.896957:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5880 x1874663043375488/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829924.896964:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829924.896968:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829924.896969:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1787829924.896969:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043375360:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787829924.896980:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11991 00000100:00080000:0.0:1787829924.896983:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829924.896985:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829924.896988:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829924.896996:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4700 x1874663043375616/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829924.897033:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829924.897062:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829924/real 1787829924] req@ffff9181c55a4380 x1874663043375360/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829949 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829924.897074:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4380 x1874663043375360/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829949 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829924.897094:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829924.897099:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829924.897113:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043375360:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829924.897132:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043375488:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829924.897153:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829924.897163:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829924/real 1787829924] req@ffff9181c55a5880 x1874663043375488/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829949 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829924.897171:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5880 x1874663043375488/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829949 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829924.897176:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829924.897178:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829924.897180:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043375488:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829924.897184:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043375616:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829924.897193:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829924.897199:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829924/real 1787829924] req@ffff9181c55a4700 x1874663043375616/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829949 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829924.897205:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4700 x1874663043375616/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829949 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829924.897210:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829924.897211:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829924.897213:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043375616:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829925.472858:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829926.496899:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829927.520869:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829928.544919:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829929.568905:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829930.016932:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829930.016943:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829930.016948:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829930.016959:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 11996 00000100:00080000:2.0:1787829930.016970:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829930.016978:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787829930.017014:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e300 x1874663043375744/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829930.017047:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829930.017052:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829930.017054:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829930.017059:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11996 00000100:00080000:2.0:1787829930.017063:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829930.017068:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829930.017079:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043375744:192.168.203.147@tcp:250:kworker.0 00000008:00100000:2.0:1787829930.017083:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829930.017103:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d180 x1874663043375872/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829930.017115:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829930.017119:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829930.017122:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829930.017127:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 11996 00000100:00080000:2.0:1787829930.017148:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829930.017153:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829930.017160:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1787829930.017163:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:2.0:1787829930.017188:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ed80 x1874663043376000/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829930.017210:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829930/real 1787829930] req@ffff9181e007e300 x1874663043375744/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829955 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829930.017227:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e300 x1874663043375744/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829955 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829930.017241:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829930.017244:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829930.017262:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043375744:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829930.017294:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043375872:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829930.017320:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829930.017331:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829930/real 1787829930] req@ffff9181e007d180 x1874663043375872/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829955 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829930.017343:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d180 x1874663043375872/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829955 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829930.017351:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829930.017354:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829930.017357:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043375872:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829930.017364:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043376000:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829930.017383:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829930.017391:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829930/real 1787829930] req@ffff9181e007ed80 x1874663043376000/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829955 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829930.017398:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ed80 x1874663043376000/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829955 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829930.017403:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829930.017405:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829930.017407:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043376000:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829930.592960:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829931.616888:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829932.640899:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829933.664975:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829934.689033:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829935.136863:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829935.136872:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829935.136888:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829935.136898:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 12001 00000100:00080000:2.0:1787829935.136907:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829935.136914:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787829935.136947:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d500 x1874663043376128/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829935.136982:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829935.136986:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829935.136988:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829935.136992:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 12001 00000100:00080000:2.0:1787829935.136996:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829935.137000:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829935.137009:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829935.137024:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043376128:192.168.203.147@tcp:250:kworker.0 00000100:00100000:2.0:1787829935.137025:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007df80 x1874663043376256/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829935.137040:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829935.137043:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829935.137045:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829935.137049:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 12001 00000100:00080000:2.0:1787829935.137052:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829935.137056:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829935.137061:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829935.137073:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ed80 x1874663043376384/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829935.137091:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829935.137127:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829935/real 1787829935] req@ffff9181e007d500 x1874663043376128/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829960 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829935.137144:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d500 x1874663043376128/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829960 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829935.137152:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829935.137156:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829935.137168:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043376128:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829935.137189:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043376256:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829935.137211:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829935.137223:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829935/real 1787829935] req@ffff9181e007df80 x1874663043376256/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829960 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829935.137232:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007df80 x1874663043376256/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829960 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829935.137237:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829935.137239:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829935.137241:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043376256:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829935.137251:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043376384:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829935.137265:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829935.137271:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829935/real 1787829935] req@ffff9181e007ed80 x1874663043376384/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829960 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829935.137279:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ed80 x1874663043376384/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829960 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829935.137284:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829935.137286:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829935.137288:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043376384:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829935.712902:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829936.736886:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829937.760910:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829938.784906:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1787829939.744909:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-cliir' failed (-108). 10000000:00000400:1.0:1787829939.744920:0:159735:0:(mgc_request.c:1900:mgc_process_log()) MGC192.168.203.147@tcp: IR log lustre-cliir failed, not fatal: rc = -108 10000000:01000000:1.0:1787829939.744927:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:1.0:1787829939.744929:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff9181c5620000 from 52 10000000:01000000:1.0:1787829939.744933:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:1.0:1787829939.744983:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f96d lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829939.744995:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f96d lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829939.745006:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f96d lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1787829939.745025:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215dc00 going to sleep for 0 seconds 00000100:00100000:1.0:1787829939.745035:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043376512:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787829939.745047:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f96d lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829939.745055:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f96d lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829939.745062:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f96d lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829939.745069:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f96d lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829939.745076:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f96d lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829939.745089:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b5800 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:1.0:1787829939.745093:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f96d lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829939.745101:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f96d lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1787829939.745109:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f96d lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787829939.745114:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1787829939.745117:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787829939.745123:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f96d lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829939.745130:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f96d lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1787829939.745149:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787829939.745152:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:0.0:1787829939.745207:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829939.745213:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829939.745216:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829939.745221:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 12006 00000100:00080000:0.0:1787829939.745229:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829939.745234:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829939.745254:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5c00 x1874663043376640/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829939.745284:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829939.745287:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829939.745289:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829939.745293:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 12006 00000100:00080000:0.0:1787829939.745302:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829939.745308:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829939.745317:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829939.745318:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043376640:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787829939.745334:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5180 x1874663043376768/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829939.745346:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829939.745350:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829939.745352:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829939.745355:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 12006 00000100:00080000:0.0:1787829939.745360:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829939.745364:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829939.745368:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1787829939.745369:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829939.745381:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4e00 x1874663043376896/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829939.745395:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829939/real 1787829939] req@ffff9181c55a5c00 x1874663043376640/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829964 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829939.745408:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5c00 x1874663043376640/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829964 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829939.745416:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829939.745419:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829939.745430:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043376640:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829939.745442:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043376768:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829939.745457:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829939.745465:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829939/real 1787829939] req@ffff9181c55a5180 x1874663043376768/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829964 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829939.745472:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5180 x1874663043376768/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829964 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829939.745481:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829939.745484:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829939.745487:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043376768:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829939.745492:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043376896:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829939.745508:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829939.745519:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829939/real 1787829939] req@ffff9181c55a4e00 x1874663043376896/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829964 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829939.745543:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4e00 x1874663043376896/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829964 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829939.745552:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829939.745555:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829939.745559:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043376896:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829939.808903:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829940.832883:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829941.856882:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829942.880884:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829943.904884:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829944.864902:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829944.864910:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829944.864913:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829944.864921:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 12011 00000100:00080000:0.0:1787829944.864931:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829944.864939:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829944.864990:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7800 x1874663043377024/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829944.865017:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829944.865021:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829944.865029:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829944.865035:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 12011 00000100:00080000:0.0:1787829944.865039:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829944.865045:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829944.865054:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829944.865062:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043377024:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787829944.865070:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5500 x1874663043377152/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829944.865083:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829944.865087:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829944.865089:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829944.865094:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 12011 00000100:00080000:0.0:1787829944.865098:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829944.865102:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829944.865107:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1787829944.865116:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:0.0:1787829944.865121:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4e00 x1874663043377280/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829944.865148:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829944/real 1787829944] req@ffff9181c55a7800 x1874663043377024/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829969 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829944.865160:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7800 x1874663043377024/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829969 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829944.865167:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829944.865169:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829944.865185:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043377024:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829944.865206:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043377152:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829944.865226:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829944.865238:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829944/real 1787829944] req@ffff9181c55a5500 x1874663043377152/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829969 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829944.865250:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5500 x1874663043377152/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829969 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829944.865258:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829944.865262:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829944.865265:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043377152:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829944.865271:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043377280:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829944.865291:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829944.865303:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829944/real 1787829944] req@ffff9181c55a4e00 x1874663043377280/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829969 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829944.865316:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4e00 x1874663043377280/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829969 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829944.865324:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829944.865327:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829944.865330:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043377280:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829944.928877:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829945.952871:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829946.976897:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829948.000872:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829949.024904:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829949.984868:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829949.984875:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829949.984878:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829949.984887:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 12016 00000100:00080000:0.0:1787829949.984895:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829949.984901:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829949.984932:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5180 x1874663043377408/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829949.984953:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829949.984956:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829949.984957:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829949.984960:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 12016 00000100:00080000:0.0:1787829949.984962:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829949.984965:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829949.984972:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829949.984981:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5c00 x1874663043377536/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829949.984988:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829949.984991:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829949.984992:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829949.984995:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 12016 00000100:00080000:0.0:1787829949.984997:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829949.985002:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829949.985003:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043377408:192.168.203.147@tcp:250:kworker.0 00000008:00100000:0.0:1787829949.985005:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829949.985014:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4700 x1874663043377664/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829949.985068:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829949.985096:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829949/real 1787829949] req@ffff9181c55a5180 x1874663043377408/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829974 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829949.985108:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5180 x1874663043377408/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829974 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829949.985116:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829949.985119:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829949.985132:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043377408:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829949.985141:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043377536:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829949.985159:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829949.985171:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829949/real 1787829949] req@ffff9181c55a5c00 x1874663043377536/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829974 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829949.985181:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5c00 x1874663043377536/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829974 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829949.985191:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829949.985193:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829949.985196:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043377536:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829949.985213:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043377664:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829949.985225:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829949.985231:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829949/real 1787829949] req@ffff9181c55a4700 x1874663043377664/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829974 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829949.985239:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4700 x1874663043377664/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829974 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829949.985249:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829949.985251:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829949.985254:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043377664:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829950.048887:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829951.072881:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829952.096965:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829953.120904:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829954.144971:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829955.105178:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829955.105187:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829955.105204:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829955.105214:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 12021 00000100:00080000:2.0:1787829955.105224:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829955.105245:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787829955.105313:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d180 x1874663043377792/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829955.105339:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829955.105343:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829955.105345:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829955.105350:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 12021 00000100:00080000:2.0:1787829955.105353:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829955.105357:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829955.105366:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829955.105379:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e300 x1874663043377920/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829955.105388:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043377792:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787829955.105392:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829955.105397:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829955.105399:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829955.105404:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 12021 00000100:00080000:2.0:1787829955.105407:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829955.105411:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829955.105442:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829955.105469:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007c700 x1874663043378048/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829955.105473:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829955.105505:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829955/real 1787829955] req@ffff9181e007d180 x1874663043377792/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829980 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829955.105524:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d180 x1874663043377792/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829980 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829955.105535:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829955.105539:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829955.105553:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043377792:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829955.105571:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043377920:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829955.105596:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829955.105629:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829955/real 1787829955] req@ffff9181e007e300 x1874663043377920/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829980 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829955.105642:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e300 x1874663043377920/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829980 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829955.105649:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829955.105652:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829955.105655:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043377920:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829955.105662:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043378048:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829955.105680:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829955.105707:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829955/real 1787829955] req@ffff9181e007c700 x1874663043378048/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829980 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829955.105718:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007c700 x1874663043378048/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829980 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829955.105725:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829955.105728:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829955.105731:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043378048:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829955.168901:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829956.192861:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829957.216887:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829958.240878:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829959.264990:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829960.224833:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829960.224842:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829960.224846:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829960.224854:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 12026 00000100:00080000:2.0:1787829960.224863:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829960.224868:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 10000000:01000000:1.0:1787829960.224897:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-client' failed (-108). 00000100:00100000:2.0:1787829960.224903:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ea00 x1874663043378176/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 10000000:00020000:1.0:1787829960.224904:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:1.0:1787829960.224910:0:159735:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:1.0:1787829960.224914:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff9181c5620000 from 7 10000000:01000000:1.0:1787829960.224919:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00000100:00080000:2.0:1787829960.224944:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829960.224948:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829960.224950:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829960.224955:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 12026 00000100:00080000:2.0:1787829960.224958:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00010000:00010000:1.0:1787829960.224960:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f974 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787829960.224962:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829960.224971:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00010000:00010000:1.0:1787829960.224974:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f974 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787829960.224985:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f100 x1874663043378432/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829960.224986:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043378176:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787829960.224990:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f974 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1787829960.225006:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829960.225010:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829960.225011:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829960.225014:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 12026 00000100:00080000:2.0:1787829960.225017:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1787829960.225018:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff91820215dc00 going to sleep for 0 seconds 00000100:00080000:2.0:1787829960.225019:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829960.225025:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829960.225032:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007d880 x1874663043378560/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787829960.225037:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043378304:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1787829960.225053:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f974 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000800:00000100:3.0:1787829960.225059:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00010000:00010000:1.0:1787829960.225066:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f974 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829960.225079:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f974 lrc: 4/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787829960.225093:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829960/real 1787829960] req@ffff9181e007ea00 x1874663043378176/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829985 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00010000:1.0:1787829960.225096:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f974 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829960.225115:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f974 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1787829960.225123:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181e01b5800 already unused, calling callback (ffffffffc1273c20) 00000100:00100000:3.0:1787829960.225124:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ea00 x1874663043378176/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829985 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 10000000:00010000:1.0:1787829960.225128:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f974 lrc: 4/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787829960.225144:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00010000:00010000:1.0:1787829960.225145:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f974 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787829960.225152:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 10000000:00010000:1.0:1787829960.225159:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f974 lrc: 5/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787829960.225160:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043378176:192.168.203.147@tcp:250:kworker.0 10000000:01000000:1.0:1787829960.225171:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:1.0:1787829960.225176:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1787829960.225183:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f974 lrc: 3/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1787829960.225183:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043378432:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829960.225201:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181e01b5800/0xbef42c0221d6f974 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000800:00000100:3.0:1787829960.225209:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829960.225224:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829960/real 1787829960] req@ffff9181e007f100 x1874663043378432/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829985 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 10000000:01000000:1.0:1787829960.225226:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:1.0:1787829960.225231:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00100000:3.0:1787829960.225239:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f100 x1874663043378432/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829985 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829960.225247:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829960.225249:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829960.225251:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043378432:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829960.225257:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043378560:192.168.203.147@tcp:38:kworker.0 00000100:00080000:2.0:1787829960.225269:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829960.225274:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829960.225277:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000800:00000100:3.0:1787829960.225279:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:2.0:1787829960.225285:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 12031 00000100:00100000:3.0:1787829960.225290:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829960/real 1787829960] req@ffff9181e007d880 x1874663043378560/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829985 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1787829960.225291:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829960.225300:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829960.225304:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007d880 x1874663043378560/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829985 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829960.225310:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829960.225313:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829960.225316:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043378560:192.168.203.147@tcp:38:kworker.0 00000100:00100000:2.0:1787829960.225328:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f480 x1874663043378688/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829960.225362:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043378688:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1787829960.225380:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829960.225392:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829960/real 1787829960] req@ffff9181e007f480 x1874663043378688/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829985 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829960.225405:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f480 x1874663043378688/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829985 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829960.225418:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829960.225421:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829960.225432:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043378688:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787829960.288896:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829961.312892:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829962.336905:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829963.360907:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829964.384839:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829965.345286:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829965.345293:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829965.345296:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829965.345304:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 12031 00000100:00080000:2.0:1787829965.345313:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829965.345318:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787829965.345345:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ce00 x1874663043378816/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829965.345369:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829965.345372:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829965.345373:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829965.345376:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 12031 00000100:00080000:2.0:1787829965.345378:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829965.345380:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829965.345395:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829965.345403:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007f800 x1874663043378944/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829965.345409:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829965.345411:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829965.345412:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829965.345415:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 12031 00000100:00080000:2.0:1787829965.345416:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829965.345419:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829965.345421:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829965.345427:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007ca80 x1874663043379072/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829965.345671:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043378816:192.168.203.147@tcp:250:kworker.0 00000800:00000100:3.0:1787829965.345731:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829965.345784:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829965/real 1787829965] req@ffff9181e007ce00 x1874663043378816/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829990 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829965.345795:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ce00 x1874663043378816/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829990 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829965.345802:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829965.345805:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829965.345819:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043378816:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829965.345825:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043378944:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829965.345842:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829965.345850:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829965/real 1787829965] req@ffff9181e007f800 x1874663043378944/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829990 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829965.345861:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007f800 x1874663043378944/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829990 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829965.345868:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829965.345871:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829965.345874:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043378944:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829965.345880:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043379072:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829965.345899:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829965.345909:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829965/real 1787829965] req@ffff9181e007ca80 x1874663043379072/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829990 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829965.345917:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007ca80 x1874663043379072/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829990 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829965.345922:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829965.345924:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829965.345926:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043379072:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829965.408877:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829966.432885:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829967.457095:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1787829968.480869:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829969.505306:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829970.464870:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829970.464878:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829970.464883:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829970.464908:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 12037 00000100:00080000:0.0:1787829970.464918:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829970.464924:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829970.464966:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5880 x1874663043379200/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829970.465005:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829970.465009:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829970.465011:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829970.465015:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 12037 00000100:00080000:0.0:1787829970.465019:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829970.465023:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829970.465030:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829970.465042:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043379200:192.168.203.147@tcp:250:kworker.0 00000100:00100000:0.0:1787829970.465043:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a4380 x1874663043379328/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829970.465073:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829970.465077:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829970.465079:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829970.465084:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 12037 00000100:00080000:0.0:1787829970.465095:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829970.465100:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829970.465109:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829970.465124:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a5f80 x1874663043379456/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787829970.465137:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043379328:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787829970.465174:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043379456:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829970.528916:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829971.553427:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829972.576948:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1787829973.537037:0:2379:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.147:988 00000400:00000100:0.0:1787829973.537087:0:2379:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.203.147@tcp at host 192.168.203.147 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1787829973.537102:0:2379:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1787829973.537145:0:2379:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000800:00000100:0.0:1787829973.537154:0:2379:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.203.147@tcp 00000100:00100000:1.0:1787829973.537785:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829970/real 1787829973] req@ffff9181c55a5f80 x1874663043379456/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829995 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829973.537809:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5f80 x1874663043379456/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829995 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829973.537819:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829973.537827:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829973.537831:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043379456:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787829973.537840:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829970/real 1787829973] req@ffff9181c55a4380 x1874663043379328/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829995 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829973.537851:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a4380 x1874663043379328/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787829995 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829973.537859:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829973.537863:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829973.537865:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043379328:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787829973.537872:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829970/real 1787829973] req@ffff9181c55a5880 x1874663043379200/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829995 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829973.537882:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a5880 x1874663043379200/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787829995 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829973.537889:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829973.537891:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829973.537909:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043379200:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787829973.602302:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829974.624904:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1787829975.585002:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829975.585012:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1787829975.585015:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829975.585024:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 12042 00000100:00080000:0.0:1787829975.585034:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1787829975.585040:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:0.0:1787829975.585069:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a6d80 x1874663043379584/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1787829975.585092:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829975.585096:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1787829975.585098:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829975.585101:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 12042 00000100:00080000:0.0:1787829975.585104:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1787829975.585142:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043379584:192.168.203.147@tcp:250:kworker.0 00000800:00000100:1.0:1787829975.585204:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787829975.585342:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829975/real 1787829975] req@ffff9181c55a6d80 x1874663043379584/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830000 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829975.585359:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a6d80 x1874663043379584/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830000 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1787829975.585372:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00080000:1.0:1787829975.585373:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829975.585380:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000008:00100000:0.0:1787829975.585381:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829975.585398:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7100 x1874663043379712/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787829975.585398:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043379584:192.168.203.147@tcp:250:kworker.0 00000100:00080000:0.0:1787829975.585420:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1787829975.585423:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1787829975.585425:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1787829975.585429:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 12042 00000100:00080000:0.0:1787829975.585434:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1787829975.585436:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043379712:192.168.203.147@tcp:38:kworker.0 00000100:00080000:0.0:1787829975.585441:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:0.0:1787829975.585447:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1787829975.585458:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c55a7b80 x1874663043379840/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1787829975.585468:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787829975.585484:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829975/real 1787829975] req@ffff9181c55a7100 x1874663043379712/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830000 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829975.585496:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7100 x1874663043379712/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830000 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829975.585504:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829975.585507:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829975.585510:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043379712:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787829975.585521:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043379840:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787829975.585554:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787829975.585570:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829975/real 1787829975] req@ffff9181c55a7b80 x1874663043379840/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830000 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829975.585584:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c55a7b80 x1874663043379840/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830000 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829975.585592:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829975.585594:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829975.585597:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c55a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043379840:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829975.649104:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829976.673427:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829977.697088:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829978.720920:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829979.746324:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:2.0:1787829980.704887:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'params' failed (-108). 10000000:00020000:2.0:1787829980.704897:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 10000000:01000000:2.0:1787829980.704902:0:159735:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:2.0:1787829980.704905:0:159735:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff9181c87b19b8 from 1 10000000:01000000:2.0:1787829980.704908:0:159735:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00010000:00010000:2.0:1787829980.704948:0:159735:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1843200/0xbef42c0221d6f97b lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829980.704959:0:159735:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.147@tcp lock: ffff9181d1843200/0xbef42c0221d6f97b lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829980.704970:0:159735:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.147@tcp lock: ffff9181d1843200/0xbef42c0221d6f97b lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1787829980.704988:0:159735:0:(client.c:2715:ptlrpc_set_wait()) set ffff9181d1d6cb40 going to sleep for 0 seconds 00000100:00100000:2.0:1787829980.704998:0:159735:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:f40147eb-9b53-47d4-b77c-99ceedd9664c:159735:1874663043379968:192.168.203.147@tcp:101:ll_cfg_requeue.0 00010000:00010000:2.0:1787829980.705011:0:159735:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: MGC192.168.203.147@tcp lock: ffff9181d1843200/0xbef42c0221d6f97b lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829980.705018:0:159735:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: MGC192.168.203.147@tcp lock: ffff9181d1843200/0xbef42c0221d6f97b lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829980.705028:0:159735:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.147@tcp lock: ffff9181d1843200/0xbef42c0221d6f97b lrc: 4/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829980.705035:0:159735:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.203.147@tcp lock: ffff9181d1843200/0xbef42c0221d6f97b lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829980.705041:0:159735:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.147@tcp lock: ffff9181d1843200/0xbef42c0221d6f97b lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829980.705047:0:159735:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9181d1843200 already unused, calling callback (ffffffffc1273c20) 10000000:00010000:2.0:1787829980.705050:0:159735:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.147@tcp lock: ffff9181d1843200/0xbef42c0221d6f97b lrc: 4/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1206400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829980.705058:0:159735:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.147@tcp lock: ffff9181d1843200/0xbef42c0221d6f97b lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120e400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1787829980.705065:0:159735:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.147@tcp lock: ffff9181d1843200/0xbef42c0221d6f97b lrc: 5/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x120f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1787829980.705071:0:159735:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:2.0:1787829980.705075:0:159735:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1787829980.705081:0:159735:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.147@tcp lock: ffff9181d1843200/0xbef42c0221d6f97b lrc: 3/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1787829980.705087:0:159735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.147@tcp lock: ffff9181d1843200/0xbef42c0221d6f97b lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5a0f400000000 nid: local remote: 0x0 expref: -99 pid: 159735 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1787829980.705100:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829980.705109:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1787829980.705114:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 10000000:01000000:2.0:1787829980.705126:0:159735:0:(mgc_request.c:1817:mgc_process_log()) Can't get cfg lock: -108 00000100:00080000:3.0:1787829980.705126:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 12047 00000100:00080000:2.0:1787829980.705134:0:159735:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:CONNECTING 00000100:00080000:3.0:1787829980.705142:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829980.705152:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:3.0:1787829980.705186:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5634380 x1874663043380096/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787829980.705266:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043380096:192.168.203.147@tcp:250:kworker.0 00000800:00000100:1.0:1787829980.705328:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00080000:3.0:1787829980.705362:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1787829980.705368:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1787829980.705371:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829980.705377:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 12047 00000100:00080000:3.0:1787829980.705382:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829980.705387:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829980.705396:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829980.705407:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5637800 x1874663043380224/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787829980.705486:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829980/real 1787829980] req@ffff9181c5634380 x1874663043380096/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830005 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829980.705499:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5634380 x1874663043380096/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830005 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829980.705521:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829980.705543:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829980.705544:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00080000:3.0:1787829980.705556:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1787829980.705558:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1787829980.705564:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 12047 00000100:00100000:1.0:1787829980.705571:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5634380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043380096:192.168.203.147@tcp:250:kworker.0 00000100:00080000:3.0:1787829980.705574:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1787829980.705584:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:3.0:1787829980.705592:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829980.705598:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043380224:192.168.203.147@tcp:38:kworker.0 00000800:00000100:1.0:1787829980.705627:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829980.705627:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5636a00 x1874663043380352/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787829980.705645:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829980/real 1787829980] req@ffff9181c5637800 x1874663043380224/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830005 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829980.705658:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5637800 x1874663043380224/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830005 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829980.705661:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829980.705668:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829980.705670:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829980.705674:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00080000:3.0:1787829980.705675:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1787829980.705682:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5637800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043380224:192.168.203.147@tcp:38:kworker.0 00000100:00080000:3.0:1787829980.705690:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 12052 00000100:00080000:3.0:1787829980.705697:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1787829980.705701:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043380352:192.168.203.147@tcp:38:kworker.0 00000100:00080000:3.0:1787829980.705709:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000800:00000100:1.0:1787829980.705729:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829980.705732:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c5635c00 x1874663043380480/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1787829980.705741:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829980/real 1787829980] req@ffff9181c5636a00 x1874663043380352/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830005 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829980.705772:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5636a00 x1874663043380352/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830005 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829980.705778:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829980.705781:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829980.705783:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5636a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043380352:192.168.203.147@tcp:38:kworker.0 00000100:00100000:1.0:1787829980.705834:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043380480:192.168.203.147@tcp:250:kworker.0 00000800:00000100:1.0:1787829980.705856:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:1.0:1787829980.705872:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829980/real 1787829980] req@ffff9181c5635c00 x1874663043380480/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830005 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787829980.705889:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c5635c00 x1874663043380480/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830005 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1787829980.705897:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1787829980.705900:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:1.0:1787829980.705916:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c5635c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043380480:192.168.203.147@tcp:250:kworker.0 00010000:00010000:1.0:1787829980.768873:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829981.793060:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829982.816935:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829983.841038:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829984.865596:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829985.824834:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829985.824844:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829985.824849:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829985.824858:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 12052 00000100:00080000:1.0:1787829985.824868:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829985.824874:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787829985.824902:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0e00 x1874663043380608/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829985.824929:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829985.824933:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829985.824935:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829985.824940:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 12052 00000100:00080000:1.0:1787829985.824944:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829985.824949:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829985.824957:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829985.824969:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0000 x1874663043380736/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829985.824978:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829985.824981:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829985.824983:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829985.824987:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 12052 00000100:00080000:1.0:1787829985.824990:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829985.824993:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829985.824998:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1787829985.825004:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043380608:192.168.203.147@tcp:250:kworker.0 00000100:00100000:1.0:1787829985.825006:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1180 x1874663043380864/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829985.825120:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829985.825154:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829985/real 1787829985] req@ffff9181c35b0e00 x1874663043380608/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830010 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829985.825170:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0e00 x1874663043380608/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830010 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829985.825178:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829985.825182:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829985.825198:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043380608:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829985.825205:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043380736:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829985.825226:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829985.825237:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829985/real 1787829985] req@ffff9181c35b0000 x1874663043380736/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830010 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829985.825247:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0000 x1874663043380736/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830010 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829985.825255:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829985.825258:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829985.825261:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043380736:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829985.825282:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043380864:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829985.825302:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829985.825311:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829985/real 1787829985] req@ffff9181c35b1180 x1874663043380864/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830010 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829985.825320:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1180 x1874663043380864/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830010 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829985.825387:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829985.825391:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829985.825394:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043380864:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829985.888902:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829986.912939:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829987.936958:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829988.961039:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829989.985033:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787829990.944923:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829990.944932:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787829990.944935:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829990.944945:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 12057 00000100:00080000:1.0:1787829990.944954:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829990.944960:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787829990.944995:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b3100 x1874663043380992/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829990.945020:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829990.945023:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787829990.945024:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829990.945028:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 12057 00000100:00080000:1.0:1787829990.945031:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787829990.945034:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829990.945041:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829990.945052:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1500 x1874663043381120/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787829990.945062:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787829990.945064:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787829990.945066:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787829990.945069:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 12057 00000100:00080000:1.0:1787829990.945073:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1787829990.945075:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043380992:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787829990.945080:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787829990.945087:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787829990.945097:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0380 x1874663043381248/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829990.945138:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829990.945169:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829990/real 1787829990] req@ffff9181c35b3100 x1874663043380992/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830015 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829990.945185:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b3100 x1874663043380992/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830015 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829990.945194:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829990.945198:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829990.945217:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043380992:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829990.945224:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043381120:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829990.945244:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829990.945255:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829990/real 1787829990] req@ffff9181c35b1500 x1874663043381120/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830015 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829990.945267:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1500 x1874663043381120/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830015 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829990.945274:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829990.945278:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829990.945281:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043381120:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829990.945304:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043381248:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829990.945582:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829990.945594:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829990/real 1787829990] req@ffff9181c35b0380 x1874663043381248/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830015 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829990.945606:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0380 x1874663043381248/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830015 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829990.945613:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829990.945616:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829990.945619:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043381248:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787829991.008951:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829992.032922:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829993.057221:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829994.081530:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829995.104936:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1787829996.064910:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829996.064919:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1787829996.064924:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829996.064936:0:111835:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 12062 00000100:00080000:2.0:1787829996.064949:0:111835:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829996.064958:0:111835:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:2.0:1787829996.065001:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007fb80 x1874663043381376/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1787829996.065023:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829996.065026:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1787829996.065027:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829996.065030:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 12062 00000100:00080000:2.0:1787829996.065033:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829996.065036:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829996.065046:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829996.065054:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007e680 x1874663043381504/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1787829996.065059:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043381376:192.168.203.147@tcp:250:kworker.0 00000100:00080000:2.0:1787829996.065062:0:111835:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1787829996.065068:0:111835:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1787829996.065069:0:111835:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1787829996.065073:0:111835:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 12062 00000100:00080000:2.0:1787829996.065075:0:111835:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1787829996.065078:0:111835:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:2.0:1787829996.065081:0:111835:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1787829996.065089:0:111835:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181e007dc00 x1874663043381632/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787829996.065136:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829996.065173:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829996/real 1787829996] req@ffff9181e007fb80 x1874663043381376/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830021 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829996.065193:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007fb80 x1874663043381376/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830021 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829996.065204:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829996.065208:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829996.065221:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043381376:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787829996.065246:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043381504:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829996.065270:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829996.065283:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829996/real 1787829996] req@ffff9181e007e680 x1874663043381504/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830021 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829996.065297:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007e680 x1874663043381504/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830021 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829996.065306:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829996.065309:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829996.065312:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043381504:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787829996.065317:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043381632:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787829996.065335:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787829996.065345:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787829996/real 1787829996] req@ffff9181e007dc00 x1874663043381632/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830021 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787829996.065357:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181e007dc00 x1874663043381632/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830021 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787829996.065365:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787829996.065368:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787829996.065371:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181e007dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043381632:192.168.203.147@tcp:38:kworker.0 00010000:00010000:1.0:1787829996.128904:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829997.152896:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787829998.176940:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1787829999.200921:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830000.225500:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1787830001.184870:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) f40147eb-9b53-47d4-b77c-99ceedd9664c->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830001.184878:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1787830001.184881:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830001.184891:0:99984:0:(import.c:510:import_select_connection()) MGC192.168.203.147@tcp: connect to NID 192.168.203.147@tcp last attempt 12067 10000000:01000000:2.0:1787830001.184900:0:159735:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.147@tcp: configuration from log 'lustre-sptlrpc' failed (-108). 00000100:00080000:1.0:1787830001.184901:0:99984:0:(import.c:589:import_select_connection()) MGC192.168.203.147@tcp: tried all connections, increasing latency to 20s 10000000:00020000:2.0:1787830001.184908:0:159735:0:(mgc_request.c:614:do_requeue()) failed processing log: -108 00000100:00080000:1.0:1787830001.184912:0:99984:0:(import.c:624:import_select_connection()) MGC192.168.203.147@tcp: import ffff9181c4d28800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000100:00100000:1.0:1787830001.184943:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1c00 x1874663043381760/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830001.184967:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830001.184971:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1787830001.184973:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830001.184977:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 12067 00000100:00080000:1.0:1787830001.184981:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830001.184984:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0000-mdc-ffff9181c5620000: import ffff9181c5bc0000 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787830001.184992:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830001.185005:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b0380 x1874663043381888/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1787830001.185015:0:99984:0:(pinger.c:229:ptlrpc_pinger_process_import()) c0d7758a-4501-4a89-8ec1-407fef4f0c7f->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1787830001.185018:0:99984:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1787830001.185020:0:99984:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1787830001.185024:0:99984:0:(import.c:510:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: connect to NID 192.168.203.147@tcp last attempt 12067 00000100:00100000:3.0:1787830001.185028:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043381760:192.168.203.147@tcp:250:kworker.0 00000100:00080000:1.0:1787830001.185030:0:99984:0:(import.c:589:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1787830001.185037:0:99984:0:(import.c:624:import_select_connection()) lustre-MDT0001-mdc-ffff9181c5620000: import ffff9181c5bc2800 using connection 192.168.203.147@tcp/192.168.203.147@tcp 00000008:00100000:1.0:1787830001.185042:0:99984:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683216 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1787830001.185055:0:99984:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9181c35b1500 x1874663043382016/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1787830001.185077:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787830001.185114:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830001/real 1787830001] req@ffff9181c35b1c00 x1874663043381760/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830026 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830001.185131:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1c00 x1874663043381760/t0(0) o250->MGC192.168.203.147@tcp@192.168.203.147@tcp:26/25 lens 520/544 e 0 to 1 dl 1787830026 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830001.185147:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c4d28800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830001.185151:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of MGS on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787830001.185155:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:f40147eb-9b53-47d4-b77c-99ceedd9664c:2388:1874663043381760:192.168.203.147@tcp:250:kworker.0 00000100:00100000:3.0:1787830001.185162:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043381888:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787830001.185181:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787830001.185189:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830001/real 1787830001] req@ffff9181c35b0380 x1874663043381888/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830026 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830001.185197:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b0380 x1874663043381888/t0(0) o38->lustre-MDT0000-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830026 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830001.185202:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc0000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830001.185204:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787830001.185206:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043381888:192.168.203.147@tcp:38:kworker.0 00000100:00100000:3.0:1787830001.185225:0:2388:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043382016:192.168.203.147@tcp:38:kworker.0 00000800:00000100:3.0:1787830001.185243:0:2388:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.203.147@tcp 00000100:00100000:3.0:1787830001.185253:0:2388:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1787830001/real 1787830001] req@ffff9181c35b1500 x1874663043382016/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830026 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787830001.185265:0:2388:0:(client.c:2534:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9181c35b1500 x1874663043382016/t0(0) o38->lustre-MDT0001-mdc-ffff9181c5620000@192.168.203.147@tcp:12/10 lens 520/544 e 0 to 1 dl 1787830026 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1787830001.185272:0:2388:0:(import.c:65:import_set_state_nolock()) ffff9181c5bc2800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1787830001.185275:0:2388:0:(import.c:1511:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.147@tcp failed (-110) 00000100:00100000:3.0:1787830001.185277:0:2388:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9181c35b1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:c0d7758a-4501-4a89-8ec1-407fef4f0c7f:2388:1874663043382016:192.168.203.147@tcp:38:kworker.0 00010000:00010000:2.0:1787830001.248990:0:159733:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1787830002.273077:0:159732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) Debug log: 75311 lines, 75311 kept, 0 dropped, 0 bad.